https://magenta.tensorflow.org/music-transformer

Bro, have a look at what Google has already done and build on top of it.