search

stevenwaterman / musetree

AI Music Generation for the Real World
http://musetree.stevenwaterman.uk/
MIT License
238 stars 33 forks source link

Do not end each note at the end of each section #60

Closed stevenwaterman closed 4 years ago

stevenwaterman commented 4 years ago

Currently MuseTree adds a note end for each note still playing at the end of a section. This resolves the infinite length notes issue but also encourages MuseNet to generate completions that are less coherent. This approach should be reconsidered

farfromsubtle commented 4 years ago

I have noticed this and thought it was a necessary downside to the way musenet generates. If you can override this it would be amazing, but I would imagine that once those segments are fed back to musenet they would also lose coherency since musenet would be seeing what looks like the end of a song as notes trail off?

stevenwaterman commented 4 years ago

In the MuseNet encoding, start and end of a note are two separate tokens. If we start a note but don't end it, MuseNet will decide when to end the note. The problem before was that sometimes MuseNet would just never end the note. I resolved that by always ending all notes at the end of the section, but it would be better to let notes last across sections and have MuseTree end them after a specfied max note length where necessary

farfromsubtle commented 4 years ago

I see, I was thinking in terms of midi not the encoding which is obviously different. Interesting that the end of a note is not determined until later. Max note length makes sense.

stevenwaterman commented 4 years ago

Midis are actually done the same way, they use a NOTE event with a volume - non-zero volume = on, zero volume = off

It's kinda crazy to me, but makes sense for a real-time system. I.e. if i hold a key down a midi keyboard then the sofware needs to know to start playing straight away even without knowing when the note wille nd.

farfromsubtle commented 4 years ago

So I was looking back at a bunch of music generated before the run-on note fix and after. there is definitely a change in the nature of the generated tracks. I would not say it is worse or better but definitely less flow now than there used to be. Where track with the cut off notes tend to have more of a hard rhythm.

farfromsubtle commented 4 years ago

Been analyzing it more and bringing in old projects that I created before the change. The note cut off actually mangles some rhythms pretty reliably. There is also an issue of atrophy as enough cut notes seems to make the model obsessed with short notes the deeper you go and songs can devolve into a mess of tiny slices.

stevenwaterman commented 4 years ago

That makes a lot of sense - rhythyms that previously only had notes of a second or longer will suddenly start having notes that are just a few ms. MuseNet will start generating those itself to be consistent. This definitely needs fixing. Marking as high priority, and it's top of my list for tomorrow

stevenwaterman commented 4 years ago

Fixed 2.15.4 - Let me know if you experience any further issues with this :)