Bpm finder

[PladsElsker]

The idea is the following:

I noticed that if you do a dft on a song, you can see that in a lot of cases, the amplitude of high frequencies pulse at the rythm of the song.

This is probably not true for all songs, but there might be a way to find the bpm of a lot of songs by taking the dft of this amplitude.

So basically, take the dft of the song for a specific frequency (or an average of a range of frequency, maybe?) for the whole duration of a song, put the results in an array, and then take the dft of that over time. My guess is that the bpm is gonna be findable in the low frequencies of the result of that dft.

[PladsElsker]

To find the offset, we need to first find the bpm.

Then, we will know for sure that the offset will be within the range of [0, 60/bpm] seconds because of the cyclic nature of the bpm.

We can then try to fit the best sine wave with the dft of the high frequencies we used to find the bpm in the first place, and try to put both waveforms in phase.

[PladsElsker]

We first need to find a way to convert mp3 files into InputStreams. Can't do anything if we can't get the data in the correct format.

Turns out java audio lib can't handle mp3 files.

[PladsElsker]

There might be a way to use the command line version of audacity to first convert the mp3 into a wav file.

Then, we can use the basic implementation of sound files in java to handle the rest.

[PladsElsker]

ok, so one word: ffmpeg

:p

actually, it's currently bugged out and doesn't seem to work 100% of the time, but it kinda works ok for testing purposes (cache file is not deletable, the command line call seems to stall sometimes even when the cached file has been deleted by hand. It won't be okay for any official release if I don't modify the code properly).

The real problem now though is that as it turns out, finding bpm with only sound samples is MUCH harder than anticipated.

I tried to take the fft of the whole song for frequencies between 1 and 5 hz, and this gives horrible results. It's not even bad, just horrible.

We get 195 bpm for a song that's 174 bpm, and 230 bpm for a song that's 201 bpm.

I could do an accuracy analysis with more songs, but it seems like it won't really be necessary...

[PladsElsker]

Therefore, I suggest we put this feature on hold until we see a clearer path.

Else, I will be trying to make this feature for the rest of the year, and I want to make progress for other commands.

So here are some ideas I had to solve this issue. I'm writting them down so I don't forget them next time I pick up this feature.

There is a HUGE database of songs that have correctly labeled bpm by humans. I'm not kidding, thousands of songs of different kind and styles.

All of this on the osu! website.

We have a beatmap parser. We can very easily retrieve bpm info from any recent enough beatmap.

We have an mp3 to wav converter (that needs to be stabilized, but it kinda works for testing purposes as of now), and the java api has wav file handling libraries. So we have song samples.

We have a nice fft library.

Therefore, I believe it's possible to find a really good data representation of any song (as frequency data or something), and train a ML model to find the exact bpm of a song using this representation.

Not easy, I get it, but probably doable.

Anyways, that's the idea. I'll be working on other features for a while I think.

[PladsElsker]

There is now a repo dedicated to the bpm finder because I think it's gonna become big enough for that.

Also, I fixed the current bpm finder (turns out I can't trust myself at 4 am lmao), and it's really cool.

It's obviously not perfect, but when you listen to the bpm it generated in parallel with the song, you kinda get why it selected this specific bpm.

It doesn't seem like it choses the bpm for the best timing, but rather it choses the bpm for the best "fit" with the drum beats and music instruments. It's actually pretty cool to listen to.

[PladsElsker]

if it wasn't clear enough: I closed this issue, not beacause the issue is fixed, but because there is a whole repo dedicated to solving this problem. It's not our potatoes anymore in this repo