Music descriptors and algorithms wishlist (post your suggestions)

[dbogdanov]

This issue summarizes our wishlist of music descriptors and algorithms to be added in Essenita. You can post your suggestions in comments.

Essentia 2.1

• [x] Implement standard mode and finish testing EBU R128 loudness descriptors (#94)
• [x] Finish testing PitchYin (unit tests) (#230)
• [ ] SuperFlux (final clean implementation, review documentation, add unit tests #226)
• [ ] Improved PredominantMelody algorithm (#42, #62)

Long-term future

• [ ] Beat tracker with downbeat detection (#253)
• [ ] Real-time beat tracker
• [ ] Algorithm for music segmentation (+ test current algorithm)
• [ ] More envelope descriptors (attack start/end time, temporal centroid, temporal increase/decrease, ADSR) (@ffont)
• [x] Constant-Q transform (#136)
• [x] Histogram streaming algorithm (map input stream of values into a configurable histogram) (#529)
• [ ] Rhythm Patterns, Rhythm Histogram, and SSD (#532)
• [ ] Beatogram and Meter features (currently marked as experimental algorithms) (#486) (@ffont)
• [x] Graham Percival's BPM [1] estimation method (@ffont)
• [x] Audio fingerprinting (Echoprint, AcoustID) (#474)
• [ ] Algorithms for estimation of time signature, bars, downbeat and offbeat (related to Beatogram)
• [ ] Melody extraction using source filter model (#469) (@juanjobosch)

[1] Graham Percival and George Tzanetakis. Streamlined tempo estimation based on autocorrelation and cross-correlation with pulses. IEEE/ACM Transac- tions on Audio, Speech, and Language Processing, 22(12):1765–1776, 2014.

[andimarafioti]

Hi! I'm working on an 'acoustical instruments detection' algorithm as a final project for my engineering degree. I know Ferdinand Fuhrmann worked a lot in this field for his phD in the Universitat Pompeu Fabra but I don't see any implementations of his work for/in Essentia. I think an implementation of an instrument recognition algorithm using the IRMAS dataset would be a great add-on to show just how complete the library is.

[andimarafioti]

On another subject, it would be cool to see more acoustics algorithms. For example, a toolkit for Room Response Measurement using Angelo Farina's method. This would mean a logsinesweep generator, maybe a convolution algorithm, a kirkeby inverse filter generator and some tools to check the output like the THD, the reverberation time (RT20, RT30), the early decay time. It could be said that these are out of the scope of the library, but this type of information could be useful when trying to retrieve information from a music signal.

[AnasGhrab]

What about implementing the Praat algorithm for Pitch detection (http://www.ucl.ac.uk/~ucjt465/tutorials/praatpitch.html), besides PitchYinFFT and PredominentMelody ?

[dbogdanov]

@AnasGhrab Does Praat algorithm have any advantage over Yin?

[sertansenturk]

I was just going to add a general histogram implementation, but the entry "Histogram streaming algorithm (map input stream of values into a configurable histogram)" seems to include it. Am I right?

[dbogdanov]

@sertansenturk yes, you are I've created a separate issue for that: #529

[dbogdanov]

@perfe

[regorxxx]

Please provide a proper guide to build Essentia + gaia since it's currently impossible on any OS, no matter what you do.

There are literally tenths of issues about this since years.