systematic comparison of methods

[Laubeee]

I would be interested in a more detailed analysis of how your method compares to pYIN. Would you be willing to run such a systematic comparison on a big dataset, such as the NSynth TESET set, and report on its accuracies? These could then even be further divided into different instrument families and pitches. If it turns out your method is significantly superior for some instrument families or pitch ranges, that would certainly raise interest. Another interesting indicator could be computation time for the benchmark for each method.

[sevagh]

By my method, do you mean PMPM/Probablistic McLeod Pitch Method?

[Laubeee]

yes, as you mention it in the README

[sevagh]

It's a good idea. I think some scripts around the fetching, loading, running pitch tracking, and evaluating the quality of the results on the NSynth data would be a good addition to the repo (and maybe even incorporating modern pitch tracking methods - CREPE is the end of my memory but there are a lot of new ones (like PESTO https://zenodo.org/records/10265343)

[Laubeee]

I think CREPE is still one of the best estimators. There are some recent (SSL) works on general music representations that perform better in some cases when fine-tuned for pitch e.g. MERT, but they also have disadvantages (based on Transformer is likely to result in higher latency for real-time applications) See also this benchmark where NSynth is one of the tasks

[Laubeee]

btw, I could provide python code for evaluating pyin (taken from librosa) on the NSynth-test split, if you are interested. It requires to manually download the data to a local directory though, but that should not be a big issue.

[sevagh]

As long as it's a configurable or well-known path e.g. ~/NSYNTH_DATA, that makes sense to me.