Guitar to Multi Hot Piano

[anthonio9]

Instead of using 6* 1440 bins on the output use one vector that will represent all string together. With that in mind use below as well:

• use sigmoid istead of softmax
• use binary cross entropy instead of categorical cross entropy

Work work work!

[anthonio9]

There are multiple ways to achieve this:

• get the current 6 one-hot vectors describing the pitch and put them together into one vector. This approach seems fine, except for the part that every string that has no pitch will get a randomized bin.
• combine all one-hot vectors into one, but randomize the bin only if all strings are mute. A bit harder to achieve with code, maybe will provide better end results?

Let's try both.

But then, how should this be evaluated? The training part is easy, cause the network will progress with the loss function, however real evaluation is a bit heavy. At first all evaluation should be ditched, only the training loss will matter and the plots with logits and the ground truth.

[anthonio9]

This log-sum-exp trick may be helpful for enhancing the loss function with sigmoid.. but first, how to apply sigmoid with binary_cross_entropy so that cuda.amp.GradScaler works fine?

EDIT: it seems that using binary_cross_entropy_with_logits is good enough for replacing sigmoid and bce.

[anthonio9]

1st approach is partially implemented, plotting should not show any ground truth yet, and the only metric tested is the loss coming from the loss function. It's a small step forward!

[anthonio9]

So how should I go about the post-processing? Here's one way, simply by finding up to 6 peaks, cause torch.topk() does not really work well for this application: https://discuss.pytorch.org/t/pytorch-argrelmax-or-c-function/36404/2

[anthonio9]

Finding peaks was a struggle and is not perfect at all. I think now is a good time to abandon this idea.