search

YuanGongND / ast

Code for the Interspeech 2021 paper "AST: Audio Spectrogram Transformer".
BSD 3-Clause "New" or "Revised" License
1.06k stars 202 forks source link

The validation loss seems too high #75

Closed haoheliu closed 1 year ago

haoheliu commented 1 year ago

Thanks for making such a wonderful repo.

When I run the model, the validation loss is very high(around 0.6940), but the mAP is keep increasing normally. Could you please explain a little bit about this?

Many thanks,

haoheliu commented 1 year ago

The same phenomenon in your log file: https://github.com/YuanGongND/ast/blob/master/egs/audioset/exp/test-full-f10-t10-pTrue-b12-lr1e-5/log_2090852.txt

YuanGongND commented 1 year ago

Hi Haohe,

Thanks for pointing this out.

I somehow know both the training and validation loss have some problems in the code. For the validation loss, one obvious reason is that we applied a Sigmoid before calculating the loss https://github.com/YuanGongND/ast/blob/d7d8b4b8e06cdaeb6c843cdb38794c1c7692234c/src/traintest.py#L280 while the nn.BCEWithLogitsLoss() expects raw logits. I cannot recall why I added the Sigmoid, but it shouldn't impact the mAP.

-Yuan

haoheliu commented 1 year ago

Many thanks for the explanation!

Haohe

YuanGongND commented 1 year ago

I apologize for the bug, I just don't have time to fix and test it, also I want to keep it consistent with my log.

More information about training and validation loss is here: https://github.com/YuanGongND/ast/issues/31.

-Yuan