F1 score mismatch

[swapb94]

python train_sedt.py --gpus $ngpu --dataname urbansed --batch_size 64 --fusion_strategy 1 --dec_at --weak_loss_coef 1 --epochs 400 # total epochs --epochs_ls 280 # epochs of learning stage --lr_drop 160 --num_queries 10

The above command provided as a part of the Readme yields a model with a below scores on the eval set, Eb_F1 Eb_P Eb_R Sb_F Sb_P Sb_R At_F1 0 27.12% 31.75% 24.00% 60.21% 69.49% 53.77% 71.37% However, the pre-trained model provided in the same readme, i.e. SEDT(E=3, Eb_F1=38.15) yields the below scores on the eval set, Eb_F1 Eb_P Eb_R Sb_F Sb_P Sb_R At_F1 0 38.13% 46.06% 33.09% 64.12% 76.33% 56.08% 73.02%

Could you please help me understand why is there a difference between the scores of these two models? Also, what are the params used for training the provided pre-trained model?

[Anaesthesiaye]

Sorry for late reply, could you check if there are overlapped events with the same class label in your training data? You can look the tsv files in ./data/URBAN-SED_v2.0.0/metadata/ for reference where overlapped events with same class labels have been merged as one.

[swapb94]

Do you mean like this, I used the command in the readme for generating these files, python ./data_utils/collapse_event.py

Attaching, the train.tsv below train.zip

[swapb94]

I think my query wasn't clear. Please note, I am using the same split provided in your repository. I am using trying to reproduce your results on the URBAN-SED_v2.0 dataset with your commands given in the readme itself, but I'm getting f1 score of 27% whereas your pre-trained model yields 38%

[wangshaoan-cn]

Hello my friend,I am reading your paper about [sound_event_detection_transformer] ,but there are some questions plaguing me and I hope you can help me and I am very interested in this code you writed ,my email is wzg001206@outlook.com. I am looking forward to Establishing contact with you. Thank you very much.