search

yinkalario / EIN-SELD

An Improved Event-Independent Network for Polyphonic Sound Event Localization and Detection
66 stars 15 forks source link

train the model #9

Closed catherine-qian closed 2 years ago

catherine-qian commented 3 years ago

Dear author,

I would like to reproduce the results. Thus, I have re-trained the model with the provided source code with the following steps: (1) python main.py -c XXX train The output is "valid: ER20: 0.436, F20: 0.660, LE20: 12.390, LR20: 0.762, seld20: 0.271, ER19: 0.348, F19: 0.763, LE19: 10.058, LR19: 0.768, seld19: 0.218" then (2) python main.py -c XXX infer (3) python main.py -c XXX evaluate The output is "test: ER20: 0.373, F20: 0.704, LE20: 13.204, LR20: 0.816, seld20: 0.232, ER19: 0.272, F19: 0.818, LE19: 10.680, LR19: 0.794, seld19: 0.180, "

Both the test and validation results are diferent from the original paper. Could you please let me know is there any mistakes I made?

Thanks a lot for your help!

yinkalario commented 3 years ago

So the performance results for the test set are using all of the development data. Can you check if you used all of the fold from 1-6? If you only used fold 2-6 of the development data, performance will be different. Thanks.

wngh1187 commented 2 years ago

Hello. I have a question related to the above question. How to validate if all folds 1 to 6 are used for training? And if "valid_fold" is set to "None" in "seld.yaml", the following error occurs.

File "seld/main.py", line 59, in sys.exit(main(args, cfg)) File "seld/main.py", line 42, in main train_initializer = initialize.init_train(args, cfg, dataset) File "/code/EIN-SELD/seld/learning/initialize.py", line 70, in init_train valid_set, validgenerator, = get_generator(args, cfg, dataset, generator_type='valid') File "/code/EIN-SELD/seld/utils/config.py", line 83, in get_generator subset = method_dict[cfg['method']].data.UserDataset(args, cfg, dataset, dataset_type='valid') File "/code/EIN-SELD/seld/methods/ein_seld/data.py", line 71, in init valid_fold = [int(fold.strip()) for fold in str(cfg['training']['valid_fold']).split(',')] File "/code/EIN-SELD/seld/methods/ein_seld/data.py", line 71, in valid_fold = [int(fold.strip()) for fold in str(cfg['training']['valid_fold']).split(',')] ValueError: invalid literal for int() with base 10: 'None'

yinkalario commented 2 years ago

I think you probably need to modify the code, or simply put valid_fold to be "eval", which uses the evaluation set to validate the performance. This is literally not the right training process. So you can directly remove the validation part and do the training. Thanks.