search

multitel-ai / urban-sound-tagging

1st place solution to the DCASE 2020 - Task 5 - Urban Sound Tagging with Spatiotemporal Context
MIT License
16 stars 2 forks source link

Missing hyper param column #2

Closed jsaksris closed 3 years ago

jsaksris commented 4 years ago

I'm trying tp train the system after manually download the dataset(automatic download keep failing).

I have ran dataprep with no issue, but training is not working. And I cannot find any file related to these meta data.

Thanks for your help

below is the error msg:

python training_system1.py --gpu 1 --seed 1

training_system1.py:57: DtypeWarning: Columns (40,54,59) have mixed types.Specify dtype option on import or set low_memory=False. self.prepare_data() Traceback (most recent call last): File "training_system1.py", line 282, in main(hparams) File "training_system1.py", line 250, in main model = DCASETALNetClassifier(hparams) File "training_system1.py", line 58, in init num_meta = len(self.dataset[0]['metadata']) File "/storage/dcase2020/prepare_data/sonycust.py", line 340, in getitem metadata = np.array(self.annotation_df[self.metadata_labels].iloc[index]) File "/usr/local/lib/python3.6/dist-packages/pandas/core/frame.py", line 2806, in getitem indexer = self.loc._get_listlike_indexer(key, axis=1, raise_missing=True)[1] File "/usr/local/lib/python3.6/dist-packages/pandas/core/indexing.py", line 1553, in _get_listlike_indexer keyarr, indexer, o._get_axis_number(axis), raise_missing=raise_missing File "/usr/local/lib/python3.6/dist-packages/pandas/core/indexing.py", line 1640, in _validate_read_indexer raise KeyError(f"None of [{key}] are in the [{axis_name}]") KeyError: "None of [Index(['latitude', 'longitude', 'week', 'day', 'hour'], dtype='object')] are in the [columns]"

TheZothen commented 4 years ago

Hi,

First, thank you for your interest in our work. It seems you have the wrong annotation.csv, I corrected some files (sonycust.py and config.py) and added more indications on the Readme so it should be easier for you to download and prepare the dataset. By running python data_prep.py --download --mel once more, everything should be working now. Let me know if you still have an error.

jsaksris commented 4 years ago

It works. Thank you.

Un related question, is there a way to train the system in multiple session? Say 50 epoch learning, and save, the restart from the 51st epoch.

TheZothen commented 4 years ago

Sure, with Pytorch Lightning it should be easy. You have to tweak the Trainer object, here is some code to guide you in the right direction. Train for 50 epoch :

model=TheModel()
trainer = pl.Trainer(max_epochs=50)
trainer.fit(model)

Then you can restart training :

trainer.max_epochs = 100
trainer.fit(model)

For the save/load part, you have to use the ModelCheckpoint callback and load_from_checkpoint(path_to_the_weights).