search

fgnt / pb_sed

Paderborn Sound Event Detection
MIT License
68 stars 10 forks source link

Lazy_Dataset not up to date #12

Closed niclas-b closed 1 year ago

niclas-b commented 1 year ago

Hello,

I got following error, trying to start a training:

Traceback (most recent calls WITHOUT Sacred internals): File "/workspaces/pb_sed/pb_sed/experiments/strong_label_crnn/training.py", line 355, in train trainer.test_run(train_set, validate_set) File "/root/.local/lib/python3.9/site-packages/padertorch/train/trainer.py", line 190, in test_run test_run( File "/root/.local/lib/python3.9/site-packages/padertorch/train/runtime_tests.py", line 201, in test_run sub_train_iterator = list(itertools.islice(train_iterator, 2 * virtual_minibatch_size)) File "/root/.local/lib/python3.9/site-packages/lazy_dataset/core.py", line 1725, in __iter__ yield from map(self.map_function, self.input_dataset) File "/root/.local/lib/python3.9/site-packages/lazy_dataset/core.py", line 2071, in __iter__ yield from self._single_thread_prefetch() File "/root/.local/lib/python3.9/site-packages/lazy_dataset/parallel_utils.py", line 301, in single_thread_prefetch raise exc_info[1].with_traceback(exc_info[2]) File "/root/.local/lib/python3.9/site-packages/lazy_dataset/parallel_utils.py", line 263, in worker for item in generator: File "/root/.local/lib/python3.9/site-packages/lazy_dataset/core.py", line 1725, in __iter__ yield from map(self.map_function, self.input_dataset) File "/root/.local/lib/python3.9/site-packages/lazy_dataset/core.py", line 3355, in __iter__ bucket = self.bucket_cls(example, **self.bucket_kwargs) File "/root/.local/lib/python3.9/site-packages/padertorch/contrib/je/data/utils.py", line 26, in __init__ super().__init__(init_example, **kwargs) TypeError: __init__() got an unexpected keyword argument 'max_buffered_examples'

I saw, that this variable max_buffered_examples exists in the current master branch of the lazy_dataset, but not in the tagged version 0.0.14 which is downloaded by following the installing instructions for the pb_sed repo. Is this a mistake that can be fixed by applying a new tag to the changes you made to lazy_datasets two month ago by adding/renaming the max_buffered_examples variable?

Best Regards

JanekEbb commented 1 year ago

Hi, actually the tag lazy_dataset.git@dc9f487bd433a9ccc8e157d58e338074e3cd8705 should already include the renaming. It could be that sed_scores_eval installation overwrites it by installing lazy_dataset from pypi, which is an older version I guess. Therefore I updated the readme so that lazy_dataset should be installed after sed_scores_eval. Simply rerun pip install --user git+https://github.com/fgnt/lazy_dataset.git@dc9f487bd433a9ccc8e157d58e338074e3cd8705and you should be good.