search

analogdevicesinc / ai8x-training

Model Training for ADI's MAX78000 and MAX78002 Edge AI Devices
Apache License 2.0
86 stars 76 forks source link

KeyError: 'shift' while trying to implement kws20 with the dataset KWS_35_unquantized #316

Closed supportman007 closed 1 month ago

supportman007 commented 1 month ago

Im trying to implement a kws20 demo with kws 35 dataset. If i use the kws 20 dataset it compiles and runs great. When i try the kws 35 dataset i get this error:

Log file for this run: /kws20/test/ai8x-training/logs/2024.06.02-122023/2024.06.02-122023.log Traceback (most recent call last): File "train.py", line 1901, in main() File "train.py", line 447, in main trainloader, val_loader, test_loader, = apputils.getdataloaders( File "/kws20/ai8x-training/distiller/distiller/apputils/data_loaders.py", line 294, in get_data_loaders train_dataset, test_dataset = datasets_fn(data_dir, load_train=not test_only, load_test=True) File "/kws20/test/ai8x-training/datasets/kws20.py", line 824, in KWS_35_get_unquantized_datasets return KWS_get_unquantized_datasets(data, load_train, load_test, num_classes=35) File "/kws20/test/ai8x-training/datasets/kws20.py", line 799, in KWS_get_unquantized_datasets train_dataset = KWS(root=data_dir, classes=classes, d_type='train', File "/kws20/test/ai8x-training/datasets/kws20.py", line 95, in __init self.download() File "/kws20/test/ai8x-training/datasets/kws20.py", line 185, in download self.gen_datasets() File "/kws20/test/ai8x-training/datasets/kws20.py", line 642, in gen_datasets self.speed_augment_multiple(record, fs, exp_len, no_augmentations) File "/kws20/test/ai8x-training/datasets/kws20.py", line 535, in speed_augment_multiple voice_begin_idx, voice_end_idx = self.get_audio_endpoints(audio, fs) File "/kws20/test/ai8x-training/datasets/kws20.py", line 512, in get_audio_endpoints return int(-self.augmentation['shift']['min'] * fs), \ KeyError: 'shift'

can someone help pls.

rotx-maxim commented 1 month ago

@supportman007, we are able to reproduce the problem. We are working on a solution.

supportman007 commented 1 month ago

@rotx-maxim if i create a new dataset in the code with 35 keywords and use the kws_loader instead of kws_35 loader. Will it make a difference? According to this: link

alicangok commented 1 month ago

Hi @supportman007,

Could you try this kws_unquantized.patch and let us know if it's working?

Please note that you will need to delete the old unquantized.pt file in your data directory, in order to generate a new unquantized dataset file that's compatible with the latest changes.

supportman007 commented 1 month ago

I tried this fix. It solved the error in the code thanks. But when i try to implement it outputs "Killed" after the data concatenation. What will be the problem?

alicangok commented 1 month ago

Could you post the actual logs?

Has the unquantized.pt file been generated successfully in your data directory?

On Thu, Jun 6, 2024, 23:38 supportman007 @.***> wrote:

I tried this fix. It solved the error in the code thanks. But when i try to implement it outputs "Killed" after the data concatenation. What will be the problem?

— Reply to this email directly, view it on GitHub https://github.com/analogdevicesinc/ai8x-training/issues/316#issuecomment-2153367035, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGIVU57TLAZDJYAKU2CBKRTZGDCDNAVCNFSM6AAAAABIU45XVCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNJTGM3DOMBTGU . You are receiving this because you were assigned.Message ID: @.***>