Error during the "Apply after_task" during the training step > 0 for "shift", "noise", and "noisytune"

[Sreeni1204]

Please let me know what to be done here:

I am using the ESC50 dataset and the train.sh file for reproducing the results that you have published. I have not changed any configurations except for batch size.

2023-07-07 19:31:50,057 - INFO - Epoch 7 | Evaluation Accuracy: 1.0
 2023-07-07 19:31:50,057 - INFO - EarlyStopping counter: 7 out of 7.
 2023-07-07 19:31:50,058 - INFO - [2-4] Update the information for the current task
 2023-07-07 19:31:50,058 - INFO - Apply after_task
 2023-07-07 19:31:50,058 - INFO - Update memory over 4 classes by uncertainty
 2023-07-07 19:31:50,058 - INFO - Compute uncertainty by noise!
 2023-07-07 19:32:00,750 - WARNING - Fill the unused slots by breaking the equilibrium.
Traceback (most recent call last):
  File "main.py", line 196, in <module>
    method.after_task(cur_iter)
  File "/home/shikkalven/SELMA/ContinualLearning/ASC_CL_spkid/ASC-CL/methods/base.py", line 108, in after_task
    self.update_memory(cur_iter)
  File "/home/shikkalven/SELMA/ContinualLearning/ASC_CL_spkid/ASC-CL/methods/base.py", line 140, in update_memory
    num_class=num_class,
  File "/home/shikkalven/SELMA/ContinualLearning/ASC_CL_spkid/ASC-CL/methods/base.py", line 379, in uncertainty_sampling
    .sample(n=num_rest_slots)
  File "/home/shikkalven/jupyter-conda-base-environment/conda/envs/CL_asc/lib/python3.7/site-packages/pandas/core/generic.py", line 5365, in sample
    locs = rs.choice(axis_length, size=n, replace=replace, p=weights)
  File "mtrand.pyx", line 903, in numpy.random.mtrand.RandomState.choice
ValueError: a must be greater than 0 unless no samples are taken

[swagshaw]

Hi @Sreeni1204

Please kindly try for "random" and check whether the data path is correct. Seems the uncertainty sampler can not choose any candidates.

[Sreeni1204]

Hello @swagshaw

If you are referring to the memory management, then it works fine with "random".

The following commands are running successfully on the same dataset, but the "uncertainity" is failing as per previous comment.

# Finetune
python main.py --dataset TAU-ASC --mode finetune

# Random
python main.py --dataset TAU-ASC --mode replay --mem_manage random

# Reservoir
python main.py --dataset TAU-ASC --mode replay --mem_manage reservoir

# Prototype
python main.py --dataset TAU-ASC --mode replay --mem_manage prototype

I am using the ESC-50 dataset

[Sreeni1204]

Hello @swagshaw

I read the paper completely again and found this information

To simulate the condition of edge devices, we set the max amount of examples as 500, 100 samples in DCASE 19 Task 1 and ESC-50 due to the memory limitation.

So, I updated the --memory_size to 100.

Let me know if this is correct.

[swagshaw]

Yes. It is correct. The ESC50 has more classes so each class has a smaller memory size. I may forget to point out it in my README.

[Sreeni1204]

Thank you for the information.

Also, if I am training on my custom dataset, how do I calculate the memory size? Is there any algorithm you have used to calculate the memory size for ESC-50 dataset or DCASE dataset?

[swagshaw]

Thank you for the information.

Also, if I am training on my custom dataset, how do I calculate the memory size? Is there any algorithm you have used to calculate the memory size for ESC-50 dataset or DCASE dataset?

Basically, when you are training on your custom dataset, you need to consider the size of your dataset. Briefly talking, the memory size should confirm each class have enough samples in the memory buffer while not above the samples for one single class. If we have a small dataset, 100 samples for 'first', 200 samples for 'second', and the rest 300 samples for 'third', then the memory size could be 10% of the whole dataset, like 60 (each class has 20 samples). But note you better not use memory size more than 100 here, because when you select the candidates for 'first', it will more than it number of samples and may raise the error.

Hope this could help you. :)

[Sreeni1204]

Thanks a lot for the clarification. This helps a lot, will close this issue for now and create a new one in case if I get into some trouble.