ContinualAI / avalanche

Avalanche: an End-to-End Library for Continual Learning based on PyTorch.
http://avalanche.continualai.org
MIT License
1.79k stars 291 forks source link

Benchmarks Zero to Hero tutorial produces unexpected output #869

Closed niniack closed 2 years ago

niniack commented 2 years ago

🐛 Describe the bug

The Zero to Hero Tutorial on Benchmarks has a line of code under the Scenarios sub-header that produces an unexpected output

🐜 To Reproduce Below is the original code, highlighting only the problematic section

from avalanche.benchmarks.classic import SplitMNIST
split_mnist = SplitMNIST(n_experiences=5, seed=1)

# A list that, for each experience (identified by its index/ID),
# stores a set of the (optionally remapped) IDs of classes of patterns
# assigned to that experience.
print('--- Classes in each experience:')
split_mnist.classes_in_experience

🐝 Expected behavior This tutorial should be printing a list of classes in each experience.

Now, if you change it to (wrap the print command)

print('--- Classes in each experience:')
print(split_mnist.classes_in_experience)

It still does not work as expected.

After doing a little bit of digging, it seems that

print('--- Classes in each experience:')
print(split_mnist.original_classes_in_exp)

OR

print('--- Classes in each experience:')
print(split_mnist._classes_in_exp)

seems to output more human-readable information. However, it seems that the latter is internal information that is unexpected to be part of the API given the variable naming.

On first glance, it seems to be a good first issue. I am happy to contribute if I could kindly get a hint on what the preferred way of fixing this is (e.g. fix tutorial? or fix source code?)

🐞 Screenshots N/A

🦋 Additional context This was run in a Colab notebook. Avalanche was installed with the following line of code

!pip install git+https://github.com/ContinualAI/avalanche.git
AntonioCarta commented 2 years ago

Thanks! I believe for the moment the best solution would be to improve the tutorial's output without changing the class internals since they may change for other reasons (the less we expose to beginner users, the better). Feel free to do it how you prefer it as long as it is understandable by a beginner.

You can change the benchmarks notebook, which will be automatically uploaded on the website.