Meet errors when using Naive method on generated benchmark

[YuanXun2024]

I generated a benchmark with "class_incremental_benchmark". But 'Naive' strategy cannot be applied to this benchmark. Can you pls give me some advice?

Code as follows:

datadir = default_dataset_location('mnist') train_MNIST = as_classification_dataset(MNIST(datadir, train=True, download=True)) test_MNIST = as_classification_dataset(MNIST(datadir, train=False, download=True)) scenario = class_incremental_benchmark({'train': train_MNIST, 'test': test_MNIST}, num_experiences=5)

n_classes = 10 model = SimpleMLP(num_classes=n_classes)

cl_strategy = Naive( model, SGD(model.parameters(), lr=0.001, momentum=0.9), CrossEntropyLoss(), train_mb_size=500, train_epochs=1, eval_mb_size=100, )

print('Starting experiment...') results = [] for experience in scenario.train_stream: print("Start of experience: ", experience.current_experience) print("Current Classes: ", experience.classes_in_this_experience)

# train returns a dictionary which contains all the metric values
print(experience)
res = cl_strategy.train(experience)
print('Training completed')

print('Computing accuracy on the whole test set')
# test also returns a dictionary which contains all the metric values
results.append(cl_strategy.eval(scenario.test_stream))

error as follows: Traceback (most recent call last): File "main.py", line 71, in res = cl_strategy.train(experience) File "/home/yuanxun/CL/avalanche/avalanche/training/templates/base_sgd.py", line 211, in train super().train(experiences, eval_streams, kwargs) File "/home/yuanxun/CL/avalanche/avalanche/training/templates/base.py", line 162, in train self._before_training_exp(kwargs) File "/home/yuanxun/CL/avalanche/avalanche/training/templates/base_sgd.py", line 291, in _before_training_exp self.make_train_dataloader(**kwargs) File "/home/yuanxun/CL/avalanche/avalanche/training/templates/base_sgd.py", line 456, in make_train_dataloader self.dataloader = TaskBalancedDataLoader( File "/home/yuanxun/CL/avalanche/avalanche/benchmarks/utils/data_loader.py", line 404, in init task_labels_field = getattr(data, "targets_task_labels") AttributeError: 'ClassificationDataset' object has no attribute 'targets_task_labels'

[YuanXun2024]

Same error with "new_instances_benchmark". I guess Naive method can only work with "nc_benchmark". Am I correct?

datadir = default_dataset_location('mnist') train_MNIST = MNIST(datadir, train=True, download=True) test_MNIST = MNIST(datadir, train=False, download=True)

train_transforms = torchvision.transforms.Compose([ torchvision.transforms.ToTensor(), transforms.Normalize((0.1307,), (0.3081,)) ]) eval_transforms = torchvision.transforms.Compose([ torchvision.transforms.ToTensor(), transforms.Normalize((0.1307,), (0.3081,)) ]) train_MNIST = as_classification_dataset( train_MNIST, transform_groups={ 'train': train_transforms, 'eval': eval_transforms } ) test_MNIST = as_classification_dataset( test_MNIST, transform_groups={ 'train': train_transforms, 'eval': eval_transforms } )

scenario_val = new_instances_benchmark( train_MNIST, test_MNIST, balance_experiences=True, shuffle=True, num_experiences=5, seed=0, )

n_classes = 10 model = SimpleMLP(num_classes=n_classes)

cl_strategy = Naive( model, SGD(model.parameters(), lr=0.001, momentum=0.9), CrossEntropyLoss(), train_mb_size=500, train_epochs=1, eval_mb_size=100, )

print('Starting experiment...') results = [] for experience in scenario_val.train_stream: print("Start of experience: ", experience.current_experience)

res = cl_strategy.train(experience)
print('Training completed')

print('Computing accuracy on the whole test set')
results.append(cl_strategy.eval(scenario_val.test_stream))

[wang-xulong]

Yes, I came cross the similiar issues, "class_incremental_benchmark" and "task_incremental_benchmark" only were presented in demo example. when taking them into train strategy, you may meet many errors