Open AlbinSou opened 2 years ago
I agree about the nc_first_task
option, we should also have it for multi-task scenarios.
Your snippets seems wrong. Instead of modifying the experiences in place, it's easier to create a new benchmark by first concatenating/splitting the datasets however you like, and the using one of the generic builders, like dataset_benchmark
.
If you still get an error using dataset_benchmark
, feel free to open a question on the Discussions.
I agree about the
nc_first_task
option, we should also have it for multi-task scenarios.Your snippets seems wrong. Instead of modifying the experiences in place, it's easier to create a new benchmark by first concatenating/splitting the datasets however you like, and the using one of the generic builders, like
dataset_benchmark
.If you still get an error using
dataset_benchmark
, feel free to open a question on the Discussions.
Yes, I agree that this is an ugly fix, I also tried like you did but the batch size is still multiplied by the number of tasks in the first experience. I think this comes from TaskBalancedDataLoader, but I don't know if it's intended that the batch size is increased that way.
Ok, now I get it. Yes, it's normal, some of the dataloaders, like the TaskBalancedDataLoader
add batch_size
samples for each group (task/experience ...). Maybe we should rename the parameter to avoid confusion.
For the moment, the nc_benchmark generator function allows for a nc_first_task option, which is good for pre-training in the class-incremental learning scenario. However, the same kind of option is not available if one wants to pretrain in the task-incremental scenario. It would be nice to have an option that could be used together with task_labels=True and allows for pretraining on multiple tasks at the same time, in a multitask training manner.
This kind of pre-training is used for instance in Lifelong Learning of Compositional Structures
A quick fix that I'm using for now, but that is breaking some things (maybe to be put in bugs?) is the following:
Doing this works as intended except that it multiplies the batch_size by the number of pretraining tasks for some reason: