AntonioCarta
commented
3 years ago Agree on basically everything.
We can still retain the task_label: int field, which would be a shortcut for task_labels[0].
Can we make it fail if len(task_labels) > 0?
11) Add some way to add a validation dataset, which I feel it shouldn't be implemented as a scenario stream. I was thinking about adding a field to Experience.
Isn't it easier to have separate streams? Training and validation streams should have different names (for logging) and may also have other different properties (i.e. set of task labels). (11) is always important, not just for meta-learning. How do you do model selection otherwise?
lrzpellegrini
vlomonaco
Hello everyone, following #293 and #313, I propose the following changes to both steps and the Transformation Dataset:
1) Rename
StepInfotoExperience. 2) InExperience, support multiple task labels by switching fromtask_label: inttotask_labels: List[int], which would contain the task labels contained in the experiences. For this fieldSet[int]may be more appropriate from a formal point of view, but i preferListbecause the user can then obtain the first (usually only) task label using.task_labels[0], which is very simple. We can still retain thetask_label: intfield, which would be a shortcut fortask_labels[0]. 3) RenameTransformationDatasetto eitherMultiTaskDatasetorAvalancheDataset. I prefer the latter asMultiTaskDatasetseems too tied with the idea of "task", which is a term subject to different interpretations in CL. Also consider that this dataset will probably contain all the goodies we are planning to add in the future. 4) Allow theTransformationDatasetto return the task label for each pattern when iterating over it (via the__getitem__method). 5) InTransformationDataset, add a field similar totargetscontaining the task labels for each pattern. I think we shouldn't name ittask_labels, as it would have the same name of theExperiencefield despite the different contents and meaning. 6) Create a field inTransformationDatasetto allow the user to obtain, given a task label, the related subset. 7) Add the+operation for theTransformationDataset, which should return a concatenated dataset. 8) Add the+operation forExperiences, which should return a concatenated Experience (wheretask_labelsand the dataset are properly merged). 9) Finally, add aconcat()method toExperienceslices, which should act exactly as the+operator when applied to multipleExperiences.For future expansions: 10) Add a field to
TransformationDatasetorExperienceto contain custom "task descriptors" (which are different from task labels, which are simpleints). 11) Add some way to add a validation dataset, which I feel it shouldn't be implemented as a scenario stream. I was thinking about adding a field toExperience.I think that 1, 2, 3, 4, 5, 6 should be implemented ASAP. Points 7, 8, 9 are just very useful goodies, so we can implement them after making avalanche public. Points 10 and 11 may be good in order to support more complex scenarios (including meta-learning ones)! Those last two points will definitely require some brainstorming.