search

brjathu / iTAML

Official implementation of "iTAML : An Incremental Task-Agnostic Meta-learning Approach". CVPR 2020
96 stars 16 forks source link

Fair comparison #5

Closed xialeiliu closed 4 years ago

xialeiliu commented 4 years ago

I got one question, I want to ask if I understood it correctly.

During the task inference, you need to have a continuum assuming they belongs to the same class. It makes comparison with other methods unfair, is it correct? What is your opinion?

salman-h-khan commented 4 years ago

We do not assume that the continuum belongs to the same "class", rather we assume it belongs to a single (unknown) task (which is a collection of classes). Note that in the literature, some methods assume that the task is itself known during inference (e.g., GEM), so we compare it with both categories of models. As shown in Fig. 5, we also compare with both task-agnostic and task-aware meta-learning algorithms for a fair comparison. Despite these comparisons, if you consider methods like RPS, then we do have an additional assumption about data continuum which is not exactly identical to its setting. We mention this in the paper too. Thanks.

xialeiliu commented 4 years ago

Sorry, I mean the same task. When I looked at Figure 6 and Table 2, it sometimes confused me because I thought all methods use the same setting. But I agree with you the proposed setting is interesting and It makes sense. Thanks.