search

tdemin16 / Continual-LayerNorm-Tuning

Official Repository of "On the Effectiveness of LayerNorm Tuning for Continual Learning in Vision Transformers" (Visual Continual Learning Workshop ICCV 2023)
Apache License 2.0
8 stars 0 forks source link

Task Incremental Scores? #2

Closed Susmit-A closed 6 months ago

Susmit-A commented 6 months ago

Hi,

I enjoyed going through your work. I would appreciate it if you could provide your model's scores in a TIL setting or let me know how to get them using your code.

Thanks, Susmit

tdemin16 commented 6 months ago

Hi, thank you for taking an interest in our work. Unfortunately, we did not compute the TIL scores. However, with little code, you should be able to compute them. In practice, in TIL, the task id is known at inference time, so you could provide the vector of IDs as input to the ViT forward by passing it to the task_selection argument and leaving the original features=None. Then, you should also mask the outputs. To do so, you should call a modified version of utils.mask_logits(...) (that you must implement), where given a vector of task IDs, it outputs a mask for each sample.

I hope this can help you with your problem. Thomas

Snimm commented 6 months ago

Hi, Regarding the WeightSelectionMechanism class, I've noticed that changing the forward method from sim, tasks = torch.max(sim, dim=-1) to tasks = torch.full((B, ), task_eval) yields the same accuracy as the oracle reported in your paper. Could you please clarify if this modification is valid for getting task incremental scores?

tdemin16 commented 6 months ago

Yes, you are correct. However you must also mask output logits to retrieve only those for the correct task. If you do both things you should be ok.