search

JH-LEE-KR / l2p-pytorch

PyTorch Implementation of Learning to Prompt (L2P) for Continual Learning @ CVPR22
Apache License 2.0
176 stars 22 forks source link

Diversifying prompt-selection #3

Open natureyoo opened 1 year ago

natureyoo commented 1 year ago

Thank you very much your pytorch implementation for L2P! I have a question about prompt selection. In the paper, they use prompt frequency based weight to select diverse prompt, but i think i can't find that part in the code. I think I can't find that part not only in your code but also in the official jax-based code, so could you let me know if there's anything I'm missing?

Thank you very much for your work!!!

pmy0792 commented 1 year ago

https://github.com/google-research/l2p/issues/9

JH-LEE-KR commented 1 year ago

Hi, thanks for your comment.

You're right. In the paper, they use frequency-based weight to select a diverse prompt. I also could not find the explicit part in the official jax code, and could not implement.

According to a recent author's explanation (linked by @pmy0792 above), they implemented the method simply and effectively using config.use_prompt_mask.

Fortunately use_prompt_mask is already implemented in my code, and I think you can change this option to True and apply a method that selects a diverse prompt.

However, they said in paper that the method was applied only to 5-datasets, not Split-CIFAR100. I recently implemented the PyTorch code for 5-datasets, and I updated it, so check it.

If you have any additional comments, please feel free to let me know.

Best, Jaeho Lee.