search

google-research / l2p

Learning to Prompt (L2P) for Continual Learning @ CVPR22 and DualPrompt: Complementary Prompting for Rehearsal-free Continual Learning @ ECCV22
https://arxiv.org/pdf/2112.08654.pdf
Apache License 2.0
416 stars 42 forks source link

Reproduce issue #13

Closed JH-LEE-KR closed 2 years ago

JH-LEE-KR commented 2 years ago

Dear author.

Thank you for your great work.

I'm having a little problem with reproducing L2P.

First, please modify the environment setup of README.md.

The link for adjust the jax version in README does not support CUDA version. I think the link

https://storage.googleapis.com/jax-releases/jax_cuda_releases.html

seems to have changed to the link above.

The other thing is that even if batch_size is set to 1, I can't run the code completely in my 4 A5000 GPUs because of out of memory issue. Can you also provide small models such as ViT-Tiny or ViT-Small?

I look forward to hearing from you.

Thank you.

KingSpencer commented 2 years ago

Hi,

Thanks for your catch! I have updated the environment setup and the codebase (I am also doing some sanity check now), also please try newer version of jax and flax. The codebase actually supports various sizes of ViT's and you can find corresponding pretrained models here: https://github.com/google-research/vision_transformer.

Best, Zifeng

JH-LEE-KR commented 2 years ago

Thanks for your reply!

I checked your corrections, and many thanks to you for release DualPrompt.

I found in the official document that when JAX executes the first JAX command, it pre-allocates 90% of the available GPU memory. As described in the document, I can either disable the pre-allocation or reduce the pre-allocation ratio to run the ViT-Base model.

Thank you once again for everything you've done.

Best, Jaeho Lee.