search

google-research / scenic

Scenic: A Jax Library for Computer Vision Research and Beyond
Apache License 2.0
3.32k stars 436 forks source link

Implementation requirements for verbs_in_action? #1072

Open rezaghoddoosian opened 5 months ago

rezaghoddoosian commented 5 months ago

My question is regarding the implementation of this paper "Verbs in Action: Improving verb understanding in video-language models" (verbs_in_action). Im having trouble figuring out the dependencies to run this code. Can this scenic code (verbs_in_action) run with jax for CUDA 11.3? Also, wondering where I can access the requirements for this implementation?

Mas-Ayb commented 5 months ago

It seems that this project has many bugs. For example, the following dependencies are missed: dmvr, clip and official. You can install them using the following code:

pip install git+https://github.com/deepmind/dmvr.git conda install --yes -c pytorch pytorch=1.7.1 torchvision cudatoolkit=11.0 pip install ftfy regex tqdm pip install git+https://github.com/openai/CLIP.git pip install -U -q "tf-models-official"

Still, some other parts of the code need to be improved.

rezaghoddoosian commented 5 months ago

Hi Masoud, Yeah I noticed. I installed some of these. My biggest issue is the system requirements. I have CUDA 11.3, so if I need to install the corresponding jaxlib for my CUDA, I wonder if this code works for CUDA 11.3? Were you able to run this code eventually?

Mas-Ayb commented 5 months ago

Hi Reza, I run the code on Colab. Although after installing dependencies the problem of requirements has been solved, it still has many problems that need to be fixed. (The code is not clean enough and needs so much improvement!) I finally gave up. I hope developers clean up the code and provide some comments.

rezaghoddoosian commented 5 months ago

Hi Masoud, 1-Can you elaborate what you mean by "it has many problems that need to be fixed"? what specific problems? 2- What version of jaxlib and Cuda did you use?

Mas-Ayb commented 5 months ago

Hi Reza,

I suggest you first run the code on Colab. When you use Colab you have no problem with versions of jaxlib and you could be sure that the code is clean. When you are sure about the code itself, then you can run it locally. I did not run the code on a local machine, so I cannot tell you about versions difference.