search

facebookresearch / atlas

Code repository for supporting the paper "Atlas Few-shot Learning with Retrieval Augmented Language Models",(https//arxiv.org/abs/2208.03299)
Other
508 stars 68 forks source link

Running Atlas on small GPU's. #11

Open prasad4fun opened 1 year ago

prasad4fun commented 1 year ago

Hi,

In the blog and paper its mentioned with faiss-pq code size 64 it needs as little as 2GB. I keep getting cuda out of memory with 12 GB gpu while trying to finetune_qa with faiss-pq code 64 and models/atlas_nq/base.

what is the minimum GPU size requirement for running atlas model during finetuning qa and at inference time?

kungfu-eric commented 7 months ago

Something is up with the finetune code. Even 2x40GB with base model and code size 1, GPU mem hits 25GB then tries to allocate 25GB more and OOM. 

  File "/home/amicus/atlas/src/index.py", line 111, in load_index
    self.embeddings = torch.concat(embeddings, dim=1)
RuntimeError: CUDA out of memory. Tried to allocate 22.99 GiB (GPU 1; 47.54 GiB total capacity; 23.00 GiB already allocated; 22.92 GiB free; 23.00 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
DanialPahlavan commented 4 months ago

is it possible to train and test a model using the free version of Google Colab without the need for high-end GPU? I'm a student and I want to train and test this model.