search

HarderThenHarder / transformers_tasks

⭐️ NLP Algorithms with transformers lib. Supporting Text-Classification, Text-Generation, Information-Extraction, Text-Matching, RLHF, SFT etc.
https://www.zhihu.com/column/c_1451236880973426688
2.11k stars 376 forks source link

单卡lora微调需要多少显存？ #63

Open CCzzzzzzz opened 1 year ago

CCzzzzzzz commented 1 year ago

超参数如下： --batch_size 1 --max_source_seq_len 250 --max_target_seq_len 150 第一个epoch可以正常微调，到第二个epoch就爆显存，是否存在什么bug？max_source_seq_len和--max_target_seq_len已经设置很多次，到第二个epoch时每次都报相同的错误： OutOfMemoryError: CUDA out of memory. Tried to allocate 128.00 MiB (GPU 0; 23.99 GiB total capacity; 22.95 GiB already allocated; 0 bytes free; 23.00 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

shangzhensen commented 1 year ago

你是微调的原模型，还是量化后的模型

CCzzzzzzz commented 1 year ago

你是微调的原型，还是量化后的模型

原型

starphantom666 commented 1 year ago

能不能出个量化后的模型微调

hsauod commented 1 year ago

同样的问题，第二个epochs，出现out of memory，请问大佬解决了吗？