search

salesforce / CodeT5

Home of CodeT5: Open Code LLMs for Code Understanding and Generation
https://arxiv.org/abs/2305.07922
BSD 3-Clause "New" or "Revised" License
2.65k stars 391 forks source link

Failed to evaluate codet5p-2b on multiple GPU cards #132

Open Cxm211 opened 11 months ago

Cxm211 commented 11 months ago

Hi, I tried to evaluate codet5p-2b. I loaded the model from huggingface and I got an error saying CUDA out of memory, then I tried to load the model into multiple GPU cards by adding device_map = 'auto' when load the model. But I got another error: CodeT5pEncoderDecoderModel does not support device_map='auto' yet. The same issue happens when I loaded my own finetuend codet5p-2b models.

yuewang-cuhk commented 11 months ago

Hi there, we've updated the model class and this issue should be fixed now.