using qlora: error:"CodeT5pEncoderDecoderModel does not support gradient checkpointing"

salesforce / CodeT5

Home of CodeT5: Open Code LLMs for Code Understanding and Generation

https://arxiv.org/abs/2305.07922

BSD 3-Clause "New" or "Revised" License

2.74k stars 401 forks source link

using qlora: error:"CodeT5pEncoderDecoderModel does not support gradient checkpointing" #169

Open hot-zhy opened 5 months ago

hot-zhy commented 5 months ago

Hello! I use qlora to fine-tune codeT5p, but it errors: CodeT5pEncoderDecoderModel does not support gradient checkpoint.

Then I remove the parameter in function train: e0add5c3ba74a166c2957f6fd881c01 but it doesn't work and it still errors: 1dd6dc9ecebe31875c03b00bdc43067 What should i do? Can codeT5 use qlora to do SFT?