search

caikit / caikit-nlp

Apache License 2.0
12 stars 39 forks source link

Add support for LoRA in prompt tuning module (Investigate) #136

Open gkumbhat opened 11 months ago

gkumbhat commented 11 months ago

Is your feature request related to a problem? Please describe.

Currently we only support vanilla prompt tuning and multi-task prompt tuning from PEFT library. LoRA has shown to give good performance on variety of datasets. This story is to investigate support for LoRA into the module.

chakrn commented 10 months ago

Should this be added to the 'Add Support for LoRA' milestone? cc: @rawkintrevo

ibm-peach-fish commented 9 months ago

@chakrn / @gkumbhat I think we can close this as "no" i did the second iteration of adding LoRA in Prompt Tuning and it was pushed back.

chakrn commented 8 months ago

cc: @Ssukriti

Ssukriti commented 8 months ago

doing this in steps, but first PR ready for review https://github.com/caikit/caikit-nlp/pull/270 @gkumbhat