search

caikit / caikit-nlp

Apache License 2.0
12 stars 49 forks source link

Lora config data model and utilities #270

Open Ssukriti opened 1 year ago

Ssukriti commented 1 year ago

For #https://github.com/caikit/caikit-nlp/issues/136

This PR is the first step to adding a data model and utility to initialize LoraConfig . It adds support for get_peft_config to return LoraConfig or PEFTTuningConfig

It also includes a refactor to

  1. move common code needed for peft and Lora config initialization to the peft_config file .
  2. create modular functions
  3. move out parts specific to pefT Config to a separate private function ultimately called from within get_peft_config based on tuning type