search

caikit / caikit-nlp

Apache License 2.0
12 stars 45 forks source link

Replace training loop in peft_prompt_tuning with HF Trainer #175

Open gkumbhat opened 1 year ago

gkumbhat commented 1 year ago

Description

Currently we use a custom training loop along with HF accelerator for training. This story is to replace that with HF Trainer similar to what we implemented for fine-tuning.

Tasks

  1. Implement HF Trainer in peft prompt tuning module
  2. Make sure all unit tests pass and add any unit tests necessary
  3. Evaluate newly implemented training loop with RTE and CNN datasets using:
    1. Bloom-560m
    2. t5-base

Acceptance Criteria

chakrn commented 10 months ago

Moving to In Review (see linked PR)

gkumbhat commented 9 months ago

Testing done for this PR merge: https://github.com/caikit/caikit-nlp/pull/203#issuecomment-1809075757