search

neulab / prompt2model

prompt2model - Generate Deployable Models from Natural Language Instructions
Apache License 2.0
1.96k stars 177 forks source link

Add mistral fine-tuning and examples #395

Closed saum7800 closed 8 months ago

saum7800 commented 8 months ago

Description

This PR contains 3 changes primarily:

  1. Add a QLora trainer that has been tested to qlora fine-tune a Mistral 7B model on created datasets, It is implemented as a class and a train_model function, similar to the SFTTrainer class in huggingface
  2. Added in context examples for data transformation prompts. This is a minor addition to the existing prompts to improve transformation. Majority of the changes brought about in the transformation process will be part of a PR by @ritugala in the coming days.
  3. Added an examples directory with 3 smaller examples for: (a) creating transfomed data (b) creating synthetic data (c) qlora fine-tuning a mistral model from an existing dataset. This sets us up well to add examples in the future relating to: model selection, hyperparam optimization, full fine-tuning, etc.
saum7800 commented 8 months ago

have resolved all the issues as you had mentioned. there was a linting issue which was only showing up on GitHub actions, and not locally. locally, all pre-commit hooks are passing. have instructed to ignore the two error codes BLK 100 (Black would have made changes) and W503 (linebreak occured before binary operator). The BLK one is definitely fine to be ignored, since we are running black separately anyways. W503 raised errors on a lot of files that have been unmodified.

saum7800 commented 8 months ago

all the proposed changes make sense. have made the changes. please re-review when you get a chance, thanks!

saum7800 commented 8 months ago

made it QLoRA in all comments. thanks!