search

hegelai / prompttools

Open-source tools for prompt testing and experimentation, with support for both LLMs (e.g. OpenAI, LLaMA) and vector databases (e.g. Chroma, Weaviate, LanceDB).
http://prompttools.readthedocs.io
Apache License 2.0
2.55k stars 216 forks source link

Add fine tuning experiment #88

Closed steventkrawczyk closed 9 months ago

steventkrawczyk commented 9 months ago

Looks good to me! Thank you for adding this. Just small comments:

  1. Might need to ask users to install datasets from HuggingFace up top
  2. Is file-name on this line openai.FineTuningJob.create(training_file="file-name", model="gpt-3.5-turbo") correct? I guess that may be the default name?
  3. How long does the fine-tuning job take? We can mention that as well so they know what to expect.

I think adding a SQL syntax validator (as an eval function) in the future can be cool.

  1. Added relevant installations to the top section
  2. Updated the name to make it more clear it's a placeholder.
  3. Added some notes, it could be a few hours for tuning.

Totally agree that validating SQL is a good one, added an issue here: https://github.com/hegelai/prompttools/issues/89