search

hegelai / prompttools

Open-source tools for prompt testing and experimentation, with support for both LLMs (e.g. OpenAI, LLaMA) and vector databases (e.g. Chroma, Weaviate, LanceDB).
http://prompttools.readthedocs.io
Apache License 2.0
2.55k stars 216 forks source link

MusicGen experiment #82 #106

Closed HashemAlsaket closed 3 months ago

HashemAlsaket commented 7 months ago

Introducing TTA model measurement to prompttools. Similar to LLM and TTI experiments already in place for prompttools, this PR aims to allow for measuring the quality of audio generation models.

Example:

  1. Developer prompts TTA model to generate a described audio
  2. Developer uses prompttools to measure similarity of tta-model-generated audio against real audio seamlessly, and at scale.

Next steps for contributors that are interested:

  1. Complete the experiment framework to work async: /prompttools/prompttools/experiment/experiments/musicgen_experiment.py
  2. Complete the example experiment for quick start: /prompttools/examples/notebooks/audio_experiments/MusicGenExperiment.ipynb
HashemAlsaket commented 4 months ago

@steventkrawczyk @NivekT ready for review!