search

defog-ai / sql-eval

Evaluate the accuracy of LLM generated outputs
Apache License 2.0
540 stars 57 forks source link

Added an MLX runner for evals #83

Closed rishsriv closed 7 months ago

rishsriv commented 7 months ago

This was tested on the amazing port of sqlcoder-7b-2 by @abhishek-ch on mlx-community here

mlx-lm does not support beam search yet – will update this to include beam search once it does (might make a pull request to mlx-lm to support beam search soon)

Additionally, this has verbose=True set up. We can choose to set that to False. Currently takes ~3-4 seconds per query on an M2 Max if your system is not on power saving mode.

rishsriv commented 7 months ago

(for anyone that's comes here later) This is how you can run an eval using MLX

python -W ignore main.py \
  -db postgres \
  -o "results/results.csv" \
  -g mlx \
  -f "prompts/prompt.md" \
  -m mlx-community/defog-sqlcoder-7b-2