issues
search
defog-ai
/
sql-eval
Evaluate the accuracy of LLM generated outputs
Apache License 2.0
485
stars
52
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Translate tsql instructions
#208
wendy-aw
closed
1 day ago
0
update run_model_cot.sh
#207
wongjingping
closed
1 week ago
0
Prompt/script updates
#206
wongjingping
closed
1 week ago
0
Pop sql_lora_name
#205
wendy-aw
closed
2 weeks ago
0
Replace weekday 1
#204
wendy-aw
closed
2 weeks ago
0
auto_error_analysis.ipynb error due to from "defog_utils.utils_sql import SqlFeatures, get_sql_features"
#203
SimaDidari
closed
2 weeks ago
1
Clarify dependency installation in README.md
#202
rishsriv
closed
3 weeks ago
0
Changed top 5 stocks to top 2 to break ties
#201
wendy-aw
closed
3 weeks ago
0
Enable evals on multiple databases for OpenAI and Anthropic generators
#200
rishsriv
closed
3 weeks ago
2
Add adapter_name arg
#199
wendy-aw
closed
3 weeks ago
0
Add onto #197
#198
wendy-aw
closed
3 weeks ago
0
Clarify integer qns
#197
wendy-aw
closed
3 weeks ago
0
More SQL fixes
#196
wendy-aw
closed
3 weeks ago
3
Rishabh/sqlite fixes
#195
rishsriv
closed
4 weeks ago
0
Add lora arguments to README for setting up vllm server
#194
rishsriv
closed
4 weeks ago
2
Fix sql for author to org ratio
#193
wendy-aw
closed
1 month ago
3
[BUG] Unable to run `compare_query_results` when using psycopg
#192
a3huang
closed
1 week ago
1
Instruction translation to SQLite
#191
wendy-aw
closed
1 month ago
0
Fix SQL errors
#190
wendy-aw
closed
1 month ago
4
Update prompts
#189
wendy-aw
closed
1 month ago
0
Modify result plots in slack
#188
wendy-aw
closed
1 month ago
0
Simplify openai prompt
#187
rishsriv
closed
1 month ago
2
Translated latest questions_gen
#186
wendy-aw
closed
1 month ago
1
Add on to #181 Question clarification
#185
wendy-aw
closed
1 month ago
0
Added a more optimal claude prompt
#184
rishsriv
closed
1 month ago
0
Clarified instructions for restaurants database
#183
rishsriv
closed
1 month ago
4
Post results to Slack after a grid search is completed
#182
rishsriv
closed
1 month ago
0
Rishabh/clarify questions system prompt
#181
rishsriv
closed
1 month ago
3
Added 10 more date questions
#180
wongjingping
closed
1 month ago
0
Fix translation script
#179
wendy-aw
closed
1 month ago
4
Update benchmark questions (1/2)
#178
wongjingping
closed
1 month ago
4
Add methods to log the results of the api runner to a server
#177
rishsriv
closed
1 month ago
1
Script
#176
wongjingping
closed
1 month ago
1
Add cot option in run_checkpoints_adapters
#175
wendy-aw
closed
1 month ago
0
Same as #172 + remove more ILIKE
#174
wendy-aw
closed
1 month ago
1
Revise 2 qn
#173
wongjingping
closed
1 month ago
1
Date arithmetic errors, ILIKE in instructions
#172
wendy-aw
closed
1 month ago
0
Eval script for vLLM LoRA
#171
wendy-aw
closed
1 month ago
0
Set default `c` to 0, set default api timeout to 15 seconds
#170
rishsriv
closed
1 month ago
0
LoRA support for vLLM
#169
wendy-aw
closed
1 month ago
0
Remove `defog_utils` dependency, to make it possible for people outside Defog to run evals
#168
rishsriv
closed
1 month ago
0
VLLM version support
#167
wongjingping
closed
1 month ago
0
Modify pruning and translation funcs
#166
wendy-aw
closed
1 month ago
0
BUG run postgreSQL sqleval not work completely.
#165
exceedzhang
closed
1 week ago
3
Upload wandb results
#164
wongjingping
closed
1 month ago
1
Clarify questions in dialect files
#163
wendy-aw
closed
1 month ago
0
Clarify questions in basic
#162
rishsriv
closed
1 month ago
1
add logprobs option to sql eval command
#161
wongjingping
closed
1 month ago
0
Add prealias option
#160
wongjingping
closed
1 month ago
0
IDK queries for BigQuery
#159
wendy-aw
closed
1 month ago
0
Next