issues
search
defog-ai
/
sql-eval
Evaluate the accuracy of LLM generated outputs
Apache License 2.0
447
stars
47
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Add onto #197
#198
wendy-aw
closed
1 day ago
0
Clarify integer qns
#197
wendy-aw
closed
2 days ago
0
More SQL fixes
#196
wendy-aw
closed
2 days ago
3
Rishabh/sqlite fixes
#195
rishsriv
closed
4 days ago
0
Add lora arguments to README for setting up vllm server
#194
rishsriv
closed
5 days ago
2
Fix sql for author to org ratio
#193
wendy-aw
closed
1 week ago
3
[BUG] Unable to run `compare_query_results` when using psycopg
#192
a3huang
opened
1 week ago
1
Instruction translation to SQLite
#191
wendy-aw
closed
1 week ago
0
Fix SQL errors
#190
wendy-aw
closed
1 week ago
4
Update prompts
#189
wendy-aw
closed
1 week ago
0
Modify result plots in slack
#188
wendy-aw
closed
1 week ago
0
Simplify openai prompt
#187
rishsriv
closed
1 week ago
2
Translated latest questions_gen
#186
wendy-aw
closed
1 week ago
1
Add on to #181 Question clarification
#185
wendy-aw
closed
1 week ago
0
Added a more optimal claude prompt
#184
rishsriv
closed
1 week ago
0
Clarified instructions for restaurants database
#183
rishsriv
closed
1 week ago
4
Post results to Slack after a grid search is completed
#182
rishsriv
closed
1 week ago
0
Rishabh/clarify questions system prompt
#181
rishsriv
closed
1 week ago
3
Added 10 more date questions
#180
wongjingping
closed
1 week ago
0
Fix translation script
#179
wendy-aw
closed
1 week ago
4
Update benchmark questions (1/2)
#178
wongjingping
closed
1 week ago
4
Add methods to log the results of the api runner to a server
#177
rishsriv
closed
1 week ago
1
Script
#176
wongjingping
closed
2 weeks ago
1
Add cot option in run_checkpoints_adapters
#175
wendy-aw
closed
2 weeks ago
0
Same as #172 + remove more ILIKE
#174
wendy-aw
closed
2 weeks ago
1
Revise 2 qn
#173
wongjingping
closed
2 weeks ago
1
Date arithmetic errors, ILIKE in instructions
#172
wendy-aw
closed
2 weeks ago
0
Eval script for vLLM LoRA
#171
wendy-aw
closed
2 weeks ago
0
Set default `c` to 0, set default api timeout to 15 seconds
#170
rishsriv
closed
2 weeks ago
0
LoRA support for vLLM
#169
wendy-aw
closed
2 weeks ago
0
Remove `defog_utils` dependency, to make it possible for people outside Defog to run evals
#168
rishsriv
closed
2 weeks ago
0
VLLM version support
#167
wongjingping
closed
3 weeks ago
0
Modify pruning and translation funcs
#166
wendy-aw
closed
3 weeks ago
0
BUG run postgreSQL sqleval not work completely.
#165
exceedzhang
opened
3 weeks ago
3
Upload wandb results
#164
wongjingping
closed
3 weeks ago
1
Clarify questions in dialect files
#163
wendy-aw
closed
3 weeks ago
0
Clarify questions in basic
#162
rishsriv
closed
3 weeks ago
1
add logprobs option to sql eval command
#161
wongjingping
closed
3 weeks ago
0
Add prealias option
#160
wongjingping
closed
4 weeks ago
0
IDK queries for BigQuery
#159
wendy-aw
closed
4 weeks ago
0
Add sql featurization into auto_error_analysis.ipynb summaries
#158
wongjingping
closed
1 month ago
2
Fix pruning and the logic of md retrieval
#157
wendy-aw
closed
1 month ago
0
Edit ambiguous qn/sql in car_dealership
#156
wendy-aw
closed
1 month ago
1
Dynamically add table aliases without an LLM + de-duplicate columns in pandas
#155
rishsriv
closed
1 month ago
1
Small pesky bugs found while examining dialect exec errors
#154
wendy-aw
closed
1 month ago
2
Add CoT
#153
wongjingping
closed
1 month ago
1
SQLite SERIAL, drop valid/err_msg cols
#152
wendy-aw
closed
1 month ago
0
Expanded correct queries for 3 questions + minor prompt/typo fixes
#151
rishsriv
closed
1 month ago
1
Usability improvements
#150
wongjingping
closed
1 month ago
1
Dialect data files
#149
wendy-aw
closed
1 month ago
0
Next