issues
search
defog-ai
/
sql-eval
Evaluate the accuracy of LLM generated outputs
Apache License 2.0
540
stars
57
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Updated sql-eval questions
#121
wongjingping
closed
5 months ago
0
Fixed a typo in the CSV
#120
rishsriv
closed
5 months ago
0
Provide multiple correct answers for some questions
#119
rishsriv
closed
5 months ago
1
Enable tgi via api type
#118
wongjingping
closed
5 months ago
0
fix temperature/top_p warning
#117
wongjingping
closed
5 months ago
0
Added a bedrock runner to make it easier to run models straight from bedrock
#116
rishsriv
closed
5 months ago
0
Remove dependency on AutoTokenizer in api_runner
#115
rishsriv
closed
5 months ago
0
Update model runners
#114
wongjingping
closed
5 months ago
1
Continuous eval script
#113
wongjingping
closed
5 months ago
0
Add joinable columns as part of the metadata string with `c=0`
#112
rishsriv
closed
5 months ago
1
Added a batch size arg for hf runner
#111
rishsriv
closed
5 months ago
0
Fixed bug where "instructions" were being added to the prompt even if not specified
#110
rishsriv
closed
5 months ago
0
Add the questions file as a required parameter in the README files
#109
rishsriv
closed
5 months ago
1
Enable multiple question files
#108
wongjingping
closed
5 months ago
0
Basic question modifications
#107
wongjingping
closed
5 months ago
0
Enable argument for specifying how much to round floats to
#106
rishsriv
closed
5 months ago
0
Small bugfix for the gemini runner
#105
rishsriv
closed
6 months ago
0
fix query
#104
wongjingping
closed
6 months ago
0
Refined advanced questions
#103
wongjingping
closed
6 months ago
0
Add advanced instruct questions
#102
wongjingping
closed
6 months ago
0
Refine basic questions
#101
wongjingping
closed
6 months ago
1
Add basic instruct questions
#100
wongjingping
closed
6 months ago
1
Creating temp dbs
#99
wendy-aw
closed
6 months ago
0
Add args for follow-on questions
#98
wendy-aw
closed
6 months ago
0
it seems sqlcoder-7b-2 is not better than 34b
#97
njhouse365
closed
6 months ago
3
handle exceptions when comparing dataframes with single value
#96
rishsriv
closed
6 months ago
0
Update HF runner to work with batched inference
#95
rishsriv
closed
6 months ago
1
Why when I run eval benchamrk, the name of categories is different from blog.
#94
cometyang
closed
6 months ago
14
Update where to instruct
#93
wongjingping
closed
7 months ago
2
Add number of columns and shuffle as args
#92
wongjingping
closed
7 months ago
2
Updated anthropic runners to reflect their new API for Claude-3
#91
rishsriv
closed
7 months ago
0
Fix pruning function
#90
wendy-aw
closed
7 months ago
0
use sqlcoder-7b-2 as default example
#89
rishsriv
closed
7 months ago
0
New prompts
#88
wendy-aw
closed
7 months ago
0
Add support for Gemini models
#87
rishsriv
closed
7 months ago
0
Added support for Mistral models
#86
rishsriv
closed
7 months ago
0
Update README
#85
wongjingping
closed
7 months ago
0
Modify prompts
#84
wendy-aw
closed
7 months ago
0
Added an MLX runner for evals
#83
rishsriv
closed
7 months ago
1
Add llama_cpp_runner.py for Llama CPP evaluation
#82
rishsriv
closed
7 months ago
0
Update prompt.md
#81
rishsriv
closed
8 months ago
0
Glossary & table metadata string
#80
wendy-aw
closed
8 months ago
0
Added ability to report results to a URL, and sample server code for saving it to a database
#79
rishsriv
closed
8 months ago
1
Order By Fix
#78
cfahlgren1
closed
8 months ago
0
add a quantized arg for vllm to run awq models
#77
rishsriv
closed
8 months ago
0
Add api server instructions
#76
wongjingping
closed
8 months ago
0
Add sqlparse to requirements.txt for vllm compatibility
#75
rishsriv
closed
8 months ago
0
Changes to sorting eval for increased robustness
#74
rishsriv
closed
8 months ago
0
removed defog-data from requirements.txt
#73
rishsriv
closed
8 months ago
0
Added quotes to fix qn in snowflake
#72
wendy-aw
closed
8 months ago
0
Previous
Next