braintrustdata autoevals issues

braintrustdata / autoevals

AutoEvals is a tool for quickly and easily evaluating AI model outputs using best practices.

MIT License

199 stars 17 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

Allow passing a custom OpenAI client to `OpenAIScorer` and subclasses.

#98 boydgreenfield opened 3 weeks ago
2
Fix ExactMatch export

#97 ankrgyl closed 4 weeks ago
2
Is there a way to access LLM token usage

#96 shekhargulati closed 4 weeks ago
3
Ensure EmbeddingSimilarity scores do not exceed 1

#95 danielericlee closed 1 month ago
1
Add customizable embedding model to `AnswerRelevancy` metric

#94 mongodben closed 1 month ago
2
Update some pacakges

#93 ankrgyl closed 1 month ago
1
Fix a few typos found via doc tests

#92 ankrgyl closed 2 months ago
2
Support AzureOpenAI service

#91 mongodben closed 2 months ago
3
Fix embedding similarity crash

#90 aphinx opened 2 months ago
2
Support of Azure Open AI models and API

#89 amir-mic-lin closed 2 months ago
4
Fix rationale

#88 ankrgyl closed 2 months ago
2
Add scorer purpose

#87 ankrgyl closed 2 months ago
3
Propagate prompt template through manifest

#86 ankrgyl closed 2 months ago
2
Change proxy URL

#85 ankrgyl closed 3 months ago
2
(`autoevals` JS): Better support for evaluating based on pre-generated answer

#84 mongodben closed 3 months ago
2
Propagate args to embedding metrics

#83 ankrgyl closed 3 months ago
4
(`autoevals` JS) Better support and documentation for using context-based evaluators in `Eval` run

#82 mongodben opened 3 months ago
3
JS `AnswerRelevancy` bug with model configuration

#81 mongodben closed 3 months ago
2
Context Relevancy issue with score not between 0 and 1.

#80 jp-agenta opened 3 months ago
0
Add ExactMatch scorer

#79 ankrgyl closed 3 months ago
2
Try gpt-4o-mini

#78 ankrgyl opened 3 months ago
1
update pre-commit prettier to 3.3.2

#77 j13huang closed 3 months ago
1
Update links for npm package

#76 tara-nagar closed 3 months ago
1
Readme fixes

#75 danielericlee closed 3 months ago
0
Add public evals

#74 ankrgyl closed 3 months ago
3
Add support for partial syntax

#73 ankrgyl closed 3 months ago
0
Some housekeeping updates

#72 ankrgyl closed 3 months ago
0
Fix list contains DX

#71 ankrgyl closed 4 months ago
0
Pin zod to 3.22.4.

#70 manugoyal closed 4 months ago
0
Update openai package

#69 ankrgyl closed 5 months ago
0
Update autoevals README to remove usage of `experiment.log`

#68 aphinx closed 5 months ago
0
Add Valid JSON as a scorer

#67 tara-nagar closed 5 months ago
0
fix matter => manner typo

#66 bardia-pourvakil closed 6 months ago
0
[DRAFT] adding claude support for llmclassifier

#65 dheerajiiitv closed 6 months ago
9
Add generation and end-to-end RAGAS metrics

#64 aphinx closed 6 months ago
0
Added Moderation as a scorer

#63 dashk closed 6 months ago
0
fix: move tsx and @types/node to devDependencies

#62 transitive-bullshit closed 6 months ago
1
Question about deps

#61 transitive-bullshit closed 6 months ago
1
RAGAS retrieval metrics

#60 ankrgyl closed 6 months ago
0
Add list comparison metric

#59 ankrgyl closed 6 months ago
0
Bump core version

#58 ankrgyl closed 7 months ago
0
Fall back to BRAINTRUST_API_KEY if OPENAI_API_KEY is not set.

#57 manugoyal closed 8 months ago
0
Don't swallow errors into scores.

#56 manugoyal closed 8 months ago
0
README fix + use enums to improve classification accuracy

#55 ankrgyl closed 8 months ago
0
Merge `node`-specific env initialization into `env.ts`.

#54 manugoyal closed 9 months ago
0
Add `Sql` scorer to python

#53 aphinx closed 9 months ago
0
Use openai wrapper

#52 ankrgyl closed 9 months ago
0
General Question about the Evaluator LLM

#51 lalehsg opened 9 months ago
3
Fix tracing bug.

#50 manugoyal closed 9 months ago
0
Support OPENAI_BASE_URL in autoevals

#49 ankrgyl closed 9 months ago
0