issues
search
citadel-ai
/
langcheck
Simple, Pythonic building blocks to evaluate LLM applications.
https://langcheck.readthedocs.io/en/latest/index.html
MIT License
184
stars
17
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Support Async API for embedding based metrics
#159
taniokay
opened
3 days ago
5
Fix typo
#158
taniokay
closed
1 week ago
0
Update the interface of `langcheck.augment.rephrase`
#157
liwii
opened
1 week ago
0
Support async OpenAI clients for embedding-based metrics
#156
liwii
opened
1 week ago
0
Add no-local-llm to avoid vllm installation
#155
taniokay
closed
1 week ago
1
Add nltk download in langcheck.augment.synonym
#154
taniokay
closed
2 weeks ago
2
langcheck.augment.synonym doesn't work because of some missing nltk package
#153
liwii
closed
2 weeks ago
1
Support other types of parameters
#152
liwii
closed
4 days ago
0
Review of "Refactor metric inputs"
#151
yosukehigashi
closed
3 weeks ago
0
Fix a typo in the augmentation template
#150
liwii
closed
3 weeks ago
0
Let users access the prompt & score_map objects of the built-in eval client metrics
#149
liwii
opened
4 weeks ago
0
Implement Simulated Annotators for estimating confidence scores for pairwise comparison
#148
conan1024hao
closed
3 weeks ago
9
Refactor metric inputs
#147
liwii
closed
3 weeks ago
2
Bump version to 0.8.0.dev6
#146
yosukehigashi
closed
1 month ago
0
Update docs to reflect new metric structure
#145
yosukehigashi
opened
1 month ago
3
Versioning of eval prompts
#144
yosukehigashi
closed
1 month ago
6
Update nltk to 3.9
#143
yosukehigashi
closed
1 month ago
0
Upgrade ruff to v0.6
#142
yosukehigashi
closed
1 month ago
0
Remove benchmarking dir
#141
yosukehigashi
closed
1 month ago
0
Upgrade `ruff` to 0.6
#140
yosukehigashi
closed
1 month ago
0
Fix typo in documentation
#139
kennysong
closed
2 months ago
0
Add safety related built-in metrics
#138
liwii
closed
1 month ago
2
More Augmentations
#137
liwii
closed
1 month ago
7
Update the toxicity metric [en, ja]
#136
conan1024hao
closed
1 month ago
3
SwallowEvalClient → LlamaEvalClient
#135
conan1024hao
closed
2 months ago
0
Improve Swallow system prompt
#134
conan1024hao
closed
2 months ago
3
Fix the bug that langcheck doesn't work when [local-llm] is not installed
#133
conan1024hao
closed
2 months ago
1
Template based jailbareak augmentation
#132
liwii
closed
2 months ago
1
Fix text encoding
#131
conan1024hao
closed
2 months ago
0
Answer correctness metric
#130
yosukehigashi
closed
2 months ago
1
Custom pairwise evaluator metric
#129
yosukehigashi
closed
2 months ago
0
Implement the Swallow Evaluation Client
#128
conan1024hao
closed
2 months ago
0
Improve the stability of metrics by repeated queries
#127
liwii
opened
3 months ago
0
[WIP] Add custom prompts for the Prometheus model
#126
conan1024hao
closed
3 months ago
0
Introduce Ruff as the formatter
#125
liwii
closed
3 months ago
3
Return `None` if the function calling step returns an invalid assessment
#124
yosukehigashi
closed
3 months ago
0
Handle `None` sources in the pairwise comparison metric
#123
yosukehigashi
closed
3 months ago
0
Add Prometheus Eval Client
#122
conan1024hao
closed
3 months ago
4
Bump version to 0.8.0.dev2
#121
yosukehigashi
closed
3 months ago
2
Custom evaluator metric
#120
yosukehigashi
closed
4 months ago
0
Bump version to 0.8.0.dev1
#119
yosukehigashi
closed
4 months ago
2
Update prompts to output the chain-of-thought reasoning first
#118
yosukehigashi
closed
4 months ago
0
Increment version to 0.7.1
#117
liwii
closed
4 months ago
0
Rename ja & de prompts
#116
liwii
closed
4 months ago
3
Fix docs for eval_client
#115
kennysong
closed
4 months ago
0
Increment version to 0.7.0
#114
liwii
closed
4 months ago
0
Add Gemini Eval Client
#113
yosukehigashi
closed
4 months ago
0
Update devcontainer.json
#112
liwii
closed
5 months ago
5
Prototype of Claude-based evaluation
#111
liwii
closed
5 months ago
7
Refactoring LLM-based metrics
#110
liwii
closed
5 months ago
4
Next