issues
search
Y-IAB
/
lm-evaluation-harness
A framework for few-shot evaluation of language models.
https://www.eleuther.ai
MIT License
0
stars
0
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
[Hotfix] Update fragma translator
#28
myeongho-jeong-yanolja
closed
2 months ago
0
Add a Translation Task as Multiple Choice
#27
hist0613
opened
4 months ago
0
[#25] Add Task List Metadata
#26
rifqiyan
closed
4 months ago
0
Hard for Puree to list all tasks
#25
rifqiyan
opened
4 months ago
1
Insecure Fragma API Key
#24
rifqiyan
opened
4 months ago
0
add json mode
#23
myeongho-jeong-yanolja
opened
5 months ago
2
Add fragma chat completion
#22
myeongho-jeong-yanolja
closed
4 months ago
5
[ #17] Update data path to puree
#21
myeongho-jeong-yanolja
opened
5 months ago
2
Add new task groups for low-cost evaluation.
#20
myeongho-jeong-yanolja
closed
5 months ago
0
[#17] Add Puree Dataset Integration
#19
rifqiyan
closed
5 months ago
0
[#16] Add Dockerfile for Running LM Evaluation Harness in Container
#18
rifqiyan
closed
4 months ago
0
Load Puree Dataset
#17
rifqiyan
closed
4 months ago
0
Run LM Evaluation Harness on VESSL AI
#16
rifqiyan
closed
4 months ago
0
Caching is not working properly.
#15
myeongho-jeong-yanolja
opened
5 months ago
0
Every evaluation costs money
#14
seungduk-yanolja
closed
5 months ago
3
FileNotFoundError: Unable to find '/home/seungduk/apps/lm-evaluation-harness/./data/yanolja_review_summarization.jsonl'
#13
seungduk-yanolja
opened
5 months ago
2
logging llm-eval results in detail
#12
myeongho-jeong-yanolja
closed
6 months ago
0
Add translation dataset
#11
myeongho-jeong-yanolja
closed
6 months ago
0
pull original repo's update
#10
myeongho-jeong-yanolja
closed
4 months ago
1
Add fragma based model & add xcomet/wmt23-cometkiwi metrics
#9
myeongho-jeong-yanolja
closed
6 months ago
0
Add Bartscore metric for summarization and fix LLM eval
#8
myeongho-jeong-yanolja
closed
6 months ago
0
update README and requiremetns
#7
myeongho-jeong-yanolja
closed
6 months ago
0
Add cli command to update task config
#6
kangsuhyun-yanolja
closed
6 months ago
0
Add metrics for translation and summarization (BLEU, ROUGE, BLEURT, BERTSCORE, COMETKIWI, Perplexity)
#5
myeongho-jeong-yanolja
closed
6 months ago
1
Update gpus arg
#4
kangsuhyun-yanolja
closed
6 months ago
0
Add Azure OpenAI
#3
kangsuhyun-yanolja
closed
6 months ago
0
Add Translator and metrics
#2
kangsuhyun-yanolja
closed
6 months ago
0
Add datasets
#1
myeongho-jeong-yanolja
closed
6 months ago
0