issues
search
shmsw25
/
FActScore
A package to evaluate factuality of long-form generation. Original implementation of our EMNLP 2023 paper "FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation"
https://arxiv.org/abs/2305.14251
MIT License
236
stars
32
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Strange token ID mapping
#49
Yingjia-Wan
opened
5 days ago
0
My FactScore
#48
IdoAmit198
closed
1 month ago
0
Maintaining core factscore in a new repository
#47
armingh2000
opened
2 months ago
0
Improved version for speed and openai updates
#46
wj210
opened
2 months ago
0
Batching for speed.
#45
wj210
opened
2 months ago
0
About the enwiki-20230401
#44
Toblame
opened
4 months ago
5
loosen overly restrictive dependencies
#43
dojoteef
opened
4 months ago
0
make factscore use chatgpt for atomic fact generation
#42
dojoteef
opened
4 months ago
1
#38 Groundwork for migration to new openai >=v1.0.0 python API
#41
GeroVanMi
closed
4 months ago
2
sqlite3.DatabaseError: file is not a database error after Installation
#40
GeroVanMi
closed
4 months ago
0
InvalidRequestError: The model `text-davinci-003` has been deprecated
#39
shrusti-ghela
closed
4 months ago
2
Move to ChatCompletion endpoint for call_GPT3
#38
martiansideofthemoon
opened
4 months ago
0
AssertionError: `topic` in your data (Jessie Mae Brown Beavers) is likely to be not a valid title in the DB.
#37
clawnotfound
opened
4 months ago
1
problem using the factscorer function with LLaMa
#36
samuel-LP
opened
5 months ago
1
Some cases couldn't be found in the default db
#35
Loose-Gu
opened
6 months ago
2
Any way to run without an OpenAI key?
#34
mohummedalee
opened
6 months ago
3
Issue running get_score() on M1 Mac due to Torch build
#33
jcblsn
closed
3 months ago
1
How to modify the code to calculate factscore between any two paragraphs/sentences without retrieval steps?
#32
laiviet
opened
7 months ago
1
Generation details for models under data/labeled
#31
caiqizh
opened
7 months ago
1
The chatgpt-based AtomicFactGenerator of FactScore sometimes abstain from answering
#30
HillZhang1999
closed
7 months ago
4
Regarding human evaluation
#29
pat-jj
closed
8 months ago
2
FileNotFoundError (demons.json) for custom knowledge base
#28
MathiasKraus
opened
8 months ago
2
Typo here
#27
CiaoHe
closed
8 months ago
1
Scoring existing generated text
#26
sofi444
closed
8 months ago
1
Having issues on using LLAMA-7B model
#25
mungg
opened
9 months ago
2
Small changes to ensure custom corpus addition works
#24
martiansideofthemoon
closed
10 months ago
0
FActScore can be slow to run
#23
martiansideofthemoon
opened
10 months ago
0
Using unlabeled data to generate atomic facts and retrieving evidence
#22
rubaha96
closed
10 months ago
1
add length penalty
#21
shmsw25
closed
1 year ago
1
Moving abstain detection to a separate module.
#20
martiansideofthemoon
closed
1 year ago
0
Move abstain detection outside the atomic fact generator
#19
martiansideofthemoon
closed
1 year ago
0
Create LICENSE
#18
martiansideofthemoon
closed
1 year ago
0
Don’t understand how true_score and false_score are represented in logits
#17
rubaha96
closed
1 year ago
1
License?
#16
brucewlee
closed
1 year ago
0
fix bugs in retrieval
#15
absolute-nil
closed
1 year ago
1
MAX_LENGTH is not defined in retrieval
#14
absolute-nil
closed
1 year ago
2
Better ranking between models
#13
martiansideofthemoon
closed
1 year ago
1
Adding cost estimates for OpenAI API usage
#12
martiansideofthemoon
closed
1 year ago
0
setup log, separating data_dir
#11
shmsw25
closed
1 year ago
0
Improving logging, show different levels
#10
martiansideofthemoon
closed
1 year ago
0
Instruct LLAMA integration
#9
martiansideofthemoon
closed
1 year ago
0
added `% of responding` and `num_facts_per_response`
#8
shmsw25
closed
1 year ago
1
Return respond rate and number of facts together with FActScore
#7
shmsw25
closed
1 year ago
1
improve processing of Perplexity AI outputs
#6
martiansideofthemoon
closed
1 year ago
2
Create HF weight diffs for LLAMA model and add it to the README
#5
martiansideofthemoon
closed
1 year ago
0
Add citation / bibtex to README
#4
martiansideofthemoon
closed
1 year ago
1
estimated cost for API before running API
#3
martiansideofthemoon
closed
1 year ago
1
small updates
#2
shmsw25
closed
1 year ago
0
new data format + atomic fact gen integrated
#1
martiansideofthemoon
closed
1 year ago
0