shmsw25 / FActScore

A package to evaluate factuality of long-form generation. Original implementation of our EMNLP 2023 paper "FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation"
https://arxiv.org/abs/2305.14251
MIT License
292 stars 43 forks source link

#38 Groundwork for migration to new openai >=v1.0.0 python API #41

Closed GeroVanMi closed 9 months ago

GeroVanMi commented 9 months ago

Hi @martiansideofthemoon (tagged you, since you were recently active on #38)

I have tried to make the necessary changes for the new OpenAI python library to be used with this codebase. I have tested the atomic_facts generator and it seems to work quite well. However, I am not very well-versed in this project, so please do some testing of your own, before considering to merge this.

Please feel free to change anything according to your needs or give feedback if you'd like me to change something. This project looks very promising and I would like to test our own solutions against this. You have done great work here and I'd be glad to hear back from you!

Changelog

GeroVanMi commented 9 months ago

I just realized that the project uses Poetry and I haven't updated the dependencies accordingly, I will do so asap.

GeroVanMi commented 9 months ago

Update: There is currently a version conflict between spacy and the new openai library. This will probably need to be resolved on the side of spacy, since they are requiring an old version of the typing-extensions library.

For now: There doesn't seem to be a direct usage (except for the download for setup) of spacy, so maybe it could just be removed from the actual dependencies and installed manually through pip? If it is needed somewhere that I didn't find, we'd probably have to wait until spacy updates to the newer version or find some alternative solution.