Pip installable package

AIPHES / DiscoScore

DiscoScore: Evaluating Text Generation with BERT and Discourse Coherence

32 stars 6 forks source link

Pip installable package #2

Closed JohnGiorgi closed 1 year ago

JohnGiorgi commented 2 years ago

This PR packages DiscoScore as a proper python package so that it is pip installable. The following changes were made

Add a setup.py with all major dependencies listed
Move most of the code under a package directory disco_score and name the main module scorer.py, similar to the popular BERTScore repo
Add a python-flavoured .gitignore
Import the project code from the main package, e.g. import metrics becomes from disco_score import metrics

With these changes, you can pip install the package right from GitHub:

pip install "git+https://github.com/AIPHES/DiscoScore.git"

And then import it anywhere

from disco_score import DiscoScorer

disco_scorer = DiscoScorer(device='cuda:0', model_name='bert-base-uncased')

@andyweizhao please take a look if you have time! I tested it using the example in the README but more testing might be a good idea. This would make it much easier for others to use your metric :)

andyweizhao commented 2 years ago

@JohnGiorgi

Thank you very much! I ran several tests (the running example, SUMMEval.py and NewsRoomEval.py) with your code. It works perfectly except for the issue of pip installation. Your installation command seems to install the latest version transformers. But DiscoScore cannot run properly on higher versions.. Perhaps after merging the PR I will have to downgrade the required version in the setup.py from "transformers>=4.6.1" to "transformers==3.1.0". Is that OK for you :))

JohnGiorgi commented 2 years ago

Hi @andyweizhao thanks for the response! If it really only works with that specific version of transformers then yes of course we can pin it in the setup.py file.

However, do you know what it is about newer versions of transformers that doesn't work with the current DiscoScore implementation? Pinning the dependency to a specific version is a little restrictive, especially given that transformers==3.1.0 is quite old.

andyweizhao commented 2 years ago

Hi @JohnGiorgi, I am not very knowledgeable about the PR management, so I extended your changes on my computer to support the latest version of transformers. I wrote you an acknowledgment in README.md :))

Many thanks for the PR!

UntotaufUrlaub commented 1 year ago

I am just curious: Is this pull request done and can be closed, or is here still something open?

andyweizhao commented 1 year ago

I made an update in https://github.com/AIPHES/DiscoScore/commit/13d7ff7f8da3d67fb0c8e48a0ed2155a65aedb3e. It resolved the PR issues and supported the newer version of the transformer lib.