Troyfeng116/integration tests

Yale-LILY / SummerTime

An open-source text summarization toolkit for non-experts. EMNLP'2021 Demo

https://arxiv.org/abs/2108.12738

Apache License 2.0

269 stars 30 forks source link

Troyfeng116/integration tests #38

Closed troyfeng116 closed 3 years ago

troyfeng116 commented 3 years ago

Add basic integration tests
Update model tests: assert model output typing + against input instances
Debug dataset + eval tests
Update model classes for new output type assertions

Note:

Eval tests still failing to run
Add py7zr pip dependency
SummEval backend eval metrics still broken on both local machine + Tangra

troyfeng116 commented 3 years ago

7/2:

summeval_metric metric raises KeyError with rouge
SummEval still broken on local machine
Individual eval tests are broken
Need to run pip install py7zr (will add to requirements.txt)

troyfeng116 commented 3 years ago

I thought I reviewed this but seems that I forgot to click submit...

Moreover, can you pull from main and see how many of these problems persist?

Yeah they do. I have bandaid fixes across a few open branches right now

niansong1996 commented 3 years ago

Okay, so the current plan for this thread is to merge the changes in the datasets to main, and then rebased this branch on main and see if more fixes are needed so that we don't break the main branch. What do you think?

troyfeng116 commented 3 years ago

Okay, so the current plan for this thread is to merge the changes in the datasets to main, and then rebased this branch on main and see if more fixes are needed so that we don't break the main branch. What do you think?

Yep sounds good, just lmk when ready to rebase