njallskarp / finetune-qa-powerset

Finetuning BERT models on a powerset of different linguistic domains
https://lvl.ru.is/
5 stars 0 forks source link

Feat/unit test setup #45

Closed Eysteinn-Orn closed 1 year ago

Eysteinn-Orn commented 1 year ago

Units tests and Github Actions for running tests

Adding unit tests and github actions such that when any one of us pushes or pulls to dev we have to pass the unit tests.

How to test

Eysteinn-Orn commented 1 year ago

The GHA seems to be working correctly now but I just need the import to work correctly now in the test file itself

Eysteinn-Orn commented 1 year ago

It seems that adding __init__.py to Every directory fixes the problem regarding imports and also removing all relative imports, additionally we have to rename the types folder since by adding a __init__.py it becomes a package and tries to overwrite an existing python package so I propose to rename that folder to 'typings' but we could also do 'definitions'.

The way I am solving this problem doesn't seem right but it does get the tests running

njallskarp commented 1 year ago

Really great addition to the codebase @eysta00, adding a few comments for improvements.

Eysteinn-Orn commented 1 year ago

Kinda stuck on this problem of when importing pytorch, no matter what, I get an import error in the github action workflow.

Honestly the way to fix this just momentarily is to decouple the evaluate_model function from metrics.py, but that won't fix the underlying problem of Github Actions having none of the CUDA libraries, been stuck on this for the last two days.

Going to come back to this in the evening and rework the actual tests to just be better.

If we run our own Github Actions server with a GPU then this should be no problem but that is also a whole new can of worms.

Eysteinn-Orn commented 1 year ago

Managed to get Pytorch to import correctly using the CPU version of Pytorch, however now there seems to be some version incompatibilities with importing the RobertaForQuestionAnswering transformer