Feature split science dvc pipeline

wellcometrust / grants_tagger

Tag grants with MeSH and other tags

MIT License

14 stars 4 forks source link

Feature split science dvc pipeline #225

Closed aCampello closed 2 years ago

aCampello commented 2 years ago

Description

Decouples science pipeline from production (xlinear) pipeline.

Dvc repro is still not up-to-date, as we'll do it after we finish all chunks of work.

Checklist

[ ] Linked to Notion or GitHub issue
[ ] Added tests
[ ] Updated README
[ ] DVC up to date

Release checklist

[ ] DVC repro up to date
[ ] Models synced in S3
[ ] Version updated
[ ] CHANGELOG.md updated
[ ] Model and package version aligned

To release:

make build
make deploy

aCampello commented 2 years ago

Forgot to draft, but this is pretty much a draft

ArneRobben commented 2 years ago

Looks good. I can see the science bit in the original dvc.yaml and parameters.yaml is now separated in it's own pipelines/science

ArneRobben commented 2 years ago

my pip install -r requirements.txt really doesn't like the last line in the requirements: git+https://github.com/nsorros/shap.git@dev, do we need this?

aCampello commented 2 years ago

my pip install -r requirements.txt really doesn't like the last line in the requirements: git+https://github.com/nsorros/shap.git@dev, do we need this?

Yep, it's currently needed because of the experimental code that does explanations require a fork of Shap. When #223 is complete this won't be needed for xlinear.