search

BorgwardtLab / proteinshake

Protein structure datasets for machine learning.
https://proteinshake.ai
BSD 3-Clause "New" or "Revised" License
101 stars 9 forks source link

test task evals #192

Closed cgoliver closed 1 year ago

cgoliver commented 1 year ago

Add more testing to tasks.

In particular, each task now implement a dummy_output() method which returns random predictions for the test set.

We test the eval by doing:

task.evaluate(task.dummy_output())

Added some other minor fixes.

codecov[bot] commented 1 year ago

Codecov Report

Patch coverage: 75.58% and project coverage change: +3.97 :tada:

Comparison is base (95ff2e8) 57.54% compared to head (ac33d1e) 61.51%.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #192 +/- ## ========================================== + Coverage 57.54% 61.51% +3.97% ========================================== Files 45 45 Lines 1498 1559 +61 ========================================== + Hits 862 959 +97 + Misses 636 600 -36 ``` | Flag | Coverage Δ | | |---|---|---| | pytest | `61.51% <75.58%> (+3.97%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=BorgwardtLab#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/BorgwardtLab/proteinshake/pull/192?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=BorgwardtLab) | Coverage Δ | | |---|---|---| | [proteinshake/tasks/enzyme\_class.py](https://codecov.io/gh/BorgwardtLab/proteinshake/pull/192?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=BorgwardtLab#diff-cHJvdGVpbnNoYWtlL3Rhc2tzL2VuenltZV9jbGFzcy5weQ==) | `76.66% <20.00%> (-7.95%)` | :arrow_down: | | [proteinshake/tasks/pfam\_task.py](https://codecov.io/gh/BorgwardtLab/proteinshake/pull/192?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=BorgwardtLab#diff-cHJvdGVpbnNoYWtlL3Rhc2tzL3BmYW1fdGFzay5weQ==) | `75.86% <20.00%> (+7.86%)` | :arrow_up: | | [proteinshake/tasks/gene\_ontology.py](https://codecov.io/gh/BorgwardtLab/proteinshake/pull/192?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=BorgwardtLab#diff-cHJvdGVpbnNoYWtlL3Rhc2tzL2dlbmVfb250b2xvZ3kucHk=) | `52.83% <50.00%> (-0.37%)` | :arrow_down: | | [proteinshake/tasks/ligand\_affinity.py](https://codecov.io/gh/BorgwardtLab/proteinshake/pull/192?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=BorgwardtLab#diff-cHJvdGVpbnNoYWtlL3Rhc2tzL2xpZ2FuZF9hZmZpbml0eS5weQ==) | `93.33% <75.00%> (+10.00%)` | :arrow_up: | | [proteinshake/tasks/protein\_protein\_interface.py](https://codecov.io/gh/BorgwardtLab/proteinshake/pull/192?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=BorgwardtLab#diff-cHJvdGVpbnNoYWtlL3Rhc2tzL3Byb3RlaW5fcHJvdGVpbl9pbnRlcmZhY2UucHk=) | `78.78% <77.77%> (+24.94%)` | :arrow_up: | | [proteinshake/tasks/structural\_class.py](https://codecov.io/gh/BorgwardtLab/proteinshake/pull/192?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=BorgwardtLab#diff-cHJvdGVpbnNoYWtlL3Rhc2tzL3N0cnVjdHVyYWxfY2xhc3MucHk=) | `90.00% <80.00%> (+5.38%)` | :arrow_up: | | [proteinshake/tasks/binding\_site\_detection.py](https://codecov.io/gh/BorgwardtLab/proteinshake/pull/192?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=BorgwardtLab#diff-cHJvdGVpbnNoYWtlL3Rhc2tzL2JpbmRpbmdfc2l0ZV9kZXRlY3Rpb24ucHk=) | `94.73% <87.50%> (+11.40%)` | :arrow_up: | | [proteinshake/tasks/virtual\_screen.py](https://codecov.io/gh/BorgwardtLab/proteinshake/pull/192?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=BorgwardtLab#diff-cHJvdGVpbnNoYWtlL3Rhc2tzL3ZpcnR1YWxfc2NyZWVuLnB5) | `93.54% <87.50%> (+45.54%)` | :arrow_up: | | [proteinshake/tasks/structure\_similarity.py](https://codecov.io/gh/BorgwardtLab/proteinshake/pull/192?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=BorgwardtLab#diff-cHJvdGVpbnNoYWtlL3Rhc2tzL3N0cnVjdHVyZV9zaW1pbGFyaXR5LnB5) | `97.05% <90.00%> (+13.72%)` | :arrow_up: | | [proteinshake/tasks/structure\_search.py](https://codecov.io/gh/BorgwardtLab/proteinshake/pull/192?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=BorgwardtLab#diff-cHJvdGVpbnNoYWtlL3Rhc2tzL3N0cnVjdHVyZV9zZWFyY2gucHk=) | `97.72% <90.90%> (+42.17%)` | :arrow_up: | | ... and [1 more](https://codecov.io/gh/BorgwardtLab/proteinshake/pull/192?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=BorgwardtLab) | | ... and [3 files with indirect coverage changes](https://codecov.io/gh/BorgwardtLab/proteinshake/pull/192/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=BorgwardtLab) Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=BorgwardtLab). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=BorgwardtLab)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.