JuliaDecisionFocusedLearning / InferOpt.jl

Combinatorial optimization layers for machine learning pipelines
https://juliadecisionfocusedlearning.github.io/InferOpt.jl/
MIT License
113 stars 4 forks source link

TestItem framework #53

Closed gdalle closed 1 year ago

gdalle commented 1 year ago

Replace the usual tests with the new testitem framework for VSCode. This allows tests to be run independently and in parallel.

Solves #46

By the way, I think we can still see the plots: there is a button "Show output" at the top of the testing tab

gdalle commented 1 year ago

@BatyLeo the tests of your general imitation loss are commented out

codecov-commenter commented 1 year ago

Codecov Report

Patch coverage: 100.00% and project coverage change: -11.32 :warning:

Comparison is base (464508c) 93.71% compared to head (74c2379) 82.39%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #53 +/- ## =========================================== - Coverage 93.71% 82.39% -11.32% =========================================== Files 22 20 -2 Lines 525 409 -116 =========================================== - Hits 492 337 -155 - Misses 33 72 +39 ``` | [Impacted Files](https://app.codecov.io/gh/axelparmentier/InferOpt.jl/pull/53?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) | Coverage Δ | | |---|---|---| | [src/InferOpt.jl](https://app.codecov.io/gh/axelparmentier/InferOpt.jl/pull/53?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-c3JjL0luZmVyT3B0Lmps) | `100.00% <ø> (ø)` | | | [src/frank\_wolfe/differentiable\_frank\_wolfe.jl](https://app.codecov.io/gh/axelparmentier/InferOpt.jl/pull/53?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-c3JjL2ZyYW5rX3dvbGZlL2RpZmZlcmVudGlhYmxlX2ZyYW5rX3dvbGZlLmps) | `93.75% <100.00%> (-2.48%)` | :arrow_down: | ... and [20 files with indirect coverage changes](https://app.codecov.io/gh/axelparmentier/InferOpt.jl/pull/53/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

BatyLeo commented 1 year ago

@gdalle ImitationLoss tests are updated

Should we remove/disable nightly from CI (and add 1.8) before merging ?

gdalle commented 1 year ago

Should we remove/disable nightly from CI (and add 1.8) before merging ?

What I usually see in CI setups is that people test on

I don't think we need to cover all versions in between. As for the bug on nightly, it's getting fixed soon: https://github.com/FluxML/Zygote.jl/pull/1420

The better option would be to ignore nightly for the badge. I opened a topic here (https://discourse.julialang.org/t/ignoring-nightly-failure-for-ci-badge/98028) but the solution I adopted doesn't seem to work anymore so :shrug: