microsoft / BackwardCompatibilityML

Project for open sourcing research efforts on Backward Compatibility in Machine Learning
https://backwardcompatibilityml.readthedocs.io/en/latest/
MIT License
71 stars 14 forks source link

H1 peformance dotted line and verification that error instances calculated correctly #122

Closed ilmarinen closed 3 years ago

ilmarinen commented 3 years ago

This PR adds some UI changes so that the h1 error instance dotted line is always shown, and not hidden in the case where it overlaps exactly with the X-axis.

This PR also extends an existing test to verify that the error instances on sweep that come from the training and testing set are stored separately and do not mix.

It was also verified that the data points int he scatterplot overlapping, was not due to the value of the loss being exactly the same. But due to the BTC and BEC scores being exactly the same. This is not surprising as it just calculated from the incompatible points.