This PR aims to introduce new features to scikit-matter, which allows the user to calculate what is called the local prediction rigidity (LPR) and the (local) component-wise prediction rigidity ((L)CPR).
LPR and CPR are metrics that allow one to assess the robustness, or quite literally the "rigidity", in the local or component-wise prediction that the machine learning model makes. More details can be found in this preprint: https://arxiv.org/pdf/2306.15638.pdf
(Note that CPR is not explained in the above preprint. It is, however, a metric that follows the same spirit as the LPR, and is something that will be shared in forthcoming works by myself and coworkers.)
I found these things to best fit in with the metrics, and hence the functions have been coded in there. Key points to note, among other things, is that the function assumes the user to (1) provide the regularization strength of a model that has been trained, and (2) consider the LPR and CPR only in the contexts where training is done on structural average feature vectors and targets, to keep the resulting metric within an interpretable and roughly transferable range.
Please let me know what you think. Tagging and assigning @agoscinski as he has helped me write up this PR. Will only post as draft at the moment.
This PR should close scikit-learn-contrib/scikit-matter#191
Hello all,
This PR aims to introduce new features to
scikit-matter, which allows the user to calculate what is called the local prediction rigidity (LPR) and the (local) component-wise prediction rigidity ((L)CPR).LPR and CPR are metrics that allow one to assess the robustness, or quite literally the "rigidity", in the local or component-wise prediction that the machine learning model makes. More details can be found in this preprint: https://arxiv.org/pdf/2306.15638.pdf (Note that CPR is not explained in the above preprint. It is, however, a metric that follows the same spirit as the LPR, and is something that will be shared in forthcoming works by myself and coworkers.)
I found these things to best fit in with the
metrics, and hence the functions have been coded in there. Key points to note, among other things, is that the function assumes the user to (1) provide the regularization strength of a model that has been trained, and (2) consider the LPR and CPR only in the contexts where training is done on structural average feature vectors and targets, to keep the resulting metric within an interpretable and roughly transferable range.Please let me know what you think. Tagging and assigning @agoscinski as he has helped me write up this PR. Will only post as draft at the moment.
This PR should close scikit-learn-contrib/scikit-matter#191
:books: Documentation preview :books:: https://scikit-matter--209.org.readthedocs.build/en/209/