This package implements the spectral framework for model evaluation. All you need to get started is (1) a model, (2) a dataset, and (3) a definition of sample to sample similarity!
The SPECTRA package generates a series of splits with decreasing train-test similarity. Evaluating your models on these splits will give a better understanding of model generalizability. Read the preprint for more info on how this works.
See https://github.com/mims-harvard/SPECTRA and https://twitter.com/YEktefaie/status/1782449554077647054