@fstrunz @andbue sorry I'm late with this, but #289 is actually still broken: the skip setting does not work.
It will always raise if some pred files are missing.
The fix actually requires two things:
if skip is set, the superfluous GT ids must be ignored
at the end of the evaluation, at the time the confusion table and worst examples are to be reported,
the existing code constructed the dataset ad-hoc anew (but without the skip rules), ending up in a similar exception.
To avoid this, the set of GT IDs must be kept and passed along to the very end
@fstrunz @andbue sorry I'm late with this, but #289 is actually still broken: the
skip
setting does not work.It will always raise if some pred files are missing.
The fix actually requires two things:
skip
is set, the superfluous GT ids must be ignored