Closed bertsky closed 1 year ago
That's weird, you're right. Does --skip_empty_gt do what you need?
Kind of. But I'd have to check against existing lines that are actually empty first...
Ah, I get it, skip_empty_gt would not count any wrong predictions in actually empty lines as errors.
There seems to be a mismatch between documentation and implementation:
--non_existing_file_handling_mode
is ignored completely.Also, IIUC that leaves me with no option on the CLI to skip missing file pairs – I can only use
--gt.non_existing_as_empty
and--skip_empty_gt
and--evaluator.non_existing_pred_as_empty
, which is obviously not the same.(My use-case: I have a subset of manually edited files and lots more of predictions, and want to restrict calculation to that subset.)