openml / server-api

Python-based server
https://openml.github.io/server-api/
BSD 3-Clause "New" or "Revised" License
1 stars 1 forks source link

Standardize Task Cost Matrix #188

Open PGijsbers opened 1 month ago

PGijsbers commented 1 month ago

There is no standard or reasonable input validation for cost matrices right now. This leads to bad matrices such as “yes”, “adam”, or “1”.

We need to define what a valid cost matrix looks like, and add input validation for new tasks with cost matrices. We might want to convert or delete old tasks with cost matrices (depending on how interpretable they are).

This might be most convenient to process when working on task 2.0, but can be done independently.