The project currently doesn't support multiclass classification models, this should be valuable and relatively simple to implement.
Possible Solution
Each yPred files have multiple columns, each indicating predicted probability of one class
Automatic clustering algorithm uses nModels * (nClasses - 1) columns to do segmentation
Visualization doesn't have to change since one data point is still associated with one metric per model, which is the log-loss.
If users want to segment by performance on one single class, we can implement that based on user-defined performance metric (#105), where user specify the metric to be some sort of loss function comparing the prediction for class N with the ground truth for class N
Does it support object detection models? If yes, how do i prepare my prediction data for manifold to be able to visualize model performace.
what would be the contents of features dataset, pixel values?
Summary
The project currently doesn't support multiclass classification models, this should be valuable and relatively simple to implement.
Possible Solution
yPred
files have multiple columns, each indicating predicted probability of one classnModels * (nClasses - 1)
columns to do segmentation