search

Caleydo / confusionflow-ui

Visualize learning dynamics in multi-class image classifiers (e.g., convolutional neural networks) 💬 @gfrogat @thinkh
https://confusionflow.org
BSD 3-Clause "New" or "Revised" License
1 stars 0 forks source link

Add class selector for datasets that have 10+ classes and hierarchy #111

Open thinkh opened 6 years ago

thinkh commented 6 years ago

In case of the CIFAR 100 dataset we need to decrease the number of classes to 10 in order to show them in the conf matrix. (Assuming that we use the current malevo approach.)

The CIFAR 100 dataset contains 100 classes that are grouped in superclasses.

Superclass Classes
aquatic mammals beaver, dolphin, otter, seal, whale
fish aquarium fish, flatfish, ray, shark, trout
flowers orchids, poppies, roses, sunflowers, tulips
food containers bottles, bowls, cans, cups, plates
fruit and vegetables apples, mushrooms, oranges, pears, sweet peppers
household electrical devices clock, computer keyboard, lamp, telephone, television
household furniture bed, chair, couch, table, wardrobe
insects bee, beetle, butterfly, caterpillar, cockroach
large carnivores bear, leopard, lion, tiger, wolf
large man-made outdoor things bridge, castle, house, road, skyscraper
large natural outdoor scenes cloud, forest, mountain, plain, sea
large omnivores and herbivores camel, cattle, chimpanzee, elephant, kangaroo
medium-sized mammals fox, porcupine, possum, raccoon, skunk
non-insect invertebrates crab, lobster, snail, spider, worm
people baby, boy, girl, man, woman
reptiles crocodile, dinosaur, lizard, snake, turtle
small mammals hamster, mouse, rabbit, shrew, squirrel
trees maple, oak, palm, pine, willow
vehicles 1 bicycle, bus, motorcycle, pickup truck, train
vehicles 2 lawn-mower, rocket, streetcar, tank, tractor
thinkh commented 6 years ago

Suggestion After selecting a run that contains more than 10 classes, we can show a Select2 containing all classes and using the groups for the superclasses. The user can select maximum 10 from the Select2, which will be shown in the conf matrix.

grafik

However, this approach is limited to a hierarchy of one depth. In case of a deeper hierarchy we need a more sophisticated approach for example a cut of the hierarchy.

NMO13 commented 6 years ago

For a depth > 1, we need something like our original approach: image

NMO13 commented 6 years ago

We could show each run as a table. Each table consists of a column with the class names as well as different kind of measures. The measure columns can be used for sorting. The top 10 rows can be selected and loaded into the confusion matrix. We could maybe even use lineup for the tables.

img_2323

NMO13 commented 6 years ago

As thinkh suggested, we could show the class chooser as popup and use lineup/taggle for class selection.

img_2324

NMO13 commented 6 years ago

We could calculate cluster metrics as hendrik suggested and show them as additional columns. This way the user could select classes which have interesting cluster similarities.

NMO13 commented 6 years ago

@gfrogat had the idea of showing a "big" confusion matrix with all classes rather than a taggle-like structure as overview. The user could select interesting classes by selecting regions. Matrix reordering would allow to create clusters. The classes could also be sorted by some criteria. I think the idea is really good, however its a lot of work to implement.

thinkh commented 6 years ago

Out of scope for the paper. Hence, moved to icebox.