Improve show_preds

[lgvaz]

🚀 Feature

If targets are available, we could also show how the predictions compare to the targets.

Maybe a side by side view? Or maybe a difference view between targets and predictions.

Maybe something like this (from this article):

[oke-aditya]

Also, currently we are drawing masks with matplotlib and stuff, I think its we can go with simply PIL. This is a feature request in torchvision too. Link here Maybe we can contribute there itself. Torchvision already has make_grid function etc.

[FraPochetti]

Hola amigos! Is this one still open? If yes, can I take it?

[lgvaz]

Hey hey @FraPochetti !!

You got it!!!

[lgvaz]

You already know, but just as a reminder, if you need any help, feel free to ask 😄

[FraPochetti]

Quick clarification on my side though. I am not sure I will be able to work on this super quick, given I will also leave one week on vacation next Monday. Is this OK with you guys? If that's urgent, I can try to figure something out.

[lgvaz]

It's totally okay Francesco, it's not urgent at all you can take your time and again feel free to ask for help as I don't consider this a trivial problem

The functions around this issue are still a little bit on the experimental side, the main two files you'll be needing to take a look are draw_data.py and show_data.py, show_annotation.py is deprecated and it'll be removed

We can discuss ideas on what would be good presentations of data before jumping to code, that way everyone can voice out their opinion =)

[FraPochetti]

Awesome man! Will take a look and revert back to the community. Hasta la vista ! 😎

[FraPochetti]

I looked into the dedicated functions and I think that a side-by-side pred VS actual is the best. Basically transposing the 6 images below (the 3 first records + preds from the validation set) to show 3 pred VS actual pairs. I am not sure the pred VS actual difference could help. It can get pretty crowded and very difficult to read. Already in the below case, it is not so immediate to spot pred VS actual deltas, as there are quite a few bounding boxes. Thoughts?

[FraPochetti]

Also, to avoid the unaesthetic black top-bottom stripes in preds, shall we resize imgs to their original size? It could be an additional (optional) step of the denormalize_fn function.

[lgvaz]

I am not sure the pred VS actual difference could help. It can get pretty crowded and very difficult to read.

Yeaah, I definitly agree with that. The only strong conclusion we can draw from the images is that there is a lot more predictions than ground truths, but I'm not sure what to make of that.

Already in the below case, it is not so immediate to spot pred VS actual deltas

Yes! And it's not so straightforward to match the ground truth with its corresponded prediciton, can you imagine a way of having the "difference" view for bounding boxes (like the one in the OP)?

[lgvaz]

Also, to avoid the unaesthetic black top-bottom stripes in preds, shall we resize imgs to their original size?

I think this will be a bit more complicated, the stripes comes from the padding and those are the images that the model actually sees. Take a look at the last image for example, on the right bottom corner there is a box that even enters the grey area.

[FraPochetti]

I think this will be a bit more complicated, the stripes come from the padding and those are the images that the model actually sees

Got it. Is the padding coming from the model directly (e.g. as a conv artifact)? I don't see it as a pre-processing step.

but I'm not sure what to make of that

Me neither. It is quite confusing actually as I am tempted to claim that the model sucks, but it is a very personal and qualitative perspective.

can you imagine a way of having the "difference" view for bounding boxes (like the one in the OP)?

I don't think there is a good way of doing that. The only thing coming to my mind is to display the image + the ground truths bboxes in, say, blue + the predictions bboxes in, say, red. No labels, as it would clutter the already crowded viz. This of course tells us if the bboxes are reasonable, but it gives no info on the labels.

I think side-by-side is the only interesting thing to display. What do you think?

[lgvaz]

Is the padding coming from the model directly (e.g. as a conv artifact)?

It's coming from albumentations, for the training data it comes from aug_tfms and for the validation data it comes from resize_and_pad

I think side-by-side is the only interesting thing to display.

Agree! That's the only one that is general enough as well

[FraPochetti]

Cool man, I will start working on it then

[FraPochetti]

I am leaving on vacation tomorrow returning on Tuesday 20. Unfortunately, I didn't have enough time to start any coding here, even though, at least, the concept has been discussed and agreed upon. I will get back onto it upon my return!

[FraPochetti]

How about this viz? What do you think?

[lgvaz]

I am leaving on vacation tomorrow returning on Tuesday 20

Have a good time Francesco, no worries here, feel free to get back once you can =)

What do you think?

Looks good! We can cleary see what images the model got right

[FraPochetti]

PR created here

[FraPochetti]

@lgvaz can we close this one?

[lgvaz]

Yes! Thanks for the heads up!