Closed melisandeteng closed 4 years ago
@melisandeteng I won't have time today but I'd be happy to peer-program with you tomorrow!
The weird coloring error for the masker logging has been found. The repeated image is due to the file val_r_resized_d.json, where each path for the depth task is the same ("d": "/network/tmp1/ccai/data/munit_dataset/testA_megadepth_resized/04.npy"). Working on finding why depth images are only logged once.
Wow, thanks for spotting that @alexrey88 . It was a bit of a stupid error, but on the other hand, I'm glad it's not something more important !
My pleasure !! The trainer is running right now on my side and depth seems to be logged more than one time, see this experiment. It's just that for now the predictions are all black but maybe that's normal since it's only been one epoch?
I can't see the experiment, but I'm glad it looks promising ! What was the issue with depth being logged only once ?
Oh I think you will be able to see it now, my experiments were set to private. Actually I don't know what the issue was, I just let it run for a first look and it was being logged!
Predictions after one epochs are just nan tensors for both masker and depth head ...
There was a mistake in the simulated data depth map decoder. Input was thought as having values between 0 and 1, but they were already pixel values between 0-255. The problem has been fixed
In this PR There are some issues in logging depth images from the real world (image is repeated) and then there's also a slight issue with colors in the mask images that are logged in comet. (see this experiment )