YoungXIAO13 / PoseFromShape

(BMVC 2019) PyTorch implementation of Paper "Pose from Shape: Deep Pose Estimation for Arbitrary 3D Objects"
http://imagine.enpc.fr/~xiaoy/PoseFromShape/
MIT License
170 stars 34 forks source link

TransLightning explanation #19

Closed ajuric closed 4 years ago

ajuric commented 4 years ago

Hi,

What is this transformation doing: https://github.com/YoungXIAO13/PoseFromShape/blob/master/auxiliary/dataset.py#L15 ? From the names of the variable, I see that it uses PCA, but don't understand the approach ...

I have tried to google it (application of methods using PCA on image preprocessing), didn't have success ...

Maybe just a link describing this technique would be enough?

Also, I have visualized the images after this transformation, and I see that it adds unreal artifacts to image (small regions of pixels with totally different colors). Also, my visualization was on images that weren't normalized, and you in the code first do the normalization and then apply this transformation (here: https://github.com/YoungXIAO13/PoseFromShape/blob/master/auxiliary/dataset.py#L202). Do you know what's the effect of this transformation on normalized images (since normalized images cannot be visualized)?

YoungXIAO13 commented 4 years ago

Hi,

The TransLighting is basically a Gaussian noise added to the three channels of the normalized image tensors. And the ColorJiterring is a random transformation on the RGB channels of the original images.

These two transformations were used to improve the network's generalization ability when trained and tested on different data (Synthetic to Real images, etc)

More data augmentation skills can be found here: https://github.com/aleju/imgaug