search

cc-ai / climategan

Code and pre-trained model for the algorithm generating visualisations of 3 climate change related events: floods, wildfires and smog.
https://thisclimatedoesnotexist.com
GNU General Public License v3.0
72 stars 18 forks source link

Rotation aug implementation #150

Closed tianyu-z closed 3 years ago

tianyu-z commented 3 years ago

I guess the most weird part of the rotation implementation is to rotate the segmentation pts. This is the logic of how it works. Hope this graph can help you review the code. image

tianyu-z commented 3 years ago

Besides, an option for HRNet Normalization is added. Please check opts.data.normalization in defaults.yaml

tianyu-z commented 3 years ago

Got some issues after merging master into this branch. Trying to fix......

vict0rsch commented 3 years ago

@tianyu-z I understood from your presentation that rotation tools induce noise in the pixel values.

Have you researched extensively existing tools and techniques? Other packages like skimage or opencv? Tried other types for the data before the rotation?

If so, then we should keep your current implementation as it seems to work.

I'd be interested in measuring the loading time overhead: can you check the time this particular transform takes on an image (vs other transforms, to understand if this adds 20% calculations or 2000%)?

tianyu-z commented 3 years ago

@tianyu-z I understood from your presentation that rotation tools induce noise in the pixel values.

Have you researched extensively existing tools and techniques? Other packages like skimage or opencv? Tried other types for the data before� the rotation?

If so, then we should keep your current implementation as it seems to work.

I'd be interested in measuring the loading time overhead: can you check the time this particular transform takes on an image (vs other transforms, to understand if this adds 20% calculations or 2000%)?

@vict0rsch I tried skimage.transform.rotate which introduced even more noise than the scipy one which is not acceptable in segmentation data. In detail: In the top20 Counter.keys, we can see some kind of numbers like 1.9999996 in Scipy and numbers like 7.4xxx are very rare in Scipy rotator. For skimage.transform.rotator, it gives a 6.45xxx in top6 Counter.keys which I think is not good. (Yes, it's faster.)

For cv2, I think they only provide rotations of multiplication of 90 degrees. Torch rotation is really slow when it works on normal pictures, but it's good on segmentation data.

When it comes to time elapsed, I was very careful about not blowing up things. Hope everything will be good. I am running an exp with norot and an exp with rot. For bs = 6, in the rot exp I got step time ranging from 4.87 to 5.23 (The prob to trigger rot was set to 0.8.). In the norot exp, I got step time ranging from 4.77 to 5.09. I will double-check everything after I get up.

tianyu-z commented 3 years ago

An example about the results. image

tianyu-z commented 3 years ago

rot exp: https://www.comet.ml/tianyu-z/omnigan/beab426b94c04843a86696f6a2e15eac?experiment-tab=metrics no rot exp: https://www.comet.ml/tianyu-z/omnigan/f6e53d7ea74f47deb488dab58f6e2431?experiment-tab=metrics

Comet.ml - Supercharging Machine Learning
Comet.ml - Supercharging Machine Learning
Comet lets you track code, experiments, and results on ML projects. It’s fast, simple, and free for open source projects.
Comet.ml - Supercharging Machine Learning
Comet.ml - Supercharging Machine Learning
Comet lets you track code, experiments, and results on ML projects. It’s fast, simple, and free for open source projects.
vict0rsch commented 3 years ago

Ok perfect, GPU utilization is still close to 100% so I'm good with this, you just need to add comments and docstrings before we can merge

tianyu-z commented 3 years ago

Ok perfect, GPU utilization is still close to 100% so I'm good with this, you just need to add comments and docstrings before we can merge

Thanks!