Unable to recreate GTA <-> Cityscapes translation (CyCADA)

[mikkelmk]

We tried to recreate the results of GTA <-> Cityscapes using the following parameters as recommended in the Torch repo:

--preprocess scale_width_and_crop
--load_size 1024
--crop_size 360

--lambda_identity 1

--n_epochs 10
--n_epochs_decay 10

We used all 24966 images from the Playing for Data GTA dataset and all 19998 images from the Cityscapes training set.

Yet, we're not able to recreate the results seen in the precomputed images provided here. Three bad examples/comparisons are shown below.

We are posting in the hope that someone have an idea what could be causing the difference in results.

We noticed that you state in the FAQ that your PyTorch and Torch implementation seemed to produce a little bit different results for the GTA <-> Cityscapes translation. Is this difference the one we are seeing in our results? If so, do you have any insight as to what causes such a difference between the two repos? We'd really like to be able to recreate the results with the PyTorch version, as we have no experience with neither Lua nor Torch (and we have a lot of problems getting the original version of Torch to even run on our system).

Example 1

Original GTA V:

Our translation:

Provided precomputed image:

Example 2

Original GTA V:

Our translation:

Provided precomputed image:

Example 3

Original GTA V:

Our translation:

Provided precomputed image:

[rdbch]

From what it seems, I suppose that they have applied a post-processing step, namely a guided filter (having the original untranslated image as guide) for removing the random artifacts. As far as I know, this approach is quite common for smoothing photorealistic translations. As an example, I first seen it used like this here, A Closed-form Solution to Photorealistic Image Stylization, and the code is here .

Hope it helps.

[junyanz]

Photorealistic image stylization was not used in the original work. But it might help you remove the artifacts.

[shreyas23]

@mikkelmk Were you able to figure out why this happened? Any hyperparameter suggestions?

[nanditam1]

Hi, was any solution found for this?