Inconsistent boundary inside and outside the hole regions

[owenzlz]

Hi, thanks for sharing the nice work and code!

I am running inference on the provided example images using the pretrained Places2 model. However, the generated images I get have pretty inconsistent pixel contrast inside and outside the holes, which are very different from the demo (same image) in the paper. I also tried many other images, and they have the same issues. I am wondering if you think I have running into some bugs when I run the model or do you have any other post-processing to smooth out the boundary? I haven't changed anything in the code yet before running the inference.

Looking forward to hearing from you. Thanks a lot!

[RenYurui]

Hi, Thanks for your interest. I think this bug may due to that the Custom CUDA module has not been correctly installed. We find that for some environments the Custom CUDA module can not be installed correctly. Please Check:

1. Whether the CUDA module is successfully installed
2. If 1 is Ture, whether the Appearance Flow module can generate correct warping results (all zeros or other strange results mean the module cannot successfully run in your experiment).

Meanwhile, there are some tips:

1. you may need to select correct 'nvcc_args' for your GPU when you build Custom CUDA Extensions. Comment or Uncomment --gencode in resample2d_package/setup.py. Please check here for details.
2. We suggest using GPUs with SM architecture higher than "SM60", bugs are found when running with GPUs: K80.
3. We suggest using PyTorch 1.0.0.

[dldms1345]

Hi, I got a same result with owenzlz. When I use pretrained model. I considered all your tips and It still make same result.

I have questions about checking list.

1. Does Custom CUDA module mean the codes of the resample package folder?

Moreover, in my case, when I test model 2, it makes error

./result_images Traceback (most recent call last): File "test.py", line 3, in main('test') File "/home/eh1404/works/StructureFlow/main.py", line 33, in main model.test() File "/home/eh1404/works/StructureFlow/src/structure_flow.py", line 209, in test outputs, flow_maps = self.flow_model.flow_forward(inputs, smooths, maps) File "/home/eh1404/works/StructureFlow/src/models.py", line 65, in flow_forward outputs, flow = self.f_gen(torch.cat((inputs, stage_1, maps),dim=1)) File "/home/eh1404/anaconda3/envs/final/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in call result = self.forward(*input, kwargs) File "/home/eh1404/works/StructureFlow/src/network.py", line 112, in forward flow_map = self.flow_column(inputs)
File "/home/eh1404/anaconda3/envs/final/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in call result = self.forward(*input, *kwargs) File "/home/eh1404/works/StructureFlow/src/network.py", line 220, in forward f_u1 = torch.cat((self.up_sample_flow0, f_x3), 1) File "/home/eh1404/anaconda3/envs/final/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in call result = self.forward(input, kwargs) File "/home/eh1404/anaconda3/envs/final/lib/python3.6/site-packages/torch/nn/modules/container.py", line 92, in forward input = module(input) File "/home/eh1404/anaconda3/envs/final/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in call result = self.forward(*input, *kwargs) File "/home/eh1404/works/StructureFlow/src/network.py", line 386, in forward x = self.transConv(x) File "/home/eh1404/anaconda3/envs/final/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in call result = self.forward(input, **kwargs) File "/home/eh1404/anaconda3/envs/final/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 757, in forward output_padding, self.groups, self.dilation) RuntimeError: cuDNN error: CUDNN_STATUS_INTERNAL_ERROR

though model 1 and 3 doesn't make that error. Instead, model 3 make segmentation fault after the result coming out.

2. This is the evaluation image during finetuning, I attach this image just for checking appearance flow result. Is there any problem in the appearance flow result?

I use pytorch 1.0.0, cudatoolkit 10.0.130, CUDA Version 10.0. GPU is Tesla V100. but GPU driver is CUDA 10.1

Thank you.

[RenYurui]

Hi! The Custom CUDA module means resample2d_package You error may due to that the Custom CUDA module is not successfully installed.

The code is tested with PyTorch 1.0.0 CUDA 9.0 and CUDA 10.0 Tesla V100

[AlextheEngineer]

Setting nvcc_args = [ '-gencode', 'arch=compute_75,code=compute_75' ] in the setup.py fixed the boundary issue for me, running with pytorch1.6.0, CUDA11.1, NVIDIA GeForce RTX 2080 Ti on Windows 10.