search

jwyang / lr-gan.pytorch

Pytorch code for our ICLR 2017 paper "Layered-Recursive GAN for image generation"
151 stars 32 forks source link

problem when importing stnm #1

Closed joemzhao closed 7 years ago

joemzhao commented 7 years ago

hi jianwei, thanks a lot for sharing the work!

i meet a problem when importing the stnm:

ImportError: dlopen(lr-gan.pytorch-master/_ext/stnm/_stnm.so, 2): no suitable image found. Did find: /lr-gan.pytorch-master/_ext/stnm/_stnm.so: unknown file type, first eight bytes: 0x7F 0x45 0x4C 0x46 0x02 0x01 0x01 0x00

i tried both on my mac and a linux server with cuda installed, this problem still raised. im wondering if you have any ideas about this? i also tried to build your stnm.pytorch project where i met this problem too.

thanks!

jwyang commented 7 years ago

Hi, @joemzhao , which system and platform are you using? It seems that the .so file cannot be loaded. You remake the so file by yourself right? You could make it but cannot load it successfully, right?

joemzhao commented 7 years ago

Hi @jwyang , thanks for the quick reply! Yes indeed, the error is raised when loading the .so: https://github.com/jwyang/lr-gan.pytorch/blob/master/functions/stnm.py#L4

I tried to remake (on server) and it passed with a few unused-variable warnings. However the test is failed: python test.py Variable containing: (0 ,.,.) = 0.8000 0.3000 1.0000 0.5000 0.0000 0.0000 [torch.FloatTensor of size 1x2x3]

Traceback (most recent call last): File "test.py", line 30, in out, aux = g(input) File "/home/chen/LRGAN/local/lib/python2.7/site-packages/torch/nn/modules/module.py", line 224, in call result = self.forward(*input, **kwargs) File "/home/chen/stnm/script/modules/gridgen.py", line 30, in forward loss = torch.sum(loss,2) File "/home/chen/LRGAN/local/lib/python2.7/site-packages/torch/autograd/variable.py", line 476, in sum return Sum.apply(self, dim, keepdim) File "/home/chen/LRGAN/local/lib/python2.7/site-packages/torch/autograd/_functions/reduce.py", line 21, in forward return input.sum(dim) RuntimeError: dimension out of range (expected to be in range of [-2, 1], but got 2)

the error was raised when calling gridgen, which is not related to the .so file I guess.

Some specs:

On my local machine: pytorch 0.2.0_1 python 2.7.10 OS X 10.11.6

On server: pytorch 0.2.0_1 python 2.7.12 CUDA Version 8.0.61 Ubuntu 16.04.2 LTS

In addition, importing the .so also failed on another mac (macOS 10.12.3, python 3.6.2)

jwyang commented 7 years ago

Hi, @joemzhao , I see, I think you can comment the loss computation in gridgen, we will not use it in lr-gan code. I will also make change to stnm repository.

joemzhao commented 7 years ago

Hi @jwyang , thank you for your reply.

I managed to load the .so file successfully, I think the problem was from my mismatch between building and loading locally and remotely.

Regarding gridgen.py yes we can comment the loss calculation. In addition, we need to comment the testing for other transforms other than the affine one. They also cause failure when running test.py.

Also I think it would be better to notify people that stnm should be rebuilt in their local machines since the one provided in this repo would throw invalid device since it was build on your machine (i guess).

Lastly, I do have two questions about the spatial transformer:

  1. Could you give some brief ideas about how the down-sampling for the foreground is done? I am writing this project in tensorflow and unfortunately I dont really have time to go through the CUDA codes. Can we use dense layers or pooling to achieve this, though the image quality may be affected?

  2. Could you give some information about where the compositing process (i guess is alpha-blending) is implemented? Maybe I missed it when I went through the codes...

Thanks a lot!

jwyang commented 7 years ago

Hi, @joemzhao , thanks for the information and suggestions.

I will clean up the useless code in STNM project, and make a detailed writeup to explain the process today. Will let you know once it is finished.

joemzhao commented 7 years ago

@jwyang Thank you!

jwyang commented 7 years ago

Hi, @joemzhao , I have updated the STNM readme and code. Regarding your questions:

  1. Could you give some brief ideas about how the down-sampling for the foreground is done? I am writing this project in tensorflow and unfortunately I dont really have time to go through the CUDA codes. Can we use dense layers or pooling to achieve this, though the image quality may be affected?

I implemented a down sampler for foreground image and mask, but found it did not work better than the original bilinear upsampler. As such, I actually used the same sampler as STN. I think it is okay to use dense layers or pooling for the down sampling. However, it seems that these kind of layers are hard to get images with specific transformation (scale, rotation and translation)?

  1. Could you give some information about where the compositing process (i guess is alpha-blending) is implemented? Maybe I missed it when I went through the codes...

Yes, I used alpha blending. The core code is located at stem_cuda_kernel.cu. There are two functions:

  1. bilinearSamplingFromGrid, for forwarding

  2. backwardBilinearSampling, for back warding

Please let me know if you have any further questions.

thanks,