search

vinthony / ghost-free-shadow-removal

[AAAI 2020] Towards Ghost-free Shadow Removal via Dual Hierarchical Aggregation Network and Shadow Matting GAN
https://arxiv.org/abs/1911.08718
297 stars 59 forks source link

楼主你好，大分辨率上测试的bug #5

Closed ersanliqiao closed 4 years ago

ersanliqiao commented 4 years ago

2020-01-06 13:21:24.480792: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcublas.so.10.0 2020-01-06 13:21:24.742605: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudnn.so.7 2020-01-06 13:21:28.135955: F tensorflow/stream_executor/cuda/cuda_dnn.cc:86] Check failed: narrow == wide (-1236407296 vs. 3058560000)checked narrowing failed; values not equal post-conversion

ersanliqiao commented 4 years ago

1920x1080或1600x1200

vinthony commented 4 years ago

我并没有复现出您所说的问题，我在demo里试了试运行大的图像（1920x1080),得到以下结果:

ResourceExhaustedError: 2 root error(s) found.
  (0) Resource exhausted: OOM when allocating tensor with shape[1,1080,1920,963] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
     [[{{node concat_3}}]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

     [[g_conv_img/BiasAdd/_415]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

  (1) Resource exhausted: OOM when allocating tensor with shape[1,1080,1920,963] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
     [[{{node concat_3}}]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

0 successful operations.
0 derived errors ignored.

During handling of the above exception, another exception occurred:

看起来是因为模型比较大，超过了显存的限制。

  1. 您可以尝试减小channel的数量并重新训练
  2. 对于比较大的图片, 可以尝试使用 guided image filter 或者deep guided image filter
ersanliqiao commented 4 years ago

楼主你好，问题定位出来了，是VGG19输出的几个feature map concat一起以后，整个维数超过了unsigned int32的范围，比如1920X1080，1920X1080X1475=3525120000> 2147483647。 另外，请教一下： https://github.com/vinthony/ghost-free-shadow-removal/blob/c2db03ffbeabb39eb45b64f628283789a3d35d7b/networks.py#L146 这行对feature map上采样以后除以255，为何需要这么做？我理解的是网络能自适应调整范围。

vinthony commented 4 years ago

您好，神经网络是可以自适应调整这些。 但是这样做的目的是让input中feature的数值范围 和 原始图像的数值范围 之间存在一定的比例。关于使用VGG16当做hyper-column，可以参照基准网络的代码Perceptual-refection-removal

ersanliqiao commented 4 years ago

哦哦赞，谢谢楼主