search

visinf / self-mono-sf

Self-Supervised Monocular Scene Flow Estimation (CVPR 2020)
Apache License 2.0
248 stars 47 forks source link

Question about Compatibility for Pytorch >= 1.6.0 #13

Closed SwagJ closed 3 years ago

SwagJ commented 3 years ago

Hi @hurjunhwa,

I am just wondering have you tested the compatibility of correlation package with Pytorch >=1.6.0? For Pytorch >=1.6.0, the cuda version supported is 10.1 & 10.2, not 10.0. If I used torch == 1.6.0+cu101 & cuda 10.1. It gives follows error in runtime:

File "/disk_ssd/self-mono-sf/models/correlation_package/correlation.py", line 4, in import correlation_cuda ImportError: libcudart.so.10.0: cannot open shared object file: No such file or directory

For this error, I am not sure why in correlation_cuda will try to find libcudart.so.10.0 instead of libcudart.so.10.1

If using torch == 1.6.0+cu101 & cuda10.0, it gives this error in runtime:

File "/disk_ssd/self-mono-sf/models/correlation_package/correlation.py", line 4, in import correlation_cuda ImportError: /home/hashswan/.conda/envs/self-mono/lib/python3.7/site-packages/correlation_cuda-0.0.0-py3.7-linux-x86_64.egg/correlation_cuda.cpython-37m-x86_64-linux-gnu.so: undefined symbol: _ZN6caffe26detail37_typeMetaDataInstance_preallocated_32E

For this error, it resulted from incompatibility of pytorch version and cuda version

In both cases, package installation is successful without any warnings.

I am looking forward to your reply. Thank you very much!

Best,

hurjunhwa commented 3 years ago

Hi, Thanks for sharing! Yes, I also had a problem with PyTorch 1.6.0, but didn't dig into it yet..

As a preliminary solution, you can use this pure python code for calculating the cost volume:

out_corr_f = compute_cost_volume(x1, x2_warp, self.search_range)

where self.search_range = 4

# https://github.com/google-research/google-research/blob/789d828d545dc35df8779ad4f9e9325fc2e3ceb0/uflow/uflow_model.py#L88
def compute_cost_volume(feat1, feat2, max_disp):

    _, _, height, width = feat1.size()
    num_shifts = 2 * max_disp + 1
    feat2_padded = tf.pad(feat2, (max_disp, max_disp, max_disp, max_disp), "constant", 0)

    cost_list = []
    for i in range(num_shifts):
        for j in range(num_shifts):
            corr = torch.mean(feat1 * feat2_padded[:, :, i:(height + i), j:(width + j)], axis=1, keepdims=True)
            cost_list.append(corr)
    cost_volume = torch.cat(cost_list, axis=1)
    return cost_volume

Please note that the code is from https://github.com/google-research/google-research/tree/master/uflow and check their licence conditions as well.

This code is a bit slower than using the CUDA code but not so significantly :)

Best,

SwagJ commented 3 years ago

Hi, @hurjunhwa ,

Thank you for your reply.