error in correlation_forward_cuda_kernel: no kernel image is available for execution on the devic

[neerajkuhike]

I am getting below error:

Traceback (most recent call last): File "train.py", line 148, in train() File "train.py", line 60, in train flow_ref, conf_ref = flowNet(real_B, real_B_prev) # reference flows and confidences
File "/home/neerajku/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in call result = self.forward(*input, kwargs) File "/home/neerajku/neerajAvatar/vid2vid/models/flownet.py", line 38, in forward flow, conf = self.compute_flow_and_conf(input_A, input_B) File "/home/neerajku/neerajAvatar/vid2vid/models/flownet.py", line 54, in compute_flow_and_conf flow1 = self.flowNet(data1) File "/home/neerajku/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in call result = self.forward(*input, *kwargs) File "/home/neerajku/neerajAvatar/vid2vid/models/flownet2_pytorch/models.py", line 105, in forward flownetc_flow2 = self.flownetc(x)[0] File "/home/neerajku/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in call result = self.forward(input, kwargs) File "/home/neerajku/neerajAvatar/vid2vid/models/flownet2_pytorch/networks/FlowNetC.py", line 89, in forward out_corr = self.corr(out_conv3a, out_conv3b) # False File "/home/neerajku/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in call result = self.forward(*input, **kwargs) File "/home/neerajku/neerajAvatar/vid2vid/models/flownet2_pytorch/networks/correlation_package/correlation.py", line 59, in forward result = CorrelationFunction(self.pad_size, self.kernel_size, self.max_displacement,self.stride1, self.stride2, self.corr_multiply)(input1, input2) File "/home/neerajku/neerajAvatar/vid2vid/models/flownet2_pytorch/networks/correlation_package/correlation.py", line 27, in forward self.pad_size, self.kernel_size, self.max_displacement,self.stride1, self.stride2, self.corr_multiply) RuntimeError: CUDA call failed (correlation_forward_cuda at correlation_cuda.cc:82) frame #0: std::function<std::string ()>::operator()() const + 0x11 (0x7fe3f3b21021 in /home/neerajku/anaconda3/lib/python3.7/site-packages/torch/lib/libc10.so) frame #1: c10::Error::Error(c10::SourceLocation, std::string const&) + 0x2a (0x7fe3f3b208ea in /home/neerajku/anaconda3/lib/python3.7/site-packages/torch/lib/libc10.so) frame #2: correlation_forward_cuda(at::Tensor&, at::Tensor&, at::Tensor&, at::Tensor&, at::Tensor&, int, int, int, int, int, int) + 0x6e7 (0x7fe3e0099cb7 in /home/neerajku/.local/lib/python3.7/site-packages/correlation_cuda-0.0.0-py3.7-linux-x86_64.egg/correlation_cuda.cpython-37m-x86_64-linux-gnu.so) frame #3: + 0x17e54 (0x7fe3e00a5e54 in /home/neerajku/.local/lib/python3.7/site-packages/correlation_cuda-0.0.0-py3.7-linux-x86_64.egg/correlation_cuda.cpython-37m-x86_64-linux-gnu.so) frame #4: + 0x1808e (0x7fe3e00a608e in /home/neerajku/.local/lib/python3.7/site-packages/correlation_cuda-0.0.0-py3.7-linux-x86_64.egg/correlation_cuda.cpython-37m-x86_64-linux-gnu.so) frame #5: + 0x14c25 (0x7fe3e00a2c25 in /home/neerajku/.local/lib/python3.7/site-packages/correlation_cuda-0.0.0-py3.7-linux-x86_64.egg/correlation_cuda.cpython-37m-x86_64-linux-gnu.so)

frame #12: THPFunction_do_forward(THPFunction*, _object*) + 0x15c (0x7fe42e0610ac in /home/neerajku/anaconda3/lib/python3.7/site-packages/torch/lib/libtorch_python.so) configuration : cuda - 10.2 pytorch- 1.3 python -3.7 V100 server configuration in three files: nvcc_args = [ '-gencode', 'arch=compute_30,code=sm_30', '-gencode', 'arch=compute_35,code=sm_35', '-gencode', 'arch=compute_37,code=sm_37', '-gencode', 'arch=compute_52,code=sm_52', '-gencode', 'arch=compute_60,code=sm_60', '-gencode', 'arch=compute_61,code=sm_61', '-gencode', 'arch=compute_70,code=sm_70', ' -gencode','arch=compute_75,code=sm_75' '-gencode', 'arch=compute_70,code=compute_70' ]