SeanNaren / warp-ctc

Pytorch Bindings for warp-ctc
Apache License 2.0
756 stars 271 forks source link

Cant print Grad tensor from CTCLoss function #183

Open adhiraj69 opened 3 years ago

adhiraj69 commented 3 years ago

when ever i try to print the gradient tensor obtained from gpu_ctc i am using pytorch and i am editing the warpctc_pytorch init.py

in the forward function i tried printing the grads tensor but i encounter the following error

i encounter this error

THCudaCheck FAIL file=/pytorch/aten/src/THC/THCCachingHostAllocator.cpp line=278 error=700 : an illegal memory access was encountered Traceback (most recent call last): File "CTC_loss_2.py", line 765, in loss1 = ctcloss(x1, labels, inp1,targs1) File "/home/dzilla69/anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 722, in _call_impl result = self.forward(*input, **kwargs) File "/home/dzilla69/anaconda3/lib/python3.8/site-packages/warpctc_pytorch-0.1-py3.8-linux-x86_64.egg/warpctc_pytorch/init.py", line 83, in forward return self.ctc(acts, labels, act_lens, label_lens, self.size_average, File "/home/dzilla69/anaconda3/lib/python3.8/site-packages/warpctc_pytorch-0.1-py3.8-linux-x86_64.egg/warpctc_pytorch/init.py", line 46, in forward print(grads) File "/home/dzilla69/anaconda3/lib/python3.8/site-packages/torch/tensor.py", line 153, in repr return torch._tensor_str._str(self) File "/home/dzilla69/anaconda3/lib/python3.8/site-packages/torch/_tensor_str.py", line 371, in _str return _str_intern(self) File "/home/dzilla69/anaconda3/lib/python3.8/site-packages/torch/_tensor_str.py", line 351, in _str_intern tensor_str = _tensor_str(self, indent) File "/home/dzilla69/anaconda3/lib/python3.8/site-packages/torch/_tensor_str.py", line 241, in _tensor_str formatter = _Formatter(get_summarized_data(self) if summarize else self) File "/home/dzilla69/anaconda3/lib/python3.8/site-packages/torch/_tensor_str.py", line 273, in get_summarized_data return torch.stack([get_summarized_data(x) for x in (start + end)]) File "/home/dzilla69/anaconda3/lib/python3.8/site-packages/torch/_tensor_str.py", line 273, in return torch.stack([get_summarized_data(x) for x in (start + end)]) File "/home/dzilla69/anaconda3/lib/python3.8/site-packages/torch/_tensor_str.py", line 275, in get_summarized_data return torch.stack([get_summarized_data(x) for x in self]) File "/home/dzilla69/anaconda3/lib/python3.8/site-packages/torch/_tensor_str.py", line 275, in return torch.stack([get_summarized_data(x) for x in self]) File "/home/dzilla69/anaconda3/lib/python3.8/site-packages/torch/_tensor_str.py", line 266, in get_summarized_data return torch.cat((self[:PRINT_OPTS.edgeitems], self[-PRINT_OPTS.edgeitems:])) RuntimeError: cuda runtime error (700) : an illegal memory access was encountered at /pytorch/aten/src/THC/THCCachingHostAllocator.cpp:278