facebookresearch / DeltaCNN

DeltaCNN End-to-End CNN Inference of Sparse Frame Differences in Videos
Other
60 stars 6 forks source link

RuntimeError using delta_cudnn mode #8

Closed kai981 closed 1 year ago

kai981 commented 1 year ago

Hi! I encountered a runtime error when I tried to run in delta_cudnn mode, as shown below:

Traceback (most recent call last):
  File "/home/efficientdet/validate_dc.py", line 258, in <module>
    main()
  File "/home/efficientdet/validate_dc.py", line 254, in main
    validate(args)
  File "/home/efficientdet/validate_dc.py", line 197, in validate
    class_out_dc, box_out_dc = dc_model(input)
  File "/home/anaconda3/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/efficientdet/effdet/efficientdet_deltacnn.py", line 598, in forward
    x = self.sparsify(x) ## added 
  File "/home/liandongze/anaconda3/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/efficientdet/src/deltacnn/sparse_layers.py", line 1260, in forward
    self.prev_in[tiled_mask] = input
RuntimeError: shape mismatch: value tensor of shape [3, 640, 640] cannot be broadcast to indexing result of shape [461448]

The same code has no issue running in cudnn mode and deltacnn mode. This error seems to occur when it's running on the second frame, any idea on what might be the problem?

Thank you!

dabeschte commented 1 year ago

Should be fixed. Please let me know, I did not test it

kai981 commented 1 year ago

Hi! The previous Runtime error is fixed, but i encountered another error as shown:

Traceback (most recent call last):
  File "/home/efficientdet/validate_dc.py", line 258, in <module>
    main()
  File "/home/efficientdet/validate_dc.py", line 254, in main
    validate(args)
  File "/home/efficientdet/validate_dc.py", line 197, in validate
    class_out_dc, box_out_dc = dc_model(input)
  File "/home/anaconda3/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/efficientdet/effdet/efficientdet_deltacnn.py", line 598, in forward
    x = self.sparsify(x) ## added 
  File "/home/anaconda3/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/efficientdet/src/deltacnn/sparse_layers.py", line 1258, in forward
    tiled_mask = torch.repeat_interleave(self.mask, dim=1, repeats=x.shape[1])
TypeError: repeat_interleave() received an invalid combination of arguments - got (NoneType, repeats=int, dim=int), but expected one of:
 * (Tensor input, Tensor repeats, int dim, *, int output_size)
 * (Tensor input, int repeats, int dim, *, int output_size)
 * (Tensor repeats, *, int output_size)
      didn't match because some of the keywords were incorrect: repeats, dim
dabeschte commented 1 year ago

should be fixed. but delta_cudnn is really not tested anymore and might not work as expected. it did not prove useful in the end and was mainly used for debugging