AttributeError: 'NoneType' object has no attribute 'serialize'

[victorhqx]

when I try to use torch2trt in yolactedge evaluation:

python eval.py --trained_model=../Downloads/yolact_edge_54_800000.pth --score_threshold=0.8 --top_k=100 --image=test_Color.jpg --use_tensorrt_safe_mode
Config not specified. Parsed yolact_edge_config from the file name.

[04/30 15:22:51 yolact.eval]: Loading model...
[04/30 15:22:56 yolact.eval]: Model loaded.
[04/30 15:22:56 yolact.eval]: Converting to TensorRT...
WARNING [04/30 15:22:56 yolact.eval]: Running TensorRT in safe mode. This is an attempt to solve various TensorRT engine errors.
[04/30 15:22:58 yolact.eval]: Converting backbone to TensorRT...
[TensorRT] ERROR: ../rtSafe/cublas/cublasWrapper.cpp (71) - cuBLAS Error in CublasWrapper: 1 (Could not initialize cublas. Please check CUDA installation.)
[TensorRT] ERROR: ../rtSafe/cublas/cublasWrapper.cpp (71) - cuBLAS Error in CublasWrapper: 1 (Could not initialize cublas. Please check CUDA installation.)
Traceback (most recent call last):
  File "eval.py", line 1275, in <module>
    convert_to_tensorrt(net, cfg, args, transform=BaseTransform())
  File "/home/hqx/yolact_edge/utils/tensorrt.py", line 156, in convert_to_tensorrt
    net.to_tensorrt_backbone(cfg.torch2trt_backbone_int8, calibration_dataset=calibration_dataset, batch_size=args.trt_batch_size)
  File "/home/hqx/yolact_edge/yolact.py", line 1501, in to_tensorrt_backbone
    self.trt_load_if("backbone", trt_fn, [x], int8_mode, batch_size=batch_size)
  File "/home/hqx/yolact_edge/yolact.py", line 1484, in trt_load_if
    self.save_trt_cached_module(module, module_name, int8_mode, batch_size=batch_size)
  File "/home/hqx/yolact_edge/yolact.py", line 1475, in save_trt_cached_module
    torch.save(module.state_dict(), module_path)
  File "/home/hqx/miniconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1078, in state_dict
    hook_result = hook(self, destination, prefix, local_metadata)
  File "/home/hqx/miniconda3/lib/python3.8/site-packages/torch2trt-0.2.0-py3.8-linux-x86_64.egg/torch2trt/torch2trt.py", line 425, in _on_state_dict
    state_dict[prefix + "engine"] = bytearray(self.engine.serialize())
AttributeError: 'NoneType' object has no attribute 'serialize'

What is the problem?

[henbucuoshanghai]

AttributeError: 'NoneType' object has no attribute 'serialize'

why

[nhewitt99]

The AttributeError isn't the main problem here; the actual error is occurring at the TensorRT cublas lines. When the TRT engine can't be initialized (in this case, because of cublas issues), it returns None, causing the AttributeError downstream.

The "Could not initialize cublas" error has two main causes: first, you may have installed CUDA incorrectly, or installed a version of CUDA incompatible with your driver. Make sure your driver, CUDA, cudnn, and TensorRT versions are all compatible by checking their official installation guides. The second main culprit behind a bad cublas init is an out-of-memory error (which isn't made clear by the given error message). I've noticed some other people complain that yolact edge has large memory demands, so I would suspect this is your problem (unless you're using a GPU with >4GB of VRAM).

Also, if this is an issue specific to yolact edge, I would suggest posting an issue on their repo instead. After all, it appears that this error occurs inside of their eval.py.

[mrpositron]

CUDA OOM was a problem in my case! Thanks @nhewitt99 !

[arunmack789]

CUDA OOM was a problem in my case! Thanks @nhewitt99 !

@nhewitt99 i can't fix error torch.save(model_trt.state_dict(),"model.trt") please help me to fix in jetson nano