NeuromorphicProcessorProject / snn_toolbox

Toolbox for converting analog to spiking neural networks (ANN to SNN), and running them in a spiking neuron simulator.
MIT License
362 stars 105 forks source link

Load model from snn_data/yolo3/yolov3_ckpt_99.onnx failed:Fatal error: ATen is not a registered function/op #109

Closed JNaranjo-Alcazar closed 3 years ago

JNaranjo-Alcazar commented 3 years ago
Found 2 images belonging to 1 classes.
Found 2 images belonging to 1 classes.
(3, 416, 416)

2021-09-01 08:05:56.318982: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2021-09-01 08:05:56.318999: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
snn_data/yolo3/yolov3_ckpt_99.py:321: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if grid_size != self.grid_size:
snn_data/yolo3/yolov3_ckpt_99.py:325: TracerWarning: Converting a tensor to a Python index might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  pred_boxes = FloatTensor(prediction[..., :4].shape)
/home/user/.local/lib/python3.6/site-packages/torch/onnx/symbolic_helper.py:243: UserWarning: You are trying to export the model with onnx:Upsample for ONNX opset version 9. This operator might cause results to not match the expected results by PyTorch.
ONNX's Upsample/Resize operator did not match Pytorch's Interpolation until opset 11. Attributes to determine how to transform the input were added in onnx:Resize in opset 11 to support Pytorch's behavior (like coordinate_transformation_mode and nearest_mode).
We recommend using opset 11 and above for models using this operator. 
  "" + str(_export_onnx_opset_version) + ". "
Warning: ATen was a removed experimental ops. In the future, we may directly reject this operator. Please update your model as soon as possible.
Warning: ATen was a removed experimental ops. In the future, we may directly reject this operator. Please update your model as soon as possible.
Warning: ATen was a removed experimental ops. In the future, we may directly reject this operator. Please update your model as soon as possible.
Warning: ATen was a removed experimental ops. In the future, we may directly reject this operator. Please update your model as soon as possible.
Warning: ATen was a removed experimental ops. In the future, we may directly reject this operator. Please update your model as soon as possible.
Warning: ATen was a removed experimental ops. In the future, we may directly reject this operator. Please update your model as soon as possible.
Warning: ATen was a removed experimental ops. In the future, we may directly reject this operator. Please update your model as soon as possible.
Warning: ATen was a removed experimental ops. In the future, we may directly reject this operator. Please update your model as soon as possible.
Warning: ATen was a removed experimental ops. In the future, we may directly reject this operator. Please update your model as soon as possible.
Warning: ATen was a removed experimental ops. In the future, we may directly reject this operator. Please update your model as soon as possible.
Warning: ATen was a removed experimental ops. In the future, we may directly reject this operator. Please update your model as soon as possible.
Warning: ATen was a removed experimental ops. In the future, we may directly reject this operator. Please update your model as soon as possible.
Warning: Checker does not support models with experimental ops: ATen
Traceback (most recent call last):
  File "/home/user/.local/bin/snntoolbox", line 33, in <module>
    sys.exit(load_entry_point('snntoolbox==0.6.0', 'console_scripts', 'snntoolbox')())
  File "/home/yser/.local/lib/python3.6/site-packages/snntoolbox/bin/run.py", line 49, in main
    run_pipeline(config)
  File "/home/user/.local/lib/python3.6/site-packages/snntoolbox/bin/utils.py", line 73, in run_pipeline
    config.get('paths', 'filename_ann'))
  File "/home/user/.local/lib/python3.6/site-packages/snntoolbox/parsing/model_libs/pytorch_input_lib.py", line 100, in load
    ort_session = onnxruntime.InferenceSession(filepath + '.onnx')
  File "/home/user/.local/lib/python3.6/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 283, in __init__
    self._create_inference_session(providers, provider_options, disabled_optimizers)
  File "/home/user/.local/lib/python3.6/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 310, in _create_inference_session
    sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Load model from /user/user/snn_data/yolo3/yolov3_ckpt_99.onnx failed:Fatal error: ATen is not a registered function/op
[paths]
path_wd = /user/user/snn_data/yolo3
dataset_path = /user/user/snn_data/datax1/
log_dir_of_current_run = /user/user/snn_data/yolo3
filename_ann = yolov3_ckpt_99

[input]
model_lib = pytorch
dataset_format = jpg

[tools]
evaluate_ann = True
convert = True
normalize = True

[simulation]
num_to_test = 2
simulator = INI
keras_backend = tensorflow
rbodo commented 3 years ago

The error is related to transfering the pytorch model to keras (the toolbox uses keras internally). This model translation is handled by the external onnx / onnx2keras tool, so I won't be able to help with this. Possible workarounds: Try to find out what the offending operation ("ATen"?) does in your model and whether it can be replaced by something that onnx can handle. Or use a Keras implementation of Yolo directly.

Unrelated to this issue, but related to using Yolo: The SNN toolbox currently supports classification tasks; object detection won't work out of the box. Several users have made the necessary modifications, but I am not aware of a public branch. See related #27 #56 for pointers on what is needed.