search

fastmachinelearning / hls4ml

Machine learning on FPGAs using HLS
https://fastmachinelearning.org/hls4ml
Apache License 2.0
1.16k stars 386 forks source link

Issue with ModelGraph function during ONNX model synthesis #983

Open sei-rquartiano opened 3 months ago

sei-rquartiano commented 3 months ago

On up-to-date main branch, running from within this docker image

Attempting to synthesize hls from ONNX model. First, I ran python code from README.md to convert Keras model without error. Then I modified the script to convert from ONNX instead (see below)

import hls4ml

# Fetch a keras model from our example repository
# This will download our example model to your working directory and return an example configuration file
config = hls4ml.utils.fetch_example_model('three_layer_keras.onnx')

# You can print the configuration to see some default parameters
print(config)

# Convert it to a hls project
hls_model = hls4ml.converters.onnx_to_hls(config)

#hls_model.build(csim=False)

# Print full list of example models if you want to explore more
hls4ml.utils.fetch_example_list()

When I run the above code, I get the following output and subsequent error message during synthesis


2024-03-18 17:48:46.474669: I tensorflow/tsl/cuda/cudart_stub.cc:28] Could not find cuda drivers on your machine, GPU will not be used.
2024-03-18 17:48:47.242971: I tensorflow/tsl/cuda/cudart_stub.cc:28] Could not find cuda drivers on your machine, GPU will not be used.
2024-03-18 17:48:47.243534: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-03-18 17:48:49.843563: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
Downloading example model files ...
{'OutputDir': 'my-hls-test', 'ProjectName': 'myproject', 'Backend': 'Vivado', 'Version': '1.0.0', 'Part': 'xcvu13p-flga2577-2-e', 'ClockPeriod': 5, 'IOType': 'io_parallel', 'HLSConfig': {'Model': {'Precision': 'ap_fixed<16,6>', 'ReuseFactor': 1}}, 'OnnxModel': 'three_layer_keras.onnx'}
Interpreting Model ...
Output layers:  ['Softmax']
Input shape: [None, 16]
Topology:
Layer name: _class__keras_layers_core_Dense__, layer type: Dense, current shape: [[None, 16]]
Layer name: Add, layer type: Add, current shape: [[None, 64]]
Layer name: Relu, layer type: Activation, current shape: [[None, 64]]
Layer name: _class__keras_layers_core_Dense__1, layer type: Dense, current shape: [[None, 64]]
Layer name: Add1, layer type: Add, current shape: [[None, 32]]
Layer name: Relu1, layer type: Activation, current shape: [[None, 32]]
Layer name: _class__keras_layers_core_Dense__2, layer type: Dense, current shape: [[None, 32]]
Layer name: Add2, layer type: Add, current shape: [[None, 32]]
Layer name: Relu2, layer type: Activation, current shape: [[None, 32]]
Layer name: _class__keras_layers_core_Dense__3, layer type: Dense, current shape: [[None, 32]]
Layer name: Add3, layer type: Add, current shape: [[None, 5]]
Layer name: Softmax, layer type: Softmax, current shape: [[None, 5]]
Creating HLS model
Traceback (most recent call last):
  File "/home/hls4ml-user/work/hls4ml/example_script.py", line 11, in <module>
    hls_model = hls4ml.converters.onnx_to_hls(config)
  File "/home/hls4ml-user/work/hls4ml/hls4ml/converters/onnx_to_hls.py", line 318, in onnx_to_hls
    hls_model = ModelGraph(config, reader, layer_list, input_layers, output_layers)
TypeError: ModelGraph.__init__() takes from 3 to 5 positional arguments but 6 were given

I originally encountered this error in a different and more complex project, but reproduced it in hls4ml's example code to facilitate debugging

Thank you!

vloncar commented 3 months ago

The current ONNX parser is broken in many ways, it is going to be updated soon. You can try out the development version in #979

sei-jgwohlbier commented 2 months ago

@vloncar what is the expectation on the tests in https://github.com/fastmachinelearning/hls4ml/pull/979 ? I get

= 58 failed, 1280 passed, 6 skipped, 649 warnings, 8 errors in 3587.97s (0:59:47) =

In particular test_qonnx.py does not pass.

(hls4ml) hls4ml-user@lx-pinkbird:~/hls4ml/test/pytest$ python -m pytest test_qonnx.py 
============================= test session starts ==============================
platform linux -- Python 3.10.14, pytest-8.1.1, pluggy-1.4.0
rootdir: /home/hls4ml-user/hls4ml
configfile: pyproject.toml
collected 9 items                                                              

test_qonnx.py EEEEEEFFF                                                  [100%]

==================================== ERRORS ====================================
___________________ ERROR at setup of test_tfc_2w2a[Vivado] ____________________

    @pytest.fixture(scope='module')
    def tfc_2w2a_model():
        '''
        Load the tiny fully-connected model
        '''
        dl_dir = test_root_path
        dl_file = str(dl_dir / "qonnx-tfc-2w2a.onnx")
        tfc_w2a2_qonnx_url = (
            "https://raw.githubusercontent.com/fastmachinelearning/"
            "QONNX_model_zoo/main/models/MNIST/Brevitas_FINN_TFC/TFC/TFC_2W2A.onnx"
        )
        urllib.request.urlretrieve(tfc_w2a2_qonnx_url, dl_file)
        assert os.path.isfile(dl_file)
        out_file = str(dl_dir / "qonnx-tfc-2w2a-clean.onnx")

        # cleanup
>       qonnx.util.cleanup.cleanup(dl_file, out_file=out_file)

test_qonnx.py:35: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/util/cleanup.py:66: in cleanup
    model = cleanup_model(model, preserve_qnt_ops=preserve_qnt_ops, override_batchsize=override_batchsize)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/util/cleanup.py:46: in cleanup_model
    model = model.transform(t)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/core/modelwrapper.py:140: in transform
    (transformed_model, model_was_changed) = transformation.apply(transformed_model)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/transformation/fold_constants.py:108: in apply
    oxe.execute_node(n, execution_context, model.graph, opset_version=opset_version)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/core/onnx_exec.py:87: in execute_node
    sess = rt.InferenceSession(node_model.SerializeToString())
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:419: in __init__
    self._create_inference_session(providers, provider_options, disabled_optimizers)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <onnxruntime.capi.onnxruntime_inference_collection.InferenceSession object at 0x7ff2b2cd78b0>
providers = [], provider_options = [], disabled_optimizers = None

    def _create_inference_session(self, providers, provider_options, disabled_optimizers=None):
        available_providers = C.get_available_providers()

        # Tensorrt can fall back to CUDA if it's explicitly assigned. All others fall back to CPU.
        if "TensorrtExecutionProvider" in available_providers:
            if providers and any(
                provider == "CUDAExecutionProvider"
                or (isinstance(provider, tuple) and provider[0] == "CUDAExecutionProvider")
                for provider in providers
            ):
                self._fallback_providers = ["CUDAExecutionProvider", "CPUExecutionProvider"]
            else:
                self._fallback_providers = ["CPUExecutionProvider"]
        # MIGraphX can fall back to ROCM if it's explicitly assigned. All others fall back to CPU.
        elif "MIGraphXExecutionProvider" in available_providers:
            if providers and any(
                provider == "ROCMExecutionProvider"
                or (isinstance(provider, tuple) and provider[0] == "ROCMExecutionProvider")
                for provider in providers
            ):
                self._fallback_providers = ["ROCMExecutionProvider", "CPUExecutionProvider"]
            else:
                self._fallback_providers = ["CPUExecutionProvider"]
        else:
            self._fallback_providers = ["CPUExecutionProvider"]

        # validate providers and provider_options before other initialization
        providers, provider_options = check_and_normalize_provider_args(
            providers, provider_options, available_providers
        )

        session_options = self._sess_options if self._sess_options else C.get_default_session_options()

        self._register_ep_custom_ops(session_options, providers, provider_options, available_providers)

        if self._model_path:
            sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
        else:
>           sess = C.InferenceSession(session_options, self._model_bytes, False, self._read_config_from_model)
E           onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Failed to load model with error: /onnxruntime_src/onnxruntime/core/graph/model.cc:179 onnxruntime::Model::Model(onnx::ModelProto&&, const onnxruntime::PathString&, const onnxruntime::IOnnxRuntimeOpSchemaRegistryList*, const onnxruntime::logging::Logger&, const onnxruntime::ModelOptions&) Unsupported model IR version: 10, max supported IR version: 9

../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:474: InvalidArgument
____________________ ERROR at setup of test_tfc_2w2a[Vitis] ____________________

    @pytest.fixture(scope='module')
    def tfc_2w2a_model():
        '''
        Load the tiny fully-connected model
        '''
        dl_dir = test_root_path
        dl_file = str(dl_dir / "qonnx-tfc-2w2a.onnx")
        tfc_w2a2_qonnx_url = (
            "https://raw.githubusercontent.com/fastmachinelearning/"
            "QONNX_model_zoo/main/models/MNIST/Brevitas_FINN_TFC/TFC/TFC_2W2A.onnx"
        )
        urllib.request.urlretrieve(tfc_w2a2_qonnx_url, dl_file)
        assert os.path.isfile(dl_file)
        out_file = str(dl_dir / "qonnx-tfc-2w2a-clean.onnx")

        # cleanup
>       qonnx.util.cleanup.cleanup(dl_file, out_file=out_file)

test_qonnx.py:35: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/util/cleanup.py:66: in cleanup
    model = cleanup_model(model, preserve_qnt_ops=preserve_qnt_ops, override_batchsize=override_batchsize)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/util/cleanup.py:46: in cleanup_model
    model = model.transform(t)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/core/modelwrapper.py:140: in transform
    (transformed_model, model_was_changed) = transformation.apply(transformed_model)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/transformation/fold_constants.py:108: in apply
    oxe.execute_node(n, execution_context, model.graph, opset_version=opset_version)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/core/onnx_exec.py:87: in execute_node
    sess = rt.InferenceSession(node_model.SerializeToString())
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:419: in __init__
    self._create_inference_session(providers, provider_options, disabled_optimizers)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <onnxruntime.capi.onnxruntime_inference_collection.InferenceSession object at 0x7ff2b2cd78b0>
providers = [], provider_options = [], disabled_optimizers = None

    def _create_inference_session(self, providers, provider_options, disabled_optimizers=None):
        available_providers = C.get_available_providers()

        # Tensorrt can fall back to CUDA if it's explicitly assigned. All others fall back to CPU.
        if "TensorrtExecutionProvider" in available_providers:
            if providers and any(
                provider == "CUDAExecutionProvider"
                or (isinstance(provider, tuple) and provider[0] == "CUDAExecutionProvider")
                for provider in providers
            ):
                self._fallback_providers = ["CUDAExecutionProvider", "CPUExecutionProvider"]
            else:
                self._fallback_providers = ["CPUExecutionProvider"]
        # MIGraphX can fall back to ROCM if it's explicitly assigned. All others fall back to CPU.
        elif "MIGraphXExecutionProvider" in available_providers:
            if providers and any(
                provider == "ROCMExecutionProvider"
                or (isinstance(provider, tuple) and provider[0] == "ROCMExecutionProvider")
                for provider in providers
            ):
                self._fallback_providers = ["ROCMExecutionProvider", "CPUExecutionProvider"]
            else:
                self._fallback_providers = ["CPUExecutionProvider"]
        else:
            self._fallback_providers = ["CPUExecutionProvider"]

        # validate providers and provider_options before other initialization
        providers, provider_options = check_and_normalize_provider_args(
            providers, provider_options, available_providers
        )

        session_options = self._sess_options if self._sess_options else C.get_default_session_options()

        self._register_ep_custom_ops(session_options, providers, provider_options, available_providers)

        if self._model_path:
            sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
        else:
>           sess = C.InferenceSession(session_options, self._model_bytes, False, self._read_config_from_model)
E           onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Failed to load model with error: /onnxruntime_src/onnxruntime/core/graph/model.cc:179 onnxruntime::Model::Model(onnx::ModelProto&&, const onnxruntime::PathString&, const onnxruntime::IOnnxRuntimeOpSchemaRegistryList*, const onnxruntime::logging::Logger&, const onnxruntime::ModelOptions&) Unsupported model IR version: 10, max supported IR version: 9

../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:474: InvalidArgument
___________________ ERROR at setup of test_tfc_2w2a[Quartus] ___________________

    @pytest.fixture(scope='module')
    def tfc_2w2a_model():
        '''
        Load the tiny fully-connected model
        '''
        dl_dir = test_root_path
        dl_file = str(dl_dir / "qonnx-tfc-2w2a.onnx")
        tfc_w2a2_qonnx_url = (
            "https://raw.githubusercontent.com/fastmachinelearning/"
            "QONNX_model_zoo/main/models/MNIST/Brevitas_FINN_TFC/TFC/TFC_2W2A.onnx"
        )
        urllib.request.urlretrieve(tfc_w2a2_qonnx_url, dl_file)
        assert os.path.isfile(dl_file)
        out_file = str(dl_dir / "qonnx-tfc-2w2a-clean.onnx")

        # cleanup
>       qonnx.util.cleanup.cleanup(dl_file, out_file=out_file)

test_qonnx.py:35: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/util/cleanup.py:66: in cleanup
    model = cleanup_model(model, preserve_qnt_ops=preserve_qnt_ops, override_batchsize=override_batchsize)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/util/cleanup.py:46: in cleanup_model
    model = model.transform(t)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/core/modelwrapper.py:140: in transform
    (transformed_model, model_was_changed) = transformation.apply(transformed_model)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/transformation/fold_constants.py:108: in apply
    oxe.execute_node(n, execution_context, model.graph, opset_version=opset_version)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/core/onnx_exec.py:87: in execute_node
    sess = rt.InferenceSession(node_model.SerializeToString())
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:419: in __init__
    self._create_inference_session(providers, provider_options, disabled_optimizers)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <onnxruntime.capi.onnxruntime_inference_collection.InferenceSession object at 0x7ff2b2cd78b0>
providers = [], provider_options = [], disabled_optimizers = None

    def _create_inference_session(self, providers, provider_options, disabled_optimizers=None):
        available_providers = C.get_available_providers()

        # Tensorrt can fall back to CUDA if it's explicitly assigned. All others fall back to CPU.
        if "TensorrtExecutionProvider" in available_providers:
            if providers and any(
                provider == "CUDAExecutionProvider"
                or (isinstance(provider, tuple) and provider[0] == "CUDAExecutionProvider")
                for provider in providers
            ):
                self._fallback_providers = ["CUDAExecutionProvider", "CPUExecutionProvider"]
            else:
                self._fallback_providers = ["CPUExecutionProvider"]
        # MIGraphX can fall back to ROCM if it's explicitly assigned. All others fall back to CPU.
        elif "MIGraphXExecutionProvider" in available_providers:
            if providers and any(
                provider == "ROCMExecutionProvider"
                or (isinstance(provider, tuple) and provider[0] == "ROCMExecutionProvider")
                for provider in providers
            ):
                self._fallback_providers = ["ROCMExecutionProvider", "CPUExecutionProvider"]
            else:
                self._fallback_providers = ["CPUExecutionProvider"]
        else:
            self._fallback_providers = ["CPUExecutionProvider"]

        # validate providers and provider_options before other initialization
        providers, provider_options = check_and_normalize_provider_args(
            providers, provider_options, available_providers
        )

        session_options = self._sess_options if self._sess_options else C.get_default_session_options()

        self._register_ep_custom_ops(session_options, providers, provider_options, available_providers)

        if self._model_path:
            sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
        else:
>           sess = C.InferenceSession(session_options, self._model_bytes, False, self._read_config_from_model)
E           onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Failed to load model with error: /onnxruntime_src/onnxruntime/core/graph/model.cc:179 onnxruntime::Model::Model(onnx::ModelProto&&, const onnxruntime::PathString&, const onnxruntime::IOnnxRuntimeOpSchemaRegistryList*, const onnxruntime::logging::Logger&, const onnxruntime::ModelOptions&) Unsupported model IR version: 10, max supported IR version: 9

../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:474: InvalidArgument
___________________ ERROR at setup of test_cnv_2w2a[Vivado] ____________________

    @pytest.fixture(scope='module')
    def cnv_2w2a_model():
        '''
        Load the small convolution model
        '''
        dl_dir = test_root_path
        dl_file = str(dl_dir / "qonnx-cnv-2w2a.onnx")
        cnv_w2a2_qonnx_url = (
            "https://raw.githubusercontent.com/fastmachinelearning/"
            "QONNX_model_zoo/main/models/CIFAR10/Brevitas_FINN_CNV/CNV_2W2A.onnx"
        )
        urllib.request.urlretrieve(cnv_w2a2_qonnx_url, dl_file)
        assert os.path.isfile(dl_file)
        out_clean = str(dl_dir / "qonnx-cnv-2w2a-clean.onnx")
        out_chanlast = str(dl_dir / "qonnx-cnv-2w2a-clean-channels-last.onnx")
        out_file = str(dl_dir / "qonnx-cnv-2w2a-clean-channels-last-clean.onnx")

        # cleanup
>       qonnx.util.cleanup.cleanup(dl_file, out_file=out_clean)

test_qonnx.py:58: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/util/cleanup.py:66: in cleanup
    model = cleanup_model(model, preserve_qnt_ops=preserve_qnt_ops, override_batchsize=override_batchsize)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/util/cleanup.py:46: in cleanup_model
    model = model.transform(t)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/core/modelwrapper.py:140: in transform
    (transformed_model, model_was_changed) = transformation.apply(transformed_model)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/transformation/fold_constants.py:108: in apply
    oxe.execute_node(n, execution_context, model.graph, opset_version=opset_version)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/core/onnx_exec.py:87: in execute_node
    sess = rt.InferenceSession(node_model.SerializeToString())
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:419: in __init__
    self._create_inference_session(providers, provider_options, disabled_optimizers)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <onnxruntime.capi.onnxruntime_inference_collection.InferenceSession object at 0x7ff2b0b907f0>
providers = [], provider_options = [], disabled_optimizers = None

    def _create_inference_session(self, providers, provider_options, disabled_optimizers=None):
        available_providers = C.get_available_providers()

        # Tensorrt can fall back to CUDA if it's explicitly assigned. All others fall back to CPU.
        if "TensorrtExecutionProvider" in available_providers:
            if providers and any(
                provider == "CUDAExecutionProvider"
                or (isinstance(provider, tuple) and provider[0] == "CUDAExecutionProvider")
                for provider in providers
            ):
                self._fallback_providers = ["CUDAExecutionProvider", "CPUExecutionProvider"]
            else:
                self._fallback_providers = ["CPUExecutionProvider"]
        # MIGraphX can fall back to ROCM if it's explicitly assigned. All others fall back to CPU.
        elif "MIGraphXExecutionProvider" in available_providers:
            if providers and any(
                provider == "ROCMExecutionProvider"
                or (isinstance(provider, tuple) and provider[0] == "ROCMExecutionProvider")
                for provider in providers
            ):
                self._fallback_providers = ["ROCMExecutionProvider", "CPUExecutionProvider"]
            else:
                self._fallback_providers = ["CPUExecutionProvider"]
        else:
            self._fallback_providers = ["CPUExecutionProvider"]

        # validate providers and provider_options before other initialization
        providers, provider_options = check_and_normalize_provider_args(
            providers, provider_options, available_providers
        )

        session_options = self._sess_options if self._sess_options else C.get_default_session_options()

        self._register_ep_custom_ops(session_options, providers, provider_options, available_providers)

        if self._model_path:
            sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
        else:
>           sess = C.InferenceSession(session_options, self._model_bytes, False, self._read_config_from_model)
E           onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Failed to load model with error: /onnxruntime_src/onnxruntime/core/graph/model.cc:179 onnxruntime::Model::Model(onnx::ModelProto&&, const onnxruntime::PathString&, const onnxruntime::IOnnxRuntimeOpSchemaRegistryList*, const onnxruntime::logging::Logger&, const onnxruntime::ModelOptions&) Unsupported model IR version: 10, max supported IR version: 9

../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:474: InvalidArgument
____________________ ERROR at setup of test_cnv_2w2a[Vitis] ____________________

    @pytest.fixture(scope='module')
    def cnv_2w2a_model():
        '''
        Load the small convolution model
        '''
        dl_dir = test_root_path
        dl_file = str(dl_dir / "qonnx-cnv-2w2a.onnx")
        cnv_w2a2_qonnx_url = (
            "https://raw.githubusercontent.com/fastmachinelearning/"
            "QONNX_model_zoo/main/models/CIFAR10/Brevitas_FINN_CNV/CNV_2W2A.onnx"
        )
        urllib.request.urlretrieve(cnv_w2a2_qonnx_url, dl_file)
        assert os.path.isfile(dl_file)
        out_clean = str(dl_dir / "qonnx-cnv-2w2a-clean.onnx")
        out_chanlast = str(dl_dir / "qonnx-cnv-2w2a-clean-channels-last.onnx")
        out_file = str(dl_dir / "qonnx-cnv-2w2a-clean-channels-last-clean.onnx")

        # cleanup
>       qonnx.util.cleanup.cleanup(dl_file, out_file=out_clean)

test_qonnx.py:58: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/util/cleanup.py:66: in cleanup
    model = cleanup_model(model, preserve_qnt_ops=preserve_qnt_ops, override_batchsize=override_batchsize)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/util/cleanup.py:46: in cleanup_model
    model = model.transform(t)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/core/modelwrapper.py:140: in transform
    (transformed_model, model_was_changed) = transformation.apply(transformed_model)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/transformation/fold_constants.py:108: in apply
    oxe.execute_node(n, execution_context, model.graph, opset_version=opset_version)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/core/onnx_exec.py:87: in execute_node
    sess = rt.InferenceSession(node_model.SerializeToString())
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:419: in __init__
    self._create_inference_session(providers, provider_options, disabled_optimizers)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <onnxruntime.capi.onnxruntime_inference_collection.InferenceSession object at 0x7ff2b0b907f0>
providers = [], provider_options = [], disabled_optimizers = None

    def _create_inference_session(self, providers, provider_options, disabled_optimizers=None):
        available_providers = C.get_available_providers()

        # Tensorrt can fall back to CUDA if it's explicitly assigned. All others fall back to CPU.
        if "TensorrtExecutionProvider" in available_providers:
            if providers and any(
                provider == "CUDAExecutionProvider"
                or (isinstance(provider, tuple) and provider[0] == "CUDAExecutionProvider")
                for provider in providers
            ):
                self._fallback_providers = ["CUDAExecutionProvider", "CPUExecutionProvider"]
            else:
                self._fallback_providers = ["CPUExecutionProvider"]
        # MIGraphX can fall back to ROCM if it's explicitly assigned. All others fall back to CPU.
        elif "MIGraphXExecutionProvider" in available_providers:
            if providers and any(
                provider == "ROCMExecutionProvider"
                or (isinstance(provider, tuple) and provider[0] == "ROCMExecutionProvider")
                for provider in providers
            ):
                self._fallback_providers = ["ROCMExecutionProvider", "CPUExecutionProvider"]
            else:
                self._fallback_providers = ["CPUExecutionProvider"]
        else:
            self._fallback_providers = ["CPUExecutionProvider"]

        # validate providers and provider_options before other initialization
        providers, provider_options = check_and_normalize_provider_args(
            providers, provider_options, available_providers
        )

        session_options = self._sess_options if self._sess_options else C.get_default_session_options()

        self._register_ep_custom_ops(session_options, providers, provider_options, available_providers)

        if self._model_path:
            sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
        else:
>           sess = C.InferenceSession(session_options, self._model_bytes, False, self._read_config_from_model)
E           onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Failed to load model with error: /onnxruntime_src/onnxruntime/core/graph/model.cc:179 onnxruntime::Model::Model(onnx::ModelProto&&, const onnxruntime::PathString&, const onnxruntime::IOnnxRuntimeOpSchemaRegistryList*, const onnxruntime::logging::Logger&, const onnxruntime::ModelOptions&) Unsupported model IR version: 10, max supported IR version: 9

../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:474: InvalidArgument
___________________ ERROR at setup of test_cnv_2w2a[Quartus] ___________________

    @pytest.fixture(scope='module')
    def cnv_2w2a_model():
        '''
        Load the small convolution model
        '''
        dl_dir = test_root_path
        dl_file = str(dl_dir / "qonnx-cnv-2w2a.onnx")
        cnv_w2a2_qonnx_url = (
            "https://raw.githubusercontent.com/fastmachinelearning/"
            "QONNX_model_zoo/main/models/CIFAR10/Brevitas_FINN_CNV/CNV_2W2A.onnx"
        )
        urllib.request.urlretrieve(cnv_w2a2_qonnx_url, dl_file)
        assert os.path.isfile(dl_file)
        out_clean = str(dl_dir / "qonnx-cnv-2w2a-clean.onnx")
        out_chanlast = str(dl_dir / "qonnx-cnv-2w2a-clean-channels-last.onnx")
        out_file = str(dl_dir / "qonnx-cnv-2w2a-clean-channels-last-clean.onnx")

        # cleanup
>       qonnx.util.cleanup.cleanup(dl_file, out_file=out_clean)

test_qonnx.py:58: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/util/cleanup.py:66: in cleanup
    model = cleanup_model(model, preserve_qnt_ops=preserve_qnt_ops, override_batchsize=override_batchsize)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/util/cleanup.py:46: in cleanup_model
    model = model.transform(t)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/core/modelwrapper.py:140: in transform
    (transformed_model, model_was_changed) = transformation.apply(transformed_model)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/transformation/fold_constants.py:108: in apply
    oxe.execute_node(n, execution_context, model.graph, opset_version=opset_version)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/core/onnx_exec.py:87: in execute_node
    sess = rt.InferenceSession(node_model.SerializeToString())
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:419: in __init__
    self._create_inference_session(providers, provider_options, disabled_optimizers)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <onnxruntime.capi.onnxruntime_inference_collection.InferenceSession object at 0x7ff2b0b907f0>
providers = [], provider_options = [], disabled_optimizers = None

    def _create_inference_session(self, providers, provider_options, disabled_optimizers=None):
        available_providers = C.get_available_providers()

        # Tensorrt can fall back to CUDA if it's explicitly assigned. All others fall back to CPU.
        if "TensorrtExecutionProvider" in available_providers:
            if providers and any(
                provider == "CUDAExecutionProvider"
                or (isinstance(provider, tuple) and provider[0] == "CUDAExecutionProvider")
                for provider in providers
            ):
                self._fallback_providers = ["CUDAExecutionProvider", "CPUExecutionProvider"]
            else:
                self._fallback_providers = ["CPUExecutionProvider"]
        # MIGraphX can fall back to ROCM if it's explicitly assigned. All others fall back to CPU.
        elif "MIGraphXExecutionProvider" in available_providers:
            if providers and any(
                provider == "ROCMExecutionProvider"
                or (isinstance(provider, tuple) and provider[0] == "ROCMExecutionProvider")
                for provider in providers
            ):
                self._fallback_providers = ["ROCMExecutionProvider", "CPUExecutionProvider"]
            else:
                self._fallback_providers = ["CPUExecutionProvider"]
        else:
            self._fallback_providers = ["CPUExecutionProvider"]

        # validate providers and provider_options before other initialization
        providers, provider_options = check_and_normalize_provider_args(
            providers, provider_options, available_providers
        )

        session_options = self._sess_options if self._sess_options else C.get_default_session_options()

        self._register_ep_custom_ops(session_options, providers, provider_options, available_providers)

        if self._model_path:
            sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
        else:
>           sess = C.InferenceSession(session_options, self._model_bytes, False, self._read_config_from_model)
E           onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Failed to load model with error: /onnxruntime_src/onnxruntime/core/graph/model.cc:179 onnxruntime::Model::Model(onnx::ModelProto&&, const onnxruntime::PathString&, const onnxruntime::IOnnxRuntimeOpSchemaRegistryList*, const onnxruntime::logging::Logger&, const onnxruntime::ModelOptions&) Unsupported model IR version: 10, max supported IR version: 9

../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:474: InvalidArgument
=================================== FAILURES ===================================
___________________________ test_jet_tagging[Vivado] ___________________________

jettagging_model = <qonnx.core.modelwrapper.ModelWrapper object at 0x7ff2b09deb30>
backend = 'Vivado'

    @pytest.mark.parametrize('backend', ['Vivado', 'Vitis', 'Quartus'])
    def test_jet_tagging(jettagging_model, backend):
        model = jettagging_model

        # Execute QONNX model inference
        # TODO make the test bigger
        ishape = (1, 16)
        X = np.random.uniform(low=-1, high=+1, size=np.prod(ishape)).reshape(ishape)
        X = (np.round(X * 2**16) * 2**-16).astype(np.float32)
        idict = {model.graph.input[0].name: X}
>       y_qonnx = oxe.execute_onnx(model, idict)[model.graph.output[0].name]

test_qonnx.py:147: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/core/onnx_exec.py:179: in execute_onnx
    execute_node(node, execution_context, graph, return_full_exec_context, opset_version)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/core/onnx_exec.py:87: in execute_node
    sess = rt.InferenceSession(node_model.SerializeToString())
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:419: in __init__
    self._create_inference_session(providers, provider_options, disabled_optimizers)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <onnxruntime.capi.onnxruntime_inference_collection.InferenceSession object at 0x7ff2b09dde70>
providers = [], provider_options = [], disabled_optimizers = None

    def _create_inference_session(self, providers, provider_options, disabled_optimizers=None):
        available_providers = C.get_available_providers()

        # Tensorrt can fall back to CUDA if it's explicitly assigned. All others fall back to CPU.
        if "TensorrtExecutionProvider" in available_providers:
            if providers and any(
                provider == "CUDAExecutionProvider"
                or (isinstance(provider, tuple) and provider[0] == "CUDAExecutionProvider")
                for provider in providers
            ):
                self._fallback_providers = ["CUDAExecutionProvider", "CPUExecutionProvider"]
            else:
                self._fallback_providers = ["CPUExecutionProvider"]
        # MIGraphX can fall back to ROCM if it's explicitly assigned. All others fall back to CPU.
        elif "MIGraphXExecutionProvider" in available_providers:
            if providers and any(
                provider == "ROCMExecutionProvider"
                or (isinstance(provider, tuple) and provider[0] == "ROCMExecutionProvider")
                for provider in providers
            ):
                self._fallback_providers = ["ROCMExecutionProvider", "CPUExecutionProvider"]
            else:
                self._fallback_providers = ["CPUExecutionProvider"]
        else:
            self._fallback_providers = ["CPUExecutionProvider"]

        # validate providers and provider_options before other initialization
        providers, provider_options = check_and_normalize_provider_args(
            providers, provider_options, available_providers
        )

        session_options = self._sess_options if self._sess_options else C.get_default_session_options()

        self._register_ep_custom_ops(session_options, providers, provider_options, available_providers)

        if self._model_path:
            sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
        else:
>           sess = C.InferenceSession(session_options, self._model_bytes, False, self._read_config_from_model)
E           onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Failed to load model with error: /onnxruntime_src/onnxruntime/core/graph/model.cc:179 onnxruntime::Model::Model(onnx::ModelProto&&, const onnxruntime::PathString&, const onnxruntime::IOnnxRuntimeOpSchemaRegistryList*, const onnxruntime::logging::Logger&, const onnxruntime::ModelOptions&) Unsupported model IR version: 10, max supported IR version: 9

../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:474: InvalidArgument
___________________________ test_jet_tagging[Vitis] ____________________________

jettagging_model = <qonnx.core.modelwrapper.ModelWrapper object at 0x7ff2b09deb30>
backend = 'Vitis'

    @pytest.mark.parametrize('backend', ['Vivado', 'Vitis', 'Quartus'])
    def test_jet_tagging(jettagging_model, backend):
        model = jettagging_model

        # Execute QONNX model inference
        # TODO make the test bigger
        ishape = (1, 16)
        X = np.random.uniform(low=-1, high=+1, size=np.prod(ishape)).reshape(ishape)
        X = (np.round(X * 2**16) * 2**-16).astype(np.float32)
        idict = {model.graph.input[0].name: X}
>       y_qonnx = oxe.execute_onnx(model, idict)[model.graph.output[0].name]

test_qonnx.py:147: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/core/onnx_exec.py:179: in execute_onnx
    execute_node(node, execution_context, graph, return_full_exec_context, opset_version)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/core/onnx_exec.py:87: in execute_node
    sess = rt.InferenceSession(node_model.SerializeToString())
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:419: in __init__
    self._create_inference_session(providers, provider_options, disabled_optimizers)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <onnxruntime.capi.onnxruntime_inference_collection.InferenceSession object at 0x7ff2b0c20c40>
providers = [], provider_options = [], disabled_optimizers = None

    def _create_inference_session(self, providers, provider_options, disabled_optimizers=None):
        available_providers = C.get_available_providers()

        # Tensorrt can fall back to CUDA if it's explicitly assigned. All others fall back to CPU.
        if "TensorrtExecutionProvider" in available_providers:
            if providers and any(
                provider == "CUDAExecutionProvider"
                or (isinstance(provider, tuple) and provider[0] == "CUDAExecutionProvider")
                for provider in providers
            ):
                self._fallback_providers = ["CUDAExecutionProvider", "CPUExecutionProvider"]
            else:
                self._fallback_providers = ["CPUExecutionProvider"]
        # MIGraphX can fall back to ROCM if it's explicitly assigned. All others fall back to CPU.
        elif "MIGraphXExecutionProvider" in available_providers:
            if providers and any(
                provider == "ROCMExecutionProvider"
                or (isinstance(provider, tuple) and provider[0] == "ROCMExecutionProvider")
                for provider in providers
            ):
                self._fallback_providers = ["ROCMExecutionProvider", "CPUExecutionProvider"]
            else:
                self._fallback_providers = ["CPUExecutionProvider"]
        else:
            self._fallback_providers = ["CPUExecutionProvider"]

        # validate providers and provider_options before other initialization
        providers, provider_options = check_and_normalize_provider_args(
            providers, provider_options, available_providers
        )

        session_options = self._sess_options if self._sess_options else C.get_default_session_options()

        self._register_ep_custom_ops(session_options, providers, provider_options, available_providers)

        if self._model_path:
            sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
        else:
>           sess = C.InferenceSession(session_options, self._model_bytes, False, self._read_config_from_model)
E           onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Failed to load model with error: /onnxruntime_src/onnxruntime/core/graph/model.cc:179 onnxruntime::Model::Model(onnx::ModelProto&&, const onnxruntime::PathString&, const onnxruntime::IOnnxRuntimeOpSchemaRegistryList*, const onnxruntime::logging::Logger&, const onnxruntime::ModelOptions&) Unsupported model IR version: 10, max supported IR version: 9

../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:474: InvalidArgument
__________________________ test_jet_tagging[Quartus] ___________________________

jettagging_model = <qonnx.core.modelwrapper.ModelWrapper object at 0x7ff2b09deb30>
backend = 'Quartus'

    @pytest.mark.parametrize('backend', ['Vivado', 'Vitis', 'Quartus'])
    def test_jet_tagging(jettagging_model, backend):
        model = jettagging_model

        # Execute QONNX model inference
        # TODO make the test bigger
        ishape = (1, 16)
        X = np.random.uniform(low=-1, high=+1, size=np.prod(ishape)).reshape(ishape)
        X = (np.round(X * 2**16) * 2**-16).astype(np.float32)
        idict = {model.graph.input[0].name: X}
>       y_qonnx = oxe.execute_onnx(model, idict)[model.graph.output[0].name]

test_qonnx.py:147: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/core/onnx_exec.py:179: in execute_onnx
    execute_node(node, execution_context, graph, return_full_exec_context, opset_version)
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/core/onnx_exec.py:87: in execute_node
    sess = rt.InferenceSession(node_model.SerializeToString())
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:419: in __init__
    self._create_inference_session(providers, provider_options, disabled_optimizers)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <onnxruntime.capi.onnxruntime_inference_collection.InferenceSession object at 0x7ff2b09fc310>
providers = [], provider_options = [], disabled_optimizers = None

    def _create_inference_session(self, providers, provider_options, disabled_optimizers=None):
        available_providers = C.get_available_providers()

        # Tensorrt can fall back to CUDA if it's explicitly assigned. All others fall back to CPU.
        if "TensorrtExecutionProvider" in available_providers:
            if providers and any(
                provider == "CUDAExecutionProvider"
                or (isinstance(provider, tuple) and provider[0] == "CUDAExecutionProvider")
                for provider in providers
            ):
                self._fallback_providers = ["CUDAExecutionProvider", "CPUExecutionProvider"]
            else:
                self._fallback_providers = ["CPUExecutionProvider"]
        # MIGraphX can fall back to ROCM if it's explicitly assigned. All others fall back to CPU.
        elif "MIGraphXExecutionProvider" in available_providers:
            if providers and any(
                provider == "ROCMExecutionProvider"
                or (isinstance(provider, tuple) and provider[0] == "ROCMExecutionProvider")
                for provider in providers
            ):
                self._fallback_providers = ["ROCMExecutionProvider", "CPUExecutionProvider"]
            else:
                self._fallback_providers = ["CPUExecutionProvider"]
        else:
            self._fallback_providers = ["CPUExecutionProvider"]

        # validate providers and provider_options before other initialization
        providers, provider_options = check_and_normalize_provider_args(
            providers, provider_options, available_providers
        )

        session_options = self._sess_options if self._sess_options else C.get_default_session_options()

        self._register_ep_custom_ops(session_options, providers, provider_options, available_providers)

        if self._model_path:
            sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
        else:
>           sess = C.InferenceSession(session_options, self._model_bytes, False, self._read_config_from_model)
E           onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Failed to load model with error: /onnxruntime_src/onnxruntime/core/graph/model.cc:179 onnxruntime::Model::Model(onnx::ModelProto&&, const onnxruntime::PathString&, const onnxruntime::IOnnxRuntimeOpSchemaRegistryList*, const onnxruntime::logging::Logger&, const onnxruntime::ModelOptions&) Unsupported model IR version: 10, max supported IR version: 9

../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:474: InvalidArgument
=============================== warnings summary ===============================
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/tensorflow_model_optimization/__init__.py:65
  /home/hls4ml-user/miniconda3/envs/hls4ml/lib/python3.10/site-packages/tensorflow_model_optimization/__init__.py:65: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
    if (distutils.version.LooseVersion(tf.version.VERSION) <

../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/tensorflow_model_optimization/__init__.py:66
  /home/hls4ml-user/miniconda3/envs/hls4ml/lib/python3.10/site-packages/tensorflow_model_optimization/__init__.py:66: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
    distutils.version.LooseVersion(required_tensorflow_version)):

../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/calmjs/parse/utils.py:13
  /home/hls4ml-user/miniconda3/envs/hls4ml/lib/python3.10/site-packages/calmjs/parse/utils.py:13: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
    from pkg_resources import working_set

../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/pkg_resources/__init__.py:2871
  /home/hls4ml-user/miniconda3/envs/hls4ml/lib/python3.10/site-packages/pkg_resources/__init__.py:2871: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('calmjs')`.
  Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
    declare_namespace(pkg)

test/pytest/test_qonnx.py: 498 warnings
  /home/hls4ml-user/miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/util/onnx.py:40: DeprecationWarning: `mapping.TENSOR_TYPE_TO_NP_TYPE` is now deprecated and will be removed in a future release.To silence this warning, please use `helper.tensor_dtype_to_np_dtype` instead.
    return np.zeros(dims, dtype=onnx.mapping.TENSOR_TYPE_TO_NP_TYPE[vi.type.tensor_type.elem_type])

test/pytest/test_qonnx.py::test_jet_tagging[Vivado]
  /home/hls4ml-user/miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/core/modelwrapper.py:93: UserWarning: Some old-style domain attributes were automatically converted to new-style,
                  i.e. domain=finn to domain=qonnx.custom_op.<general|fpgadataflow|...>
    warnings.warn(

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED test_qonnx.py::test_jet_tagging[Vivado] - onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeEr...
FAILED test_qonnx.py::test_jet_tagging[Vitis] - onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeEr...
FAILED test_qonnx.py::test_jet_tagging[Quartus] - onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeEr...
ERROR test_qonnx.py::test_tfc_2w2a[Vivado] - onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeEr...
ERROR test_qonnx.py::test_tfc_2w2a[Vitis] - onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeEr...
ERROR test_qonnx.py::test_tfc_2w2a[Quartus] - onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeEr...
ERROR test_qonnx.py::test_cnv_2w2a[Vivado] - onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeEr...
ERROR test_qonnx.py::test_cnv_2w2a[Vitis] - onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeEr...
ERROR test_qonnx.py::test_cnv_2w2a[Quartus] - onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeEr...
================== 3 failed, 503 warnings, 6 errors in 5.61s ===================
sei-jgwohlbier commented 2 months ago

Installing onnx==1.15 and onnxruntime==1.17 seems to have cleared those onnxruntime errors.

sei-jgwohlbier commented 2 months ago

After clearing that issue, I do get one failure in test_qonnx.py, seen below.

(hls4ml) hls4ml-user@lx-pinkbird:~/hls4ml/test/pytest$ python -m pytest test_qonnx.py 
============================= test session starts ==============================
platform linux -- Python 3.10.14, pytest-8.1.1, pluggy-1.4.0
rootdir: /home/hls4ml-user/hls4ml
configfile: pyproject.toml
collected 9 items                                                              

test_qonnx.py .....F...                                                  [100%]

=================================== FAILURES ===================================
____________________________ test_cnv_2w2a[Quartus] ____________________________

cnv_2w2a_model = <qonnx.core.modelwrapper.ModelWrapper object at 0x7fce0ad28c70>
backend = 'Quartus'

    @pytest.mark.parametrize('backend', ['Vivado', 'Vitis', 'Quartus'])
    def test_cnv_2w2a(cnv_2w2a_model, backend):
        """
        This tests a convolution model. Note:  the batch normalizations weights not quantized, so it
        is difficult to make this match perfectly. It is also a slow test.
        """
        model = cnv_2w2a_model

        ishape = (1, 32, 32, 3)
        X = np.random.uniform(low=-1, high=+1, size=np.prod(ishape)).reshape(ishape)
        X = (np.round(X * 2**6) * 2**-6).astype(np.float32)
        idict = {model.graph.input[0].name: X}
        y_qonnx = oxe.execute_onnx(model, idict)[model.graph.output[0].name]

        # Convert QONNX model, compile, and run inference
        config = hls4ml.utils.config_from_onnx_model(model, backend=backend, default_precision='fixed<32,6>')
        hls_model = hls4ml.converters.convert_from_onnx_model(
            model,
            output_dir=str(test_root_path / f'hls4mlprj_qonnx_cnv-2w2a_{backend}'),
            io_type='io_stream',
            backend=backend,
            hls_config=config,
        )
>       hls_model.compile()

test_qonnx.py:131: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/hls4ml/model/graph.py:672: in compile
    self._compile()
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/hls4ml/model/graph.py:691: in _compile
    self._top_function_lib = ctypes.cdll.LoadLibrary(lib_name)
../../../miniconda3/envs/hls4ml/lib/python3.10/ctypes/__init__.py:452: in LoadLibrary
    return self._dlltype(name)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <CDLL '/home/hls4ml-user/hls4ml/test/pytest/hls4mlprj_qonnx_cnv-2w2a_Quartus/firmware/myproject-D9B7bA7c.so', handle 0 at 0x7fce0a534f40>
name = '/home/hls4ml-user/hls4ml/test/pytest/hls4mlprj_qonnx_cnv-2w2a_Quartus/firmware/myproject-D9B7bA7c.so'
mode = 0, handle = None, use_errno = False, use_last_error = False
winmode = None

    def __init__(self, name, mode=DEFAULT_MODE, handle=None,
                 use_errno=False,
                 use_last_error=False,
                 winmode=None):
        self._name = name
        flags = self._func_flags_
        if use_errno:
            flags |= _FUNCFLAG_USE_ERRNO
        if use_last_error:
            flags |= _FUNCFLAG_USE_LASTERROR
        if _sys.platform.startswith("aix"):
            """When the name contains ".a(" and ends with ")",
               e.g., "libFOO.a(libFOO.so)" - this is taken to be an
               archive(member) syntax for dlopen(), and the mode is adjusted.
               Otherwise, name is presented to dlopen() as a file argument.
            """
            if name and name.endswith(")") and ".a(" in name:
                mode |= ( _os.RTLD_MEMBER | _os.RTLD_NOW )
        if _os.name == "nt":
            if winmode is not None:
                mode = winmode
            else:
                import nt
                mode = nt._LOAD_LIBRARY_SEARCH_DEFAULT_DIRS
                if '/' in name or '\\' in name:
                    self._name = nt._getfullpathname(self._name)
                    mode |= nt._LOAD_LIBRARY_SEARCH_DLL_LOAD_DIR

        class _FuncPtr(_CFuncPtr):
            _flags_ = flags
            _restype_ = self._func_restype_
        self._FuncPtr = _FuncPtr

        if handle is None:
>           self._handle = _dlopen(self._name, mode)
E           OSError: /home/hls4ml-user/hls4ml/test/pytest/hls4mlprj_qonnx_cnv-2w2a_Quartus/firmware/myproject-D9B7bA7c.so: cannot open shared object file: No such file or directory

../../../miniconda3/envs/hls4ml/lib/python3.10/ctypes/__init__.py:374: OSError
----------------------------- Captured stdout call -----------------------------
Output layers:  ['Add_0']
Input shape: [32, 32, 3]
Topology:
Layer name: Quant_0, layer type: Quant, current shape: [[64, 3, 3, 3]]
Layer name: Quant_1, layer type: Quant, current shape: [[64, 3, 3, 64]]
Layer name: Quant_2, layer type: Quant, current shape: [[128, 3, 3, 64]]
Layer name: Quant_3, layer type: Quant, current shape: [[128, 3, 3, 128]]
Layer name: Quant_4, layer type: Quant, current shape: [[256, 3, 3, 128]]
Layer name: Quant_5, layer type: Quant, current shape: [[256, 3, 3, 256]]
Layer name: Quant_6, layer type: Quant, current shape: [[256, 512]]
Layer name: Quant_7, layer type: Quant, current shape: [[512, 512]]
Layer name: Quant_8, layer type: Quant, current shape: [[512, 10]]
Layer name: Mul_0, layer type: Merge, current shape: [[1, 32, 32, 3]]
Layer name: Sub_0, layer type: Merge, current shape: [[1, 32, 32, 3], [1]]
Layer name: Quant_9, layer type: Quant, current shape: [[1, 32, 32, 3]]
Layer name: Conv_0, layer type: Conv, current shape: [[1, 32, 32, 3], [64, 3, 3, 3]]
Layer name: BatchNormalization_0, layer type: BatchNormOnnx, current shape: [[1, 30, 30, 64], [64], [64], [64], [64]]
Layer name: Quant_10, layer type: Quant, current shape: [[1, 30, 30, 64]]
Layer name: Conv_1, layer type: Conv, current shape: [[1, 30, 30, 64], [64, 3, 3, 64]]
Layer name: BatchNormalization_1, layer type: BatchNormOnnx, current shape: [[1, 28, 28, 64], [64], [64], [64], [64]]
Layer name: Quant_11, layer type: Quant, current shape: [[1, 28, 28, 64]]
Layer name: MaxPool_0, layer type: MaxPooling2D, current shape: [[1, 28, 28, 64]]
Layer name: Conv_2, layer type: Conv, current shape: [[1, 14, 14, 64], [128, 3, 3, 64]]
Layer name: BatchNormalization_2, layer type: BatchNormOnnx, current shape: [[1, 12, 12, 128], [128], [128], [128], [128]]
Layer name: Quant_12, layer type: Quant, current shape: [[1, 12, 12, 128]]
Layer name: Conv_3, layer type: Conv, current shape: [[1, 12, 12, 128], [128, 3, 3, 128]]
Layer name: BatchNormalization_3, layer type: BatchNormOnnx, current shape: [[1, 10, 10, 128], [128], [128], [128], [128]]
Layer name: Quant_13, layer type: Quant, current shape: [[1, 10, 10, 128]]
Layer name: MaxPool_1, layer type: MaxPooling2D, current shape: [[1, 10, 10, 128]]
Layer name: Conv_4, layer type: Conv, current shape: [[1, 5, 5, 128], [256, 3, 3, 128]]
Layer name: BatchNormalization_4, layer type: BatchNormOnnx, current shape: [[1, 3, 3, 256], [256], [256], [256], [256]]
Layer name: Quant_14, layer type: Quant, current shape: [[1, 3, 3, 256]]
Layer name: Conv_5, layer type: Conv, current shape: [[1, 3, 3, 256], [256, 3, 3, 256]]
Layer name: BatchNormalization_5, layer type: BatchNormOnnx, current shape: [[1, 1, 1, 256], [256], [256], [256], [256]]
Layer name: Quant_15, layer type: Quant, current shape: [[1, 1, 1, 256]]
Layer name: Flatten_0, layer type: Reshape, current shape: [[1, 1, 1, 256]]
Layer name: MatMul_0, layer type: MatMul, current shape: [[1, 256], [256, 512]]
Layer name: BatchNormalization_6, layer type: BatchNormOnnx, current shape: [[1, 512], [512], [512], [512], [512]]
Layer name: Quant_16, layer type: Quant, current shape: [[1, 512]]
Layer name: MatMul_1, layer type: MatMul, current shape: [[1, 512], [512, 512]]
Layer name: BatchNormalization_7, layer type: BatchNormOnnx, current shape: [[1, 512], [512], [512], [512], [512]]
Layer name: Quant_17, layer type: Quant, current shape: [[1, 512]]
Layer name: MatMul_2, layer type: MatMul, current shape: [[1, 512], [512, 10]]
Layer name: Sub_1, layer type: Merge, current shape: [[1, 10], [1]]
Layer name: Div_0, layer type: Merge, current shape: [[1, 10], [1]]
Layer name: Mul_1, layer type: Merge, current shape: [[1, 10], [1]]
Layer name: Add_0, layer type: Merge, current shape: [[1, 10], [1]]
Interpreting Model ...
Output layers:  ['Add_0']
Input shape: [32, 32, 3]
Topology:
Layer name: Quant_0, layer type: Quant, current shape: [[64, 3, 3, 3]]
Layer name: Quant_1, layer type: Quant, current shape: [[64, 3, 3, 64]]
Layer name: Quant_2, layer type: Quant, current shape: [[128, 3, 3, 64]]
Layer name: Quant_3, layer type: Quant, current shape: [[128, 3, 3, 128]]
Layer name: Quant_4, layer type: Quant, current shape: [[256, 3, 3, 128]]
Layer name: Quant_5, layer type: Quant, current shape: [[256, 3, 3, 256]]
Layer name: Quant_6, layer type: Quant, current shape: [[256, 512]]
Layer name: Quant_7, layer type: Quant, current shape: [[512, 512]]
Layer name: Quant_8, layer type: Quant, current shape: [[512, 10]]
Layer name: Mul_0, layer type: Merge, current shape: [[1, 32, 32, 3]]
Layer name: Sub_0, layer type: Merge, current shape: [[1, 32, 32, 3], [1]]
Layer name: Quant_9, layer type: Quant, current shape: [[1, 32, 32, 3]]
Layer name: Conv_0, layer type: Conv, current shape: [[1, 32, 32, 3], [64, 3, 3, 3]]
Layer name: BatchNormalization_0, layer type: BatchNormOnnx, current shape: [[1, 30, 30, 64], [64], [64], [64], [64]]
Layer name: Quant_10, layer type: Quant, current shape: [[1, 30, 30, 64]]
Layer name: Conv_1, layer type: Conv, current shape: [[1, 30, 30, 64], [64, 3, 3, 64]]
Layer name: BatchNormalization_1, layer type: BatchNormOnnx, current shape: [[1, 28, 28, 64], [64], [64], [64], [64]]
Layer name: Quant_11, layer type: Quant, current shape: [[1, 28, 28, 64]]
Layer name: MaxPool_0, layer type: MaxPooling2D, current shape: [[1, 28, 28, 64]]
Layer name: Conv_2, layer type: Conv, current shape: [[1, 14, 14, 64], [128, 3, 3, 64]]
Layer name: BatchNormalization_2, layer type: BatchNormOnnx, current shape: [[1, 12, 12, 128], [128], [128], [128], [128]]
Layer name: Quant_12, layer type: Quant, current shape: [[1, 12, 12, 128]]
Layer name: Conv_3, layer type: Conv, current shape: [[1, 12, 12, 128], [128, 3, 3, 128]]
Layer name: BatchNormalization_3, layer type: BatchNormOnnx, current shape: [[1, 10, 10, 128], [128], [128], [128], [128]]
Layer name: Quant_13, layer type: Quant, current shape: [[1, 10, 10, 128]]
Layer name: MaxPool_1, layer type: MaxPooling2D, current shape: [[1, 10, 10, 128]]
Layer name: Conv_4, layer type: Conv, current shape: [[1, 5, 5, 128], [256, 3, 3, 128]]
Layer name: BatchNormalization_4, layer type: BatchNormOnnx, current shape: [[1, 3, 3, 256], [256], [256], [256], [256]]
Layer name: Quant_14, layer type: Quant, current shape: [[1, 3, 3, 256]]
Layer name: Conv_5, layer type: Conv, current shape: [[1, 3, 3, 256], [256, 3, 3, 256]]
Layer name: BatchNormalization_5, layer type: BatchNormOnnx, current shape: [[1, 1, 1, 256], [256], [256], [256], [256]]
Layer name: Quant_15, layer type: Quant, current shape: [[1, 1, 1, 256]]
Layer name: Flatten_0, layer type: Reshape, current shape: [[1, 1, 1, 256]]
Layer name: MatMul_0, layer type: MatMul, current shape: [[1, 256], [256, 512]]
Layer name: BatchNormalization_6, layer type: BatchNormOnnx, current shape: [[1, 512], [512], [512], [512], [512]]
Layer name: Quant_16, layer type: Quant, current shape: [[1, 512]]
Layer name: MatMul_1, layer type: MatMul, current shape: [[1, 512], [512, 512]]
Layer name: BatchNormalization_7, layer type: BatchNormOnnx, current shape: [[1, 512], [512], [512], [512], [512]]
Layer name: Quant_17, layer type: Quant, current shape: [[1, 512]]
Layer name: MatMul_2, layer type: MatMul, current shape: [[1, 512], [512, 10]]
Layer name: Sub_1, layer type: Merge, current shape: [[1, 10], [1]]
Layer name: Div_0, layer type: Merge, current shape: [[1, 10], [1]]
Layer name: Mul_1, layer type: Merge, current shape: [[1, 10], [1]]
Layer name: Add_0, layer type: Merge, current shape: [[1, 10], [1]]
Creating HLS model
Writing HLS project
Done
----------------------------- Captured stderr call -----------------------------
g++: internal compiler error: Segmentation fault (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-7/README.Bugs> for instructions.
g++: error: myproject.o: No such file or directory
=============================== warnings summary ===============================
../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/tensorflow_model_optimization/__init__.py:65
  /home/hls4ml-user/miniconda3/envs/hls4ml/lib/python3.10/site-packages/tensorflow_model_optimization/__init__.py:65: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
    if (distutils.version.LooseVersion(tf.version.VERSION) <

../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/tensorflow_model_optimization/__init__.py:66
  /home/hls4ml-user/miniconda3/envs/hls4ml/lib/python3.10/site-packages/tensorflow_model_optimization/__init__.py:66: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
    distutils.version.LooseVersion(required_tensorflow_version)):

../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/calmjs/parse/utils.py:13
  /home/hls4ml-user/miniconda3/envs/hls4ml/lib/python3.10/site-packages/calmjs/parse/utils.py:13: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
    from pkg_resources import working_set

../../../miniconda3/envs/hls4ml/lib/python3.10/site-packages/pkg_resources/__init__.py:2871
  /home/hls4ml-user/miniconda3/envs/hls4ml/lib/python3.10/site-packages/pkg_resources/__init__.py:2871: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('calmjs')`.
  Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
    declare_namespace(pkg)

test/pytest/test_qonnx.py: 2101 warnings
  /home/hls4ml-user/miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/util/onnx.py:40: DeprecationWarning: `mapping.TENSOR_TYPE_TO_NP_TYPE` is now deprecated and will be removed in a future release.To silence this warning, please use `helper.tensor_dtype_to_np_dtype` instead.
    return np.zeros(dims, dtype=onnx.mapping.TENSOR_TYPE_TO_NP_TYPE[vi.type.tensor_type.elem_type])

test/pytest/test_qonnx.py::test_jet_tagging[Vivado]
  /home/hls4ml-user/miniconda3/envs/hls4ml/lib/python3.10/site-packages/qonnx/core/modelwrapper.py:93: UserWarning: Some old-style domain attributes were automatically converted to new-style,
                  i.e. domain=finn to domain=qonnx.custom_op.<general|fpgadataflow|...>
    warnings.warn(

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED test_qonnx.py::test_cnv_2w2a[Quartus] - OSError: /home/hls4ml-user/hls4ml/test/pytest/hls4mlprj_qonnx_cnv-2w2a_Quar...
=========== 1 failed, 8 passed, 2106 warnings in 3564.98s (0:59:24) ============