neo-ai / neo-ai-dlr

Neo-AI-DLR is a common runtime for machine learning models compiled by AWS SageMaker Neo, TVM, or TreeLite.
Apache License 2.0
492 stars 106 forks source link

Use metadata first to determine input/output names with TF2-C-API #421

Closed ghost closed 2 years ago

ghost commented 2 years ago

We will first check if there is a compiled.meta file exists, if so, we will read input and output tensor names from the file. An example compiled.meta file generated from ssd_mobilenet_v2_320x320_coco17_tpu-8_float_batchN_nms

{
    "Requirements": {
        "TargetDevice": "jetson_xavier",
        "TargetDeviceType": "gpu"
    },
    "Compilation": {
        "CreatedTime": 1648771122.42479
    },
    "Model": {
        "Inputs": [{
            "name": "serving_default_input_tensor:0",
            "dtype": "float32",
            "shape": [-1, -1, -1, 3]
        }],
        "Outputs": [{
            "name": "PartitionedCall:0",
            "dtype": "float32",
            "shape": []
        }, {
            "name": "PartitionedCall:1",
            "dtype": "float32",
            "shape": [-1, 100]
        }, {
            "name": "PartitionedCall:2",
            "dtype": "float32",
            "shape": [-1, 100]
        }, {
            "name": "PartitionedCall:3",
            "dtype": "float32",
            "shape": [-1]
        }, {
            "name": "PartitionedCall:4",
            "dtype": "float32",
            "shape": []
        }, {
            "name": "PartitionedCall:5",
            "dtype": "float32",
            "shape": []
        }]
    }
}