Cannot save the output model of the TVM-CLI compiling

[thuongtrannc]

Hi everyone,

I ran the docker as instruction to save the model for point pillars but there is no saved file in the output folder. After running the docker the output in the terminal is like this:

W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcudart.so.10.1'; dlerror: libcudart.so.10.1: cannot open shared object file: No such file or directory 2021-10-06 01:14:50.245545: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine. WARNING:root:Attribute momentum is ignored in relay.sym.batch_norm ...100%, 0.02 MB, 95 KB/s, 0 seconds passed WARNING:autotvm:Cannot find config for target=llvm -keys=cpu, workload=('conv2d_NCHWc.x86', ('TENSOR', (1, 9, 12000, 100), 'float32'), ('TENSOR', (64, 9, 1, 1), 'float32'), (1, 1), (0, 0, 0, 0), (1, 1), 'NCHW', 'NCHW', 'float32'). A fallback configuration is used, which may bring great performance regression. WARNING:autotvm:Cannot find config for target=llvm -keys=cpu, workload=('conv2d_NCHWc.x86', ('TENSOR', (1, 64, 12000, 100), 'float32'), ('TENSOR', (64, 64, 1, 34), 'float32'), (1, 1), (0, 0, 0, 0), (1, 3), 'NCHW', 'NCHW', 'float32'). A fallback configuration is used, which may bring great performance regression. WARNING:autotvm:Cannot find config for target=llvm -keys=cpu, workload=('conv2d_NCHWc.x86', ('TENSOR', (1, 8, 12000, 100, 8), 'float32'), ('TENSOR', (8, 8, 1, 34, 8, 8), 'float32'), (1, 1), (0, 0, 0, 0), (1, 3), 'NCHW8c', 'NCHW8c', 'float32'). A fallback configuration is used, which may bring great performance regression. WARNING:autotvm:Cannot find config for target=llvm -keys=cpu, workload=('conv2d_NCHWc.x86', ('TENSOR', (1, 3, 12000, 100, 3), 'float32'), ('TENSOR', (8, 3, 1, 1, 3, 8), 'float32'), (1, 1), (0, 0, 0, 0), (1, 1), 'NCHW3c', 'NCHW8c', 'float32'). A fallback configuration is used, which may bring great performance regression. /tvm_cli/tvm_cli:174: DeprecationWarning: legacy graph runtime behaviour of producing json / lib / params will be removed in the next release. Please see documents of tvm.contrib.graph_runtime.GraphModule for the new recommended usage. graph, lib, params = relay.build(mod, Writing library to /home/thuong/refer-source/modelzoo/perception/lidar_obstacle_detection/output_model/deploy_lib.so Writing graph to /home/thuong/refer-source/modelzoo/perception/lidar_obstacle_detection/output_model/deploy_graph.json Writing weights to /home/thuong/refer-source/modelzoo/perception/lidar_obstacle_detection/output_model/deploy_param.params Writing pipeline configuration to /home/thuong/refer-source/modelzoo/perception/lidar_obstacle_detection/output_model/inference_engine_tvm_config.hpp

I wonder that is cuda installed right version and should the output be like above? Currently, cuda toolkit version is 10.2 and cudnn is 7.6.5.

[ambroise-arm]

Hello. From looking at the logs it looks like the script finished normally. Could you provide the command you used?

[thuongtrannc]

Hi @ambroise-arm. Thank you very much for your replying. My commands are:

$ export MODEL_DIR=/home/thuong/refer-source/modelzoo/perception/lidar_obstacle_detection/point_pillars_pfe/onnx_fp32_kitti

$ docker run \

    -it --rm \

    -v ${MODEL_DIR}:${MODEL_DIR} -w ${MODEL_DIR} \

    -u $(id -u ${USER}):$(id -g ${USER}) \

    autoware/model-zoo-tvm-cli:latest \

        compile \

        --config ${MODEL_DIR}/definition.yaml \

        --output_path /home/thuong/refer-source/modelzoo/perception/lidar_obstacle_detection/output_model

I also downloaded the model here and put to the model directory above. If there is any mistake in command please let me know.

[ambroise-arm]

Using the -v ${MODEL_DIR}:${MODEL_DIR} argument to docker run, it mounts only this directory inside of the container. So /home/thuong/refer-source/modelzoo/perception/lidar_obstacle_detection/point_pillars_pfe/onnx_fp32_kitti has the same state inside and outside the container. But that's not the case for the output path, where /home/thuong/refer-source/modelzoo/perception/lidar_obstacle_detection/output_model will have a different state inside and outside. So I am guessing what happens is that the script running inside the container outputs the files correctly, but they are only existing inside of the container. Can you try changing output_path to ${MODEL_DIR}/output_model?

[thuongtrannc]

Hi @ambroise-arm. Sorry for late replying. It worked for me and could convert the model. Thank you very much!