utils/RunONNXModel.py does not support string-type model inputs

[negiyas]

"bidaf" model in the model zoo uses string-type inputs, so that we need string-type inputs in order to execute the model. If possible, I want to use utils/RunONNXModel.py to execute the model, but it does not support string-type inputs, even if the following ad-hoc patch is used to get input_names and output_names correctly without errors. It seems that we need (1) "generator of random string array" and (2) "loader of protobuf file (e.g. input_0.pb) supporting string type".

$ git diff
diff --git a/utils/RunONNXModel.py b/utils/RunONNXModel.py
index f2b77ad4..d28386d8 100755
--- a/utils/RunONNXModel.py
+++ b/utils/RunONNXModel.py
@@ -552,8 +552,8 @@ def main():
         print("  took ", end - start, " seconds.\n")

         # Get the input and output signature.
-        input_signature = sess.input_signature()
-        output_signature = sess.output_signature()
+        input_signature = sess.input_signature().replace('!krnl.string', '\"string\"')
+        output_signature = sess.output_signature().replace('!krnl.string', '\"string\"')
         input_names = get_names_in_signature(input_signature)
         output_names = get_names_in_signature(output_signature)

The following errors occur with and without specifying the reference holder.

Use reference folder specified by "--load-ref"

$ ONNX_MLIR_HOME=/Dreamer/trl/negishi/src/dlc.git/onnx-mlir/build/Debug ./utils/RunONNXModel.py --compile-args='--O3 --EmitLib --mtriple=s390x-ibm-loz --mcpu=z16 --onnx-op-stats TXT' --verify ref --verify-every-value --load-ref bidaf/test_data_set_0 --model bidaf/bidaf.onnx --load-so bidaf/bidaf-noshape-cpu.so
Numpy type not supported: object.

Use random data without using "--load-ref"

$ ONNX_MLIR_HOME=/Dreamer/trl/negishi/src/dlc.git/onnx-mlir/build/Debug ./utils/RunONNXModel.py --compile-args='--O3 --EmitLib --mtriple=s390x-ibm-loz --mcpu=z16  --shapeInformation=0:1x1,1:1x1x1x16,2:1x1,3:1x1x1x16 --onnx-op-stats TXT' --shape-info=0:1x1,1:1x1x1x16,2:1x1,3:1x1x1x16 --model bidaf/bidaf.onnx --load-so bidaf/bidaf-noshape-cpu.so
Temporary directory has been created at /tmp/tmpys7ijhab
Loading the compiled model ...
  took  0.0008687269873917103  seconds.

Generating random inputs ...
Traceback (most recent call last):
  File "./utils/RunONNXModel.py", line 667, in <module>
    main()
  File "./utils/RunONNXModel.py", line 565, in main
    inputs = generate_random_input(input_signature, input_shapes)
  File "./utils/RunONNXModel.py", line 362, in generate_random_input
    np_elem_type = MLIR_TYPE_TO_NP_TYPE[elem_type]
KeyError: 'string'

Your comments and advises are very welcome to support string-type inputs.

[hamptonm1]

@gongsu832 Would this be something you can take a look at? I just added support for Equal Operation that requires input type string and the backend tests are failing due to this type not being supported as of yet. If we are able to get this support in, we can enable the backend test for categorymapper and equal operation. Let me know your thoughts. Thanks!