search

yoeo / guesslang

Detect the programming language of a source code
https://guesslang.readthedocs.io
MIT License
798 stars 114 forks source link

Conversion to ONNX and TFJS fails. #69

Closed harishankar-gopalan closed 1 year ago

harishankar-gopalan commented 1 year ago

Has anyone been able to convert the default model to either ONNX or TFJS ? I have tried but mostly it seems to be an issue of unsupported operators. I am attaching the output that I get for each of them.

Command line for TFJS: tensorflowjs_converter --input_format=tf_saved_model --output_format=tfjs_graph_model <path-to-default-saved-model> <path-to-new-folder>

Output for the above:

2022-10-10 15:18:45.017186: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-10-10 15:18:45.161669: E tensorflow/stream_executor/cuda/cuda_blas.cc:2981] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2022-10-10 15:18:45.657095: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory
2022-10-10 15:18:45.657174: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory
2022-10-10 15:18:45.657185: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
2022-10-10 15:18:55.993704: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcusolver.so.11'; dlerror: libcusolver.so.11: cannot open shared object file: No such file or directory
2022-10-10 15:18:55.994301: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudnn.so.8'; dlerror: libcudnn.so.8: cannot open shared object file: No such file or directory
2022-10-10 15:18:55.994318: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1934] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
2022-10-10 15:18:55.994979: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
WARNING:tensorflow:Issue encountered when serializing global_step.
Type is unsupported, or the types of the items don't match field type in CollectionDef. Note this is a warning and probably safe to ignore.
This operation is not supported when eager execution is enabled.
WARNING:tensorflow:Issue encountered when serializing global_step.
Type is unsupported, or the types of the items don't match field type in CollectionDef. Note this is a warning and probably safe to ignore.
This operation is not supported when eager execution is enabled.
WARNING:tensorflow:Issue encountered when serializing variables.
Type is unsupported, or the types of the items don't match field type in CollectionDef. Note this is a warning and probably safe to ignore.
This operation is not supported when eager execution is enabled.
WARNING:tensorflow:Issue encountered when serializing variables.
Type is unsupported, or the types of the items don't match field type in CollectionDef. Note this is a warning and probably safe to ignore.
This operation is not supported when eager execution is enabled.
WARNING:tensorflow:Issue encountered when serializing trainable_variables.
Type is unsupported, or the types of the items don't match field type in CollectionDef. Note this is a warning and probably safe to ignore.
This operation is not supported when eager execution is enabled.
WARNING:tensorflow:Issue encountered when serializing trainable_variables.
Type is unsupported, or the types of the items don't match field type in CollectionDef. Note this is a warning and probably safe to ignore.
This operation is not supported when eager execution is enabled.
2022-10-10 15:18:56.598822: I tensorflow/core/grappler/devices.cc:66] Number of eligible GPUs (core count >= 8, compute capability >= 0.0): 8
2022-10-10 15:18:56.599119: I tensorflow/core/grappler/clusters/single_machine.cc:358] Starting new session
2022-10-10 15:18:57.721903: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1934] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
2022-10-10 15:18:57.757114: E tensorflow/core/grappler/grappler_item_builder.cc:670] Init node head/predictions/class_string_lookup/table_init/LookupTableImportV2 doesn't exist in graph
WARNING:tensorflow:From /home/hari/miniconda3/envs/srclangdetect/lib/python3.8/site-packages/tensorflowjs/converters/tf_saved_model_conversion_v2.py:398: load (from tensorflow.python.saved_model.loader_impl) is deprecated and will be removed in a future version.
Instructions for updating:
This function will only be available through the v1 compatibility library as tf.compat.v1.saved_model.loader.load or tf.compat.v1.saved_model.load. There will be a new function for importing SavedModels in Tensorflow 2.0.
WARNING:tensorflow:From /home/hari/miniconda3/envs/srclangdetect/lib/python3.8/site-packages/tensorflowjs/converters/tf_saved_model_conversion_v2.py:398: load (from tensorflow.python.saved_model.loader_impl) is deprecated and will be removed in a future version.
Instructions for updating:
This function will only be available through the v1 compatibility library as tf.compat.v1.saved_model.loader.load or tf.compat.v1.saved_model.load. There will be a new function for importing SavedModels in Tensorflow 2.0.
2022-10-10 15:18:57.904875: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:354] MLIR V1 optimization pass is not enabled
2022-10-10 15:18:57.917561: W tensorflow/core/common_runtime/forward_type_inference.cc:332] Type inference failed. This indicates an invalid graph that escaped type checking. Error message: INVALID_ARGUMENT: expected compatible input types, but input 1:
type_id: TFT_OPTIONAL
args {
  type_id: TFT_PRODUCT
  args {
    type_id: TFT_TENSOR
    args {
      type_id: TFT_INT64
    }
  }
}
 is neither a subtype nor a supertype of the combined inputs preceding it:
type_id: TFT_OPTIONAL
args {
  type_id: TFT_PRODUCT
  args {
    type_id: TFT_TENSOR
    args {
      type_id: TFT_INT32
    }
  }
}

    while inferring type of node 'dnn/zero_fraction/cond/output/_44'
WARNING:tensorflow:From /home/hari/miniconda3/envs/srclangdetect/lib/python3.8/site-packages/tensorflowjs/converters/tf_saved_model_conversion_v2.py:402: convert_variables_to_constants (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.compat.v1.graph_util.convert_variables_to_constants`
WARNING:tensorflow:From /home/hari/miniconda3/envs/srclangdetect/lib/python3.8/site-packages/tensorflowjs/converters/tf_saved_model_conversion_v2.py:402: convert_variables_to_constants (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.compat.v1.graph_util.convert_variables_to_constants`
WARNING:tensorflow:From /home/hari/miniconda3/envs/srclangdetect/lib/python3.8/site-packages/tensorflow/python/framework/convert_to_constants.py:936: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.compat.v1.graph_util.extract_sub_graph`
WARNING:tensorflow:From /home/hari/miniconda3/envs/srclangdetect/lib/python3.8/site-packages/tensorflow/python/framework/convert_to_constants.py:936: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.compat.v1.graph_util.extract_sub_graph`
Traceback (most recent call last):
  File "/home/hari/miniconda3/envs/srclangdetect/bin/tensorflowjs_converter", line 8, in <module>
    sys.exit(pip_main())
  File "/home/hari/miniconda3/envs/srclangdetect/lib/python3.8/site-packages/tensorflowjs/converters/converter.py", line 827, in pip_main
    main([' '.join(sys.argv[1:])])
  File "/home/hari/miniconda3/envs/srclangdetect/lib/python3.8/site-packages/tensorflowjs/converters/converter.py", line 831, in main
    convert(argv[0].split(' '))
  File "/home/hari/miniconda3/envs/srclangdetect/lib/python3.8/site-packages/tensorflowjs/converters/converter.py", line 817, in convert
    _dispatch_converter(input_format, output_format, args, quantization_dtype_map,
  File "/home/hari/miniconda3/envs/srclangdetect/lib/python3.8/site-packages/tensorflowjs/converters/converter.py", line 528, in _dispatch_converter
    tf_saved_model_conversion_v2.convert_tf_saved_model(
  File "/home/hari/miniconda3/envs/srclangdetect/lib/python3.8/site-packages/tensorflowjs/converters/tf_saved_model_conversion_v2.py", line 809, in convert_tf_saved_model
    _convert_tf_saved_model(output_dir, saved_model_dir=saved_model_dir,
  File "/home/hari/miniconda3/envs/srclangdetect/lib/python3.8/site-packages/tensorflowjs/converters/tf_saved_model_conversion_v2.py", line 699, in _convert_tf_saved_model
    optimize_graph(frozen_graph, signature,
  File "/home/hari/miniconda3/envs/srclangdetect/lib/python3.8/site-packages/tensorflowjs/converters/tf_saved_model_conversion_v2.py", line 157, in optimize_graph
    raise ValueError('Unsupported Ops in the model before optimization\n' +
ValueError: Unsupported Ops in the model before optimization
OptionalNone, ReadVariableOp, OptionalFromValue

Command line for ONNX: python -m tf2onnx.convert --saved-model <path-to-model-directory> --output guesslang.onnx --opset 11 --verbose

Output for the above:

2022-10-10 15:22:09.947615: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-10-10 15:22:10.093875: E tensorflow/stream_executor/cuda/cuda_blas.cc:2981] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2022-10-10 15:22:10.584761: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory
2022-10-10 15:22:10.584838: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory
2022-10-10 15:22:10.584845: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
/home/hari/miniconda3/envs/srclangdetect/lib/python3.8/runpy.py:127: RuntimeWarning: 'tf2onnx.convert' found in sys.modules after import of package 'tf2onnx', but prior to execution of 'tf2onnx.convert'; this may result in unpredictable behaviour
  warn(RuntimeWarning(msg))
2022-10-10 15:22:20.974883: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcusolver.so.11'; dlerror: libcusolver.so.11: cannot open shared object file: No such file or directory
2022-10-10 15:22:20.975415: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudnn.so.8'; dlerror: libcudnn.so.8: cannot open shared object file: No such file or directory
2022-10-10 15:22:20.975432: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1934] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
2022-10-10 15:22:20.976057: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-10-10 15:22:21,027 - WARNING - tf2onnx.tf_loader: '--tag' not specified for saved_model. Using --tag serve
2022-10-10 15:22:21,588 - INFO - tf2onnx.tf_loader: Signatures found in model: [serving_default,classification,predict].
2022-10-10 15:22:21,588 - WARNING - tf2onnx.tf_loader: '--signature_def' not specified, using first signature: serving_default
2022-10-10 15:22:21,588 - INFO - tf2onnx.tf_loader: Output names: ['classes', 'scores']
WARNING:tensorflow:Issue encountered when serializing global_step.
Type is unsupported, or the types of the items don't match field type in CollectionDef. Note this is a warning and probably safe to ignore.
This operation is not supported when eager execution is enabled.
2022-10-10 15:22:21,601 - WARNING - tensorflow: Issue encountered when serializing global_step.
Type is unsupported, or the types of the items don't match field type in CollectionDef. Note this is a warning and probably safe to ignore.
This operation is not supported when eager execution is enabled.
WARNING:tensorflow:Issue encountered when serializing variables.
Type is unsupported, or the types of the items don't match field type in CollectionDef. Note this is a warning and probably safe to ignore.
This operation is not supported when eager execution is enabled.
2022-10-10 15:22:21,601 - WARNING - tensorflow: Issue encountered when serializing variables.
Type is unsupported, or the types of the items don't match field type in CollectionDef. Note this is a warning and probably safe to ignore.
This operation is not supported when eager execution is enabled.
WARNING:tensorflow:Issue encountered when serializing trainable_variables.
Type is unsupported, or the types of the items don't match field type in CollectionDef. Note this is a warning and probably safe to ignore.
This operation is not supported when eager execution is enabled.
2022-10-10 15:22:21,602 - WARNING - tensorflow: Issue encountered when serializing trainable_variables.
Type is unsupported, or the types of the items don't match field type in CollectionDef. Note this is a warning and probably safe to ignore.
This operation is not supported when eager execution is enabled.
2022-10-10 15:22:21.620369: I tensorflow/core/grappler/devices.cc:66] Number of eligible GPUs (core count >= 8, compute capability >= 0.0): 8
2022-10-10 15:22:21.620657: I tensorflow/core/grappler/clusters/single_machine.cc:358] Starting new session
2022-10-10 15:22:22.735793: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1934] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
2022-10-10 15:22:22.771439: E tensorflow/core/grappler/grappler_item_builder.cc:670] Init node head/predictions/class_string_lookup/table_init/LookupTableImportV2 doesn't exist in graph
Traceback (most recent call last):
  File "/home/hari/miniconda3/envs/srclangdetect/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/hari/miniconda3/envs/srclangdetect/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/hari/miniconda3/envs/srclangdetect/lib/python3.8/site-packages/tf2onnx/convert.py", line 706, in <module>
    main()
  File "/home/hari/miniconda3/envs/srclangdetect/lib/python3.8/site-packages/tf2onnx/convert.py", line 238, in main
    graph_def, inputs, outputs, initialized_tables, tensors_to_rename = tf_loader.from_saved_model(
  File "/home/hari/miniconda3/envs/srclangdetect/lib/python3.8/site-packages/tf2onnx/tf_loader.py", line 614, in from_saved_model
    _from_saved_model_v2(model_path, input_names, output_names,
  File "/home/hari/miniconda3/envs/srclangdetect/lib/python3.8/site-packages/tf2onnx/tf_loader.py", line 598, in _from_saved_model_v2
    frozen_graph, initialized_tables = from_trackable(imported, concrete_func, inputs, outputs, large_model)
  File "/home/hari/miniconda3/envs/srclangdetect/lib/python3.8/site-packages/tf2onnx/tf_loader.py", line 225, in from_trackable
    raise e
  File "/home/hari/miniconda3/envs/srclangdetect/lib/python3.8/site-packages/tf2onnx/tf_loader.py", line 221, in from_trackable
    frozen_graph = from_function(concrete_func, inputs, outputs, large_model)
  File "/home/hari/miniconda3/envs/srclangdetect/lib/python3.8/site-packages/tf2onnx/tf_loader.py", line 280, in from_function
    raise e
  File "/home/hari/miniconda3/envs/srclangdetect/lib/python3.8/site-packages/tf2onnx/tf_loader.py", line 273, in from_function
    frozen_func = convert_variables_to_constants_v2(func, lower_control_flow=False, aggressive_inlining=True)
  File "/home/hari/miniconda3/envs/srclangdetect/lib/python3.8/site-packages/tensorflow/python/framework/convert_to_constants.py", line 1154, in convert_variables_to_constants_v2
    converter_data = _FunctionConverterDataInEager(
  File "/home/hari/miniconda3/envs/srclangdetect/lib/python3.8/site-packages/tensorflow/python/framework/convert_to_constants.py", line 817, in __init__
    graph_def = _run_inline_graph_optimization(func, lower_control_flow,
  File "/home/hari/miniconda3/envs/srclangdetect/lib/python3.8/site-packages/tensorflow/python/framework/convert_to_constants.py", line 1052, in _run_inline_graph_optimization
    return tf_optimizer.OptimizeGraph(config, meta_graph)
  File "/home/hari/miniconda3/envs/srclangdetect/lib/python3.8/site-packages/tensorflow/python/grappler/tf_optimizer.py", line 65, in OptimizeGraph
    out_graph = tf_opt.TF_OptimizeGraph(cluster.tf_cluster,
ValueError: Failed to import metagraph, check error log for more info.

Environment:

absl-py==1.2.0
astunparse==1.6.3
cachetools==5.2.0
certifi @ file:///croot/certifi_1665076670883/work/certifi
charset-normalizer==2.1.1
chex==0.1.5
coloredlogs==15.0.1
commonmark==0.9.1
contourpy==1.0.5
cycler==0.11.0
dm-tree==0.1.7
etils==0.8.0
flatbuffers==22.9.24
flax==0.6.1
fonttools==4.37.4
gast==0.4.0
google-auth==2.12.0
google-auth-oauthlib==0.4.6
google-pasta==0.2.0
grpcio==1.34.1
guesslang==2.2.1
h5py==3.1.0
humanfriendly==10.0
idna==3.4
importlib-metadata==5.0.0
importlib-resources==5.10.0
jax==0.3.21
jaxlib==0.3.20
keras==2.10.0
keras-nightly==2.5.0.dev2021032900
Keras-Preprocessing==1.1.2
kiwisolver==1.4.4
libclang==14.0.6
Markdown==3.4.1
MarkupSafe==2.1.1
matplotlib==3.6.1
mpmath==1.2.1
msgpack==1.0.4
numpy==1.23.3
oauthlib==3.2.1
onnx==1.12.0
onnxruntime==1.12.1
opt-einsum==3.3.0
optax==0.1.3
packaging==20.9
Pillow==9.2.0
protobuf==3.19.6
pyasn1==0.4.8
pyasn1-modules==0.2.8
Pygments==2.13.0
pyparsing==3.0.9
python-dateutil==2.8.2
PyYAML==6.0
requests==2.28.1
requests-oauthlib==1.3.1
rich==12.6.0
rsa==4.9
scipy==1.9.2
six==1.15.0
sympy==1.11.1
tensorboard==2.10.1
tensorboard-data-server==0.6.1
tensorboard-plugin-wit==1.8.1
tensorflow==2.10.0
tensorflow-estimator==2.10.0
tensorflow-hub==0.12.0
tensorflow-io-gcs-filesystem==0.27.0
tensorflowjs==3.21.0
termcolor==1.1.0
tf2onnx==1.12.1
toolz==0.12.0
typing_extensions==4.4.0
urllib3==1.26.12
Werkzeug==2.2.2
wrapt==1.12.1
zipp==3.9.0
NilSet commented 1 year ago

For tfjs, the unsupported ops may actually be part of the training half of the model and not be needed for inference. We were able to convert the model with --skip_op_check and then perform inference using the resulting model.

tensorflowjs_converter --skip_op_check --input_format=tf_saved_model models/saved_model  models/model-tfjs
harishankar-gopalan commented 1 year ago

Hi @NilSet thanks for the information. Will check the same and get back.

harishankar-gopalan commented 1 year ago

Hi @NilSet I can confirm that I was able to convert the model using the --skip_op_check option and also load the model using loadGraphModel from the TFJS SDK. Thanks for the help.

asiryan commented 1 year ago

@harishankar-gopalan could you share onnx model?