search

NVIDIA / OpenSeq2Seq

Toolkit for efficient experimentation with Speech Recognition, Text2Speech and NLP
https://nvidia.github.io/OpenSeq2Seq
Apache License 2.0
1.54k stars 369 forks source link

incompatible with expected resource #407

Closed 2020zyc closed 5 years ago

2020zyc commented 5 years ago

hi, I use the transformer model in OpenSeq2Seq to run the built-in example, en-de machine translation.

I follow the tutorial of OpenSeq2Seq except using transformer model instead of nmt.

I can train the model and do inference with tensorflow 13.1, cuda10.1. But I can't do trt inference with built-in tensorrt 5 in tensorflow13.1. My gpu is GTX 2080TI.

The main error is: ValueError: Input 0 of node ForwardPass/transformer_decoder/decode/while/layer_0/self_attention/self_attention/q/Tensordot/ReadVariableOp/Enter was passed float from ForwardPass/transformer_decoder/layer_0/self_attention/self_attention/q/kernel:0 incompatible with expected resource.

And the whole error log is :

Traceback (most recent call last):
  File "/home/xxx/pycharm_proj/OpenSeq2Seq_raw/run.py", line 101, in <module>
    main()
  File "/home/xxx/pycharm_proj/OpenSeq2Seq_raw/run.py", line 81, in main
    args, base_config, config_module, base_model, hvd, checkpoint)
  File "/home/xxx/pycharm_proj/OpenSeq2Seq_raw/open_seq2seq/utils/utils.py", line 790, in create_model
    model.compile(checkpoint=checkpoint)
  File "/home/xxx/pycharm_proj/OpenSeq2Seq_raw/open_seq2seq/models/model.py", line 445, in compile
    checkpoint=checkpoint
  File "/home/xxx/pycharm_proj/OpenSeq2Seq_raw/open_seq2seq/models/model.py", line 645, in build_trt_forward_pass_graph
    maximum_cached_engines=trt_params["trt_maximum_cached_engines"]
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/contrib/tensorrt/python/trt_convert.py", line 333, in create_inference_graph
    importer.import_graph_def(input_graph_def, name="")
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/importer.py", line 430, in import_graph_def
    raise ValueError(str(e))
ValueError: Input 0 of node ForwardPass/transformer_decoder/decode/while/layer_0/self_attention/self_attention/q/Tensordot/ReadVariableOp/Enter was passed float from ForwardPass/transformer_decoder/layer_0/self_attention/self_attention/q/kernel:0 incompatible with expected resource.

Process finished with exit code 1

SOLUTION: I find a solution from the network by modifying some node.op, such as RefSwitch, AssignSub, AssignAdd, to Switch, Sub, Add respectively. But it is useless. The modification happens as follows:

        # Restore checkpoint here because we have to freeze the graph
        tf_saver = tf.train.Saver()
        tf_saver.restore(save_path=checkpoint, sess=tf_sess)

        # I ALSO TRY TO ADD THE MODIFICATION HERE. BUT USELESS.#

        frozen_graph = tf.graph_util.convert_variables_to_constants(
            tf_sess,
            tf_sess.graph_def,
            output_node_names=output_node_names
        )
        num_nodes = len(frozen_graph.node)
        print('Converting graph using TensorFlow-TensorRT...')

        # THIS IS THE MODIFICATION 
        # gd = tf_sess.graph.as_graph_def()
        for node in frozen_graph.node:
            if node.op == 'RefSwitch':
                node.op = 'Switch'
                for index in xrange(len(node.input)):
                    if 'moving_' in node.input[index]:
                        node.input[index] = node.input[index] + '/read'
            elif node.op == 'AssignSub':
                node.op = 'Sub'
                if 'use_locking' in node.attr: del node.attr['use_locking']
            elif node.op == 'AssignAdd':
                node.op = 'Add'
                if 'use_locking' in node.attr: del node.attr['use_locking']

        # ERROR OCCURS IN THE FOLLOWING FUNCTION.
        frozen_graph = trt.create_inference_graph(
            input_graph_def=frozen_graph,
            outputs=output_node_names,
            max_batch_size=trt_params["batch_size_per_gpu"],
            max_workspace_size_bytes=trt_params["trt_max_workspace_size_bytes"],
            precision_mode=trt_params["trt_precision_mode"],
            minimum_segment_size=trt_params["trt_minimum_segment_size"],
            is_dynamic_op=trt_params["trt_is_dynamic_op"],
            maximum_cached_engines=trt_params["trt_maximum_cached_engines"]
        )

I will be very grateful for any help.

okuchaiev commented 5 years ago

@trevor-m do you have any ideas?

trevor-m commented 5 years ago

Add this additional modification to fix RefEnter ops:

for node in frozen_graph.node:
  # ...
  if node.op == 'RefEnter':
    node.op = 'Enter'
2020zyc commented 5 years ago

@trevor-m @okuchaiev Thanks for your response. I try the suggestion(fix RefEnter), and get the same error. The following is the complete output in the console:

ssh://root@xxx:50025/usr/bin/python -u /home/abc/pycharm_proj/OpenSeq2Seq_raw/run.py
rm: cannot remove '/lib': Is a directory
*** Restoring from the best checkpoint
*** Using horovod
*** Loading model from logs/transformer/best_models/val_loss=2.4235-step-24007
*** Inference config:
{'batch_size_per_gpu': 1,
 'data_layer': <class 'open_seq2seq.data.text2text.text2text.ParallelTextDataLayer'>,
 'data_layer_params': {'delimiter': ' ',
                       'max_length': 256,
                       'repeat': False,
                       'shuffle': False,
                       'source_file': '/home/abc/pycharm_proj/OpenSeq2Seq_raw/data/wmt16_de_en/wmt14-en-de.src.BPE_common.32K.tok',
                       'src_vocab_file': '/home/abc/pycharm_proj/OpenSeq2Seq_raw/data/wmt16_de_en/m_common.vocab',
                       'target_file': '/home/abc/pycharm_proj/OpenSeq2Seq_raw/data/wmt16_de_en/wmt14-en-de.src.BPE_common.32K.tok',
                       'tgt_vocab_file': '/home/abc/pycharm_proj/OpenSeq2Seq_raw/data/wmt16_de_en/m_common.vocab'},
 'decoder': <class 'open_seq2seq.decoders.transformer_decoder.TransformerDecoder'>,
 'decoder_params': {'END_SYMBOL': 1,
                    'EOS_ID': 1,
                    'GO_SYMBOL': 2,
                    'PAD_SYMBOL': 0,
                    'alpha': 0.6,
                    'attention_dropout': 0.1,
                    'beam_size': 4,
                    'extra_decode_length': 50,
                    'filter_size': 2048,
                    'hidden_size': 512,
                    'layer_postprocess_dropout': 0.1,
                    'num_heads': 8,
                    'num_hidden_layers': 3,
                    'relu_dropout': 0.1},
 'dtype': tf.float32,
 'encoder': <class 'open_seq2seq.encoders.transformer_encoder.TransformerEncoder'>,
 'encoder_params': {'attention_dropout': 0.1,
                    'encoder_layers': 3,
                    'filter_size': 2048,
                    'hidden_size': 512,
                    'layer_postprocess_dropout': 0.1,
                    'num_heads': 8,
                    'pad_embeddings_2_eight': True,
                    'relu_dropout': 0.1,
                    'remove_padding': True},
 'eval_steps': 4001,
 'load_model': '',
 'logdir': 'logs/transformer/',
 'loss': <class 'open_seq2seq.losses.sequence_loss.PaddedCrossEntropyLossWithSmoothing'>,
 'loss_params': {'label_smoothing': 0.1},
 'lr_policy': <function transformer_policy at 0x7ff4db6f6510>,
 'lr_policy_params': {'d_model': 512,
                      'learning_rate': 2.0,
                      'warmup_steps': 8000},
 'max_steps': 300000,
 'num_gpus': 1,
 'optimizer': <class 'tensorflow.contrib.opt.python.training.lazy_adam_optimizer.LazyAdamOptimizer'>,
 'optimizer_params': {'beta1': 0.9, 'beta2': 0.997, 'epsilon': 1e-09},
 'print_loss_steps': 100,
 'print_samples_steps': 100,
 'save_checkpoint_steps': 299998,
 'save_summaries_steps': 100,
 'trt_precision_mode': 'INT8',
 'use_horovod': True,
 'use_trt': True}
*** Building graph in Horovod rank: 0
WARNING:tensorflow:From /home/abc/pycharm_proj/OpenSeq2Seq_raw/open_seq2seq/data/text2text/text2text.py:199: py_func (from tensorflow.python.ops.script_ops) is deprecated and will be removed in a future version.
Instructions for updating:
tf.py_func is deprecated in TF V2. Instead, use
    tf.py_function, which takes a python function which manipulates tf eager
    tensors instead of numpy arrays. It's easy to convert a tf eager tensor to
    an ndarray (just call tensor.numpy()) but having access to eager tensors
    means `tf.py_function`s can use accelerators such as GPUs as well as
    being differentiable using a gradient tape.

WARNING:tensorflow:From /usr/local/lib/python3.5/dist-packages/tensorflow/python/data/ops/dataset_ops.py:1419: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
2019-04-19 01:30:59.713265: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2097620000 Hz
2019-04-19 01:30:59.715443: I tensorflow/compiler/xla/service/service.cc:161] XLA service 0x827e7c0 executing computations on platform Host. Devices:
2019-04-19 01:30:59.715493: I tensorflow/compiler/xla/service/service.cc:168]   StreamExecutor device (0): <undefined>, <undefined>
2019-04-19 01:31:03.080964: I tensorflow/compiler/xla/service/service.cc:161] XLA service 0x82f9470 executing computations on platform CUDA. Devices:
2019-04-19 01:31:03.081046: I tensorflow/compiler/xla/service/service.cc:168]   StreamExecutor device (0): GeForce RTX 2080 Ti, Compute Capability 7.5
2019-04-19 01:31:03.082447: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1433] Found device 0 with properties: 
name: GeForce RTX 2080 Ti major: 7 minor: 5 memoryClockRate(GHz): 1.545
pciBusID: 0000:83:00.0
totalMemory: 10.73GiB freeMemory: 10.57GiB
2019-04-19 01:31:03.082535: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1512] Adding visible gpu devices: 0
2019-04-19 01:31:03.865351: I tensorflow/core/common_runtime/gpu/gpu_device.cc:984] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-04-19 01:31:03.865420: I tensorflow/core/common_runtime/gpu/gpu_device.cc:990]      0 
2019-04-19 01:31:03.865434: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1003] 0:   N 
2019-04-19 01:31:03.866018: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10141 MB memory) -> physical GPU (device: 0, name: GeForce RTX 2080 Ti, pci bus id: 0000:83:00.0, compute capability: 7.5)
Encoder: layernorm_L2 infer
Decoder: layernorm_L2 infer
WARNING:tensorflow:From /home/abc/pycharm_proj/OpenSeq2Seq_raw/open_seq2seq/parts/transformer/beam_search.py:426: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
WARNING:tensorflow:From /home/abc/pycharm_proj/OpenSeq2Seq_raw/open_seq2seq/parts/transformer/beam_search.py:421: calling reduce_logsumexp_v1 (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version.
Instructions for updating:
keep_dims is deprecated, use keepdims instead
*** Inference Mode. Loss part of graph isn't built.
WARNING:tensorflow:From /usr/local/lib/python3.5/dist-packages/tensorflow/python/training/saver.py:1266: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
WARNING:tensorflow:From /home/abc/pycharm_proj/OpenSeq2Seq_raw/open_seq2seq/models/model.py:617: convert_variables_to_constants (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.compat.v1.graph_util.convert_variables_to_constants
WARNING:tensorflow:From /usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/graph_util_impl.py:245: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.compat.v1.graph_util.extract_sub_graph
Converting graph using TensorFlow-TensorRT...
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/importer.py", line 426, in import_graph_def
    graph._c_graph, serialized, options)  # pylint: disable=protected-access
tensorflow.python.framework.errors_impl.InvalidArgumentError: Input 0 of node ForwardPass/transformer_decoder/decode/while/layer_0/self_attention/self_attention/q/Tensordot/ReadVariableOp/Enter was passed float from ForwardPass/transformer_decoder/layer_0/self_attention/self_attention/q/kernel:0 incompatible with expected resource.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/abc/pycharm_proj/OpenSeq2Seq_raw/run.py", line 101, in <module>
    main()
  File "/home/abc/pycharm_proj/OpenSeq2Seq_raw/run.py", line 81, in main
    args, base_config, config_module, base_model, hvd, checkpoint)
  File "/home/abc/pycharm_proj/OpenSeq2Seq_raw/open_seq2seq/utils/utils.py", line 790, in create_model
    model.compile(checkpoint=checkpoint)
  File "/home/abc/pycharm_proj/OpenSeq2Seq_raw/open_seq2seq/models/model.py", line 445, in compile
    checkpoint=checkpoint
  File "/home/abc/pycharm_proj/OpenSeq2Seq_raw/open_seq2seq/models/model.py", line 647, in build_trt_forward_pass_graph
    maximum_cached_engines=trt_params["trt_maximum_cached_engines"]
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/contrib/tensorrt/python/trt_convert.py", line 333, in create_inference_graph
    importer.import_graph_def(input_graph_def, name="")
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/importer.py", line 430, in import_graph_def
    raise ValueError(str(e))
ValueError: Input 0 of node ForwardPass/transformer_decoder/decode/while/layer_0/self_attention/self_attention/q/Tensordot/ReadVariableOp/Enter was passed float from ForwardPass/transformer_decoder/layer_0/self_attention/self_attention/q/kernel:0 incompatible with expected resource.

Process finished with exit code 1
2020zyc commented 5 years ago

P.S: my modification now image

What is "was passed float"? I also encountered a similar problem before like "was passed half from xxx"? Is it a dtype problem?

2020zyc commented 5 years ago

I try an alternative method using UFF(##first train a model using TensorFlow, freeze the model and write it to a protobuf file, convert it to UFF, and finally run inference using TensorRT.).

With independent tensorRT5.1.2.2, tensorflow1.13, cuda10.1. After get the frozen_graph (p.s the content in the blue frame above is removed), I serialize it as .pb file.

The serialize code: image

Then use "convert-to-uff" utility to generate the .uff file(convert-to-uff xxx.pb).

An Error occurs during the convert: uff.model.exceptions.UffException: Transpose permutation has op ConcatV2, expected Const. Only constant permuations are supported in UFF.

Loading xxx.pb
NOTE: UFF has been tested with TensorFlow 1.12.0. Other versions are not guaranteed to work
WARNING: The version of TensorFlow installed on this system is not guaranteed to work with UFF.
UFF Version 0.6.3
=== Automatically deduced input nodes ===
[name: "input_map_0"
op: "Placeholder"
attr {
  key: "dtype"
  value {
    type: DT_INT32
  }
}
attr {
  key: "shape"
  value {
    shape {
      dim {
        size: -1
      }
      dim {
        size: -1
      }
    }
  }
}
]
=========================================

=== Automatically deduced output nodes ===
[name: "ForwardPass/transformer_decoder/decode/strided_slice_13"
op: "StridedSlice"
input: "ForwardPass/transformer_decoder/decode/Select"
input: "ForwardPass/transformer_decoder/decode/strided_slice_13/stack"
input: "ForwardPass/transformer_decoder/decode/strided_slice_13/stack_1"
input: "ForwardPass/transformer_decoder/decode/strided_slice_13/stack_2"
attr {
  key: "Index"
  value {
    type: DT_INT32
  }
}
attr {
  key: "T"
  value {
    type: DT_INT32
  }
}
attr {
  key: "begin_mask"
  value {
    i: 1
  }
}
attr {
  key: "ellipsis_mask"
  value {
    i: 0
  }
}
attr {
  key: "end_mask"
  value {
    i: 5
  }
}
attr {
  key: "new_axis_mask"
  value {
    i: 0
  }
}
attr {
  key: "shrink_axis_mask"
  value {
    i: 2
  }
}
]
==========================================

Using output node ForwardPass/transformer_decoder/decode/strided_slice_13
Converting to UFF graph
Warning: No conversion function registered for layer: Select yet.
Converting ForwardPass/transformer_decoder/decode/Select as custom op: Select
Warning: No conversion function registered for layer: Exit yet.
Converting ForwardPass/transformer_decoder/decode/while/Exit_9 as custom op: Exit
Warning: No conversion function registered for layer: Switch yet.
Converting ForwardPass/transformer_decoder/decode/while/Switch_9 as custom op: Switch
Warning: No conversion function registered for layer: LoopCond yet.
Converting ForwardPass/transformer_decoder/decode/while/LoopCond as custom op: LoopCond
Warning: No conversion function registered for layer: LogicalAnd yet.
Converting ForwardPass/transformer_decoder/decode/while/LogicalAnd as custom op: LogicalAnd
Warning: No conversion function registered for layer: LogicalNot yet.
Converting ForwardPass/transformer_decoder/decode/while/LogicalNot as custom op: LogicalNot
Warning: No conversion function registered for layer: All yet.
Converting ForwardPass/transformer_decoder/decode/while/All as custom op: All
Warning: No conversion function registered for layer: Merge yet.
Converting ForwardPass/transformer_decoder/decode/while/Merge as custom op: Merge
Warning: No conversion function registered for layer: NextIteration yet.
Converting ForwardPass/transformer_decoder/decode/while/NextIteration as custom op: NextIteration
Warning: No conversion function registered for layer: GatherNd yet.
Converting ForwardPass/transformer_decoder/decode/while/GatherNd_11 as custom op: GatherNd
Warning: No conversion function registered for layer: TopKV2 yet.
Converting ForwardPass/transformer_decoder/decode/while/TopKV2_1 as custom op: TopKV2
Warning: No conversion function registered for layer: Switch yet.
Converting ForwardPass/transformer_decoder/decode/while/Switch as custom op: Switch
Warning: No conversion function registered for layer: Cast yet.
Converting ForwardPass/transformer_decoder/decode/while/ToFloat_3 as custom op: Cast
Warning: No conversion function registered for layer: Equal yet.
Converting ForwardPass/transformer_decoder/decode/while/Equal as custom op: Equal
Warning: No conversion function registered for layer: FloorMod yet.
Converting ForwardPass/transformer_decoder/decode/while/mod as custom op: FloorMod
Warning: No conversion function registered for layer: TopKV2 yet.
Converting ForwardPass/transformer_decoder/decode/while/TopKV2 as custom op: TopKV2
Warning: No conversion function registered for layer: Switch yet.
Converting ForwardPass/transformer_decoder/decode/while/Switch_8 as custom op: Switch
Warning: No conversion function registered for layer: Merge yet.
Converting ForwardPass/transformer_decoder/decode/while/Merge_8 as custom op: Merge
Warning: No conversion function registered for layer: NextIteration yet.
Converting ForwardPass/transformer_decoder/decode/while/NextIteration_8 as custom op: NextIteration
Warning: No conversion function registered for layer: GatherNd yet.
Converting ForwardPass/transformer_decoder/decode/while/GatherNd_10 as custom op: GatherNd
Warning: No conversion function registered for layer: Enter yet.
Converting ForwardPass/transformer_decoder/decode/while/Enter_8 as custom op: Enter
Warning: No conversion function registered for layer: Tile yet.
Converting ForwardPass/transformer_decoder/decode/Tile_1 as custom op: Tile
Warning: No conversion function registered for layer: Fill yet.
Converting ForwardPass/transformer_decoder/decode/zeros as custom op: Fill
Warning: keepdims is ignored by the UFF Parser and defaults to True
Warning: keepdims is ignored by the UFF Parser and defaults to True
Warning: keepdims is ignored by the UFF Parser and defaults to True
Warning: keepdims is ignored by the UFF Parser and defaults to True
Warning: No conversion function registered for layer: BatchMatMul yet.
Converting ForwardPass/transformer_encoder/encode/layer_2/self_attention/self_attention/MatMul_1 as custom op: BatchMatMul
Warning: keepdims is ignored by the UFF Parser and defaults to True
Warning: keepdims is ignored by the UFF Parser and defaults to True
Warning: keepdims is ignored by the UFF Parser and defaults to True
Warning: keepdims is ignored by the UFF Parser and defaults to True
Warning: No conversion function registered for layer: BatchMatMul yet.
Converting ForwardPass/transformer_encoder/encode/layer_1/self_attention/self_attention/MatMul_1 as custom op: BatchMatMul
Warning: keepdims is ignored by the UFF Parser and defaults to True
Warning: keepdims is ignored by the UFF Parser and defaults to True
Warning: keepdims is ignored by the UFF Parser and defaults to True
Warning: keepdims is ignored by the UFF Parser and defaults to True
Warning: No conversion function registered for layer: BatchMatMul yet.
Converting ForwardPass/transformer_encoder/encode/layer_0/self_attention/self_attention/MatMul_1 as custom op: BatchMatMul
Warning: keepdims is ignored by the UFF Parser and defaults to True
Warning: keepdims is ignored by the UFF Parser and defaults to True
Warning: No conversion function registered for layer: Cast yet.
Converting ForwardPass/transformer_encoder/encode/add_pos_encoding/Cast_1 as custom op: Cast
Warning: No conversion function registered for layer: Cast yet.
Converting ForwardPass/transformer_encoder/encode/add_pos_encoding/Cast_2 as custom op: Cast
Warning: No conversion function registered for layer: Range yet.
Converting ForwardPass/transformer_encoder/encode/add_pos_encoding/range_1 as custom op: Range
Warning: No conversion function registered for layer: Cast yet.
Converting ForwardPass/transformer_encoder/encode/add_pos_encoding/Cast as custom op: Cast
Warning: No conversion function registered for layer: Range yet.
Converting ForwardPass/transformer_encoder/encode/add_pos_encoding/range as custom op: Range
Warning: No conversion function registered for layer: Cast yet.
Converting ForwardPass/transformer_encoder/encode/embedding_shared_weights/embedding/padding/Cast as custom op: Cast
Warning: No conversion function registered for layer: Equal yet.
Converting ForwardPass/transformer_encoder/encode/embedding_shared_weights/embedding/padding/Equal as custom op: Equal
Warning: No conversion function registered for layer: Cast yet.
Converting ForwardPass/transformer_encoder/encode/embedding_shared_weights/embedding/Cast as custom op: Cast
Warning: No conversion function registered for layer: Greater yet.
Converting ForwardPass/transformer_encoder/encode/embedding_shared_weights/embedding/Greater as custom op: Greater
Warning: keepdims is ignored by the UFF Parser and defaults to True
Warning: keepdims is ignored by the UFF Parser and defaults to True
Traceback (most recent call last):
  File "/usr/local/bin/convert-to-uff", line 10, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.5/dist-packages/uff/bin/convert_to_uff.py", line 89, in main
    debug_mode=args.debug
  File "/usr/local/lib/python3.5/dist-packages/uff/converters/tensorflow/conversion_helpers.py", line 233, in from_tensorflow_frozen_model
    return from_tensorflow(graphdef, output_nodes, preprocessor, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/uff/converters/tensorflow/conversion_helpers.py", line 181, in from_tensorflow
    debug_mode=debug_mode)
  File "/usr/local/lib/python3.5/dist-packages/uff/converters/tensorflow/converter.py", line 94, in convert_tf2uff_graph
    uff_graph, input_replacements, debug_mode=debug_mode)
  File "/usr/local/lib/python3.5/dist-packages/uff/converters/tensorflow/converter.py", line 79, in convert_tf2uff_node
    op, name, tf_node, inputs, uff_graph, tf_nodes=tf_nodes, debug_mode=debug_mode)
  File "/usr/local/lib/python3.5/dist-packages/uff/converters/tensorflow/converter.py", line 47, in convert_layer
    return cls.registry_[op](name, tf_node, inputs, uff_graph, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/uff/converters/tensorflow/converter_functions.py", line 320, in convert_transpose
    raise UffException("Transpose permutation has op " + str(tf_permutation_node.op) + ", expected Const. Only constant permuations are supported in UFF.")
uff.model.exceptions.UffException: Transpose permutation has op ConcatV2, expected Const. Only constant permuations are supported in UFF.
trevor-m commented 5 years ago

Sorry, you will have to update my previous suggestion. This will convert the type that the op expects from DT_(type)_REF to DT_(type). These issues are from bugs in the convert_variables_to_constant tool. It's possible with a newer version of TF that it would be fixed.

for node in frozen_graph.node:
  # ...
  if node.op == 'RefEnter':
    node.op = 'Enter'
    if node.attr["T"].type > 100:
      node.attr["T"].type -= 100

One thing I should mention is that Transformer in not supported in TF-TRT for TF 1.13.1 (completely unrelated reasons to this issue). You will have to use a newer build from source or tf-nightly-gpu. We are still working on supporting transformer in TF-TRT. Transformer will work in TF 1.14, but there may not be much performance gain.

2020zyc commented 5 years ago

thanks very much，I will try it.

yjiangling commented 4 years ago

thanks very much，I will try it.

So did you solve the error when conduct model transform from pb to uff?

uff.model.exceptions.UffException: Transpose permutation has op ConcatV2, expected Const. Only constant permuations are supported in UFF.

How to solve it? Thanks a lot!