Closed andrey-klochkov-liftoff closed 10 months ago
I'm not an expert on TF-TRT, but TRT doesn't support IndexLookup layer, onnx doesn't support this layer too?
Also from your log, I don't think TensorRT gets involved here. so it would be better to ask in the Tensorflow repo.
@zerollzeng
I'm not an expert on TF-TRT, but TRT doesn't support IndexLookup layer, onnx doesn't support this layer too?
When I hit the problem I suspected that neither of the converters support IndexLookup (or rather TF hash tables) but I didn't find any evidence of that. Should this be documented somewhere or explained in the error messages?
Also from your log, I don't think TensorRT gets involved here. so it would be better to ask in the Tensorflow repo.
Good point, somehow I thought the converter (that produces the model that can't be saved) is the scope of TensorRT itself, but you're right it's all in the TF repo so I should've filed this bug there.
So there are several issues in both the TF and TensorRT repos that mention this problem (without anybody clearly confirming that TensorRT or TF-TRT do not support lookup tables indeed). This is the list of issues that cross reference each other (I'm linking to particular comments that say "it seems like" lookup tables are not supported): https://github.com/tensorflow/tensorflow/issues/46254#issuecomment-759866072 https://github.com/tensorflow/tensorrt/issues/233#issuecomment-756354519 https://github.com/tensorflow/text/issues/486#issuecomment-756333011 https://github.com/tensorflow/tensorrt/issues/233#issuecomment-756354519
The last comment from @bixia1 said that was fixed, and she linked the fix I mentioned in the description, which didn't actually fix the problem.
I don't think I should file yet another issue like this in the TF repo.
Able to reproduce this in TF2.9 (using nvidia/tensorflow:22.08-tf2-py3
)
I will closing this since no activity for a long time, also FYI, there are more TF-TRT experts for this issue in https://github.com/tensorflow/tensorrt/issues, thanks!
Description
The TF-TRT converter fails to save the model if it uses a lookup table. This colab illustrates how I create a simple model that uses a Keras
IndexLookup
layer that in turn makes the savedmodel contain a DT_RESOURCE tensor with the lookup table contents. The last cell of the notebook that does the conversion to TensorRT fails because TensorRT is not currently supported on Colab. I use the NVIDIA Tensorflow container to actually convert the model from savedmodel to tensorrt, see "steps to reproduce" below.Does TF-TRT support lookup tables? If not, are there known workarounds?
I tried to search for related bug reports or changes, and found that older versions of TF failed with a different error (see e.g. https://github.com/tensorflow/tensorflow/issues/42673). This commit seems to have changed the error to the one I'm reporting.
I also attempted to convert to ONNX but hit a similar problem, found an existing open bug report and attached a colab reproducing the error there, see https://github.com/onnx/tensorflow-onnx/issues/1867.
Environment
TensorRT Version: 8.2.5-1+cuda11.4 NVIDIA GPU: A10G NVIDIA Driver Version: 470.57.02 (in CUDA Forward Compatibility mode "Using CUDA 11.7 driver version 515.48.08 with kernel driver version 470.57.02") CUDA Version: 11.7 Operating System: Ubuntu 20.04 Python Version (if applicable): 3.8.10 Tensorflow Version (if applicable): 2.9.1 Baremetal or Container (if so, version): nvcr.io/nvidia/tensorflow:22.06-tf2-py3
Steps To Reproduce
Train a model using the code from the colab notebook, store the model to
/models/model
.Start the NVIDIA Tensorflow container with:
docker run --gpus all -it -v/models:/models --rm nvcr.io/nvidia/tensorflow:22.06-tf2-py3
Start the Python REPL and run the following script:
from tensorflow.python.compiler.tensorrt import trt_convert as trt conversion_params = trt.DEFAULT_TRT_CONVERSION_PARAMS converter = trt.TrtGraphConverterV2(input_saved_model_dir="/models/model", conversion_params=conversion_params) converter.convert() converter.save("/models/output")
The error traceback: