search

NVIDIA / TensorRT

NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.
https://developer.nvidia.com/tensorrt
Apache License 2.0
10.79k stars 2.13k forks source link

EfficientDet: 'AttributeError: 'Variable' object has no attribute 'values' #1705

Closed ntakouris closed 2 years ago

ntakouris commented 2 years ago

I am trying to convert the efficientdet model to onnx. With the latest container releases (21.12-py3), the onnx version is updated and the nms plugin is supported. However, I get this 'AttributeError: 'Variable' object has no attribute 'values' on the following line: https://github.com/NVIDIA/TensorRT/blob/96e23978cd6e4a8fe869696d3d8ec2b47120629b/samples/python/efficientdet/create_onnx.py#L277

wraveane commented 2 years ago

Which size/variation of EfficientDet are you trying to convert? And from which source? (AutoML or TensorFlow OD API)

ntakouris commented 2 years ago

Which size/variation of EfficientDet are you trying to convert? And from which source? (AutoML or TensorFlow OD API)

I’ve tried both versions using the instructions from the tensorrt sample. I could not make it work using the PyTorch 21.12-py3 image or the latest tf2 image from nvcr.

As far as I have understood, the example was old and Google changed some sort of semantic labelling of the nodes inside the NNs graph and now graph surgeon can’t find what node to replace.

wraveane commented 2 years ago

So I tried it completely from scratch just now and I could not replicate the error message you mentioned. I tested three versions of EfficientDet-D0:

AutoML: https://storage.googleapis.com/cloud-tpu-checkpoints/efficientdet/coco2/efficientdet-d0.tar.gz (checkpoint file, exported to saved model with the AutoML's model_inspect.py script, as per the converter instructions) TFOD: http://download.tensorflow.org/models/object_detection/tf2/20200711/efficientdet_d0_coco17_tpu-32.tar.gz TFHub: https://storage.googleapis.com/tfhub-modules/tensorflow/efficientdet/d0/1.tar.gz

All three convert properly to ONNX.

I did run on a 22.01 docker, but I don't think that should matter too much. Please make sure your other dependencies are also installed correctly, as a different version of onnx or the other onnx-related python packages could indeed lead to conversion problems. This is the environment I tested with just now:

Docker Container: nvcr.io/nvidia/tensorflow:22.01-tf2-py3 (This is TF 2.7 + TRT 8.2)

Python Packages: onnx==1.9.0 onnx-graphsurgeon==0.3.14 onnxruntime==1.8.1 tf2onnx==1.8.1

You said you're trying on a PyTorch docker, which should be ok in theory, as long as you install a compatible TensorFlow version (I believe any between 2.5 - 2.7 should be good) -- but just in case, please try with the same nvcr.io/nvidia/tensorflow:22.01-tf2-py3 to verify if it works for you.

If you still have problems though, can you please post a copy of the exact model you're trying to convert, and I can try to debug further, as if there's any conversion bugs I'm overlooking, I'd definitely like to get them fixed of course :)

ntakouris commented 2 years ago

Here is the proccess I follow:

  1. Clone this repo git clone https://github.com/NVIDIA/TensorRT.git and cd TensorRT/samples/python/efficientdet

  2. I want custom image size, so we need to rewrite the input part of the graph checkpoints: git clone https://github.com/google/automl.git and pip3 install tensorflow_model_optimization

  3. Download some ckpt to extract saved_model from: wget https://storage.googleapis.com/cloud-tpu-checkpoints/efficientdet/coco2/efficientdet-d0.tar.gz mkdir efficientdet-d0-ckpt && tar -xf efficientdet-d0.tar.gz -C efficientdet-d0-ckpt --strip-components 1 rm efficientdet-d0.tar.gz

  4. Export the model in 1280x1024 image size cd automl/efficientdet python3 model_inspect.py --hparams 'image_size=1280x1024' --runmode saved_model --model_name efficientdet-d0 --ckpt_path ../../efficientdet-d0-ckpt --saved_model_dir ../../efficientdet-d0

  5. Export to ONNX (N = 3) python3 create_onnx.py -m efficientdet-d0 -o efficientdet-d0.onnx -i N,1024,1280,3

  6. Export TRT Engine: python3 build_engine.py --onnx efficientdet-d0.onnx --engine efficientdet-d0.fp16.trt --precision fp16

  7. (Optionally) Run a benchmark sanity test: trtexec --loadEngine=/workspace/TensorRT/samples/python/efficientdet/efficientdet-d0.fp16.trt --useCudaGraph --noDataTransfers --iterations=1000 --avgRuns=1000

I did execute those commands on a fresh tensorflow 21.12-tf2-py3 container, and it works this time. I am not sure if google changed something in the graph again (silently) -- which is the fix (and the changed that caused problems beforehand).

Also, with the new container releases, #1538 should also be fixed.

Thank you @wraveane for the quick response, this thing works as expected now.