search

NVIDIA / TensorRT

NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.
https://developer.nvidia.com/tensorrt
Apache License 2.0
10.87k stars 2.14k forks source link

!config.getFlag(BuilderFlag::kFP16). ) failure of TensorRT 10.6 when running demo_txt2img_flux.py on GPU h20 #4253

Open yja1 opened 1 week ago

yja1 commented 1 week ago

I want to run this demo in torch.float16, so I change utilities.py: engine = engine_from_network( network, config=CreateConfig(fp16=True, bf16=False, tf32=False, int8=False, fp8=False, refittable=enable_refit, profiles=[p], load_timing_cache=timing_cache, **extra_build_args ), save_timing_cache=timing_cache )

error: [E] IBuilder::buildSerializedNetwork: Error Code 3: API Usage Error (Parameter check failed, condition: !config.getFlag(BuilderFlag::kFP16). ) [!] Invalid Engine. Please ensure the engine was built correctly polygraphy.exception.exception.PolygraphyException: Invalid Engine. Please ensure the engine was built correctly /usr/lib/python3.10/tempfile.py:1008: ResourceWarning: Implicitly cleaning up <TemporaryDirectory '/tmp/tmpin7obfa_'>

Environment

cuda-python 12.6.0 cupy-cuda12x 13.2.0 dask-cuda 24.8.0 nvidia-dali-cuda120 1.41.0 pytorch-triton 3.0.0+dedb7bdf3 torch 2.5.0a0+b465a5843b.nv24.9 torch_tensorrt 2.5.0a0 torchprofile 0.0.4 torchvision 0.20.0a0

lix19937 commented 2 days ago

Invalid Engine. Please ensure the engine was built correctly.

TRy to step debug.