I'm getting the following error when trying to build TensorRT engine with dynamic shape flag enabled. Tried combination with other flags, but it doesn't seem to matter and dynamic shape is probably the reason. Builds fine without it, but I need dynamic shape to work to use Highres fix.
[I] Building engine with configuration:
Flags | [FP16, REFIT]
Engine Capability | EngineCapability.DEFAULT
Memory Pools | [WORKSPACE: 8406.00 MiB, TACTIC_DRAM: 24563.38 MiB]
Tactic Sources | []
Profiling Verbosity | ProfilingVerbosity.DETAILED
Preview Features | [FASTER_DYNAMIC_SHAPES_0805, DISABLE_EXTERNAL_TACTIC_SOURCES_FOR_CORE_0805]
[E] 10: Could not find any implementation for node {ForeignNode[onnx::LayerNormalization_9032 + (Unnamed Layer* 1229) [Shuffle].../down_blocks.0/attentions.0/Reshape_1 + /down_blocks.0/attentions.0/Transpose_1]}.
[E] 10: [optimizer.cpp::nvinfer1::builder::cgraph::LeafCNode::computeCosts::3869] Error Code 10: Internal Error (Could not find any implementation for node {ForeignNode[onnx::LayerNormalization_9032 + (Unnamed Layer* 1229) [Shuffle].../down_blocks.0/attentions.0/Reshape_1 + /down_blocks.0/attentions.0/Transpose_1]}.)
[!] Invalid Engine. Please ensure the engine was built correctly
Traceback (most recent call last):
File "C:\Users\Princess\projects\Radiata\venv\lib\site-packages\gradio\routes.py", line 399, in run_predict
output = await app.get_blocks().process_api(
File "C:\Users\Princess\projects\Radiata\venv\lib\site-packages\gradio\blocks.py", line 1299, in process_api
result = await self.call_function(
File "C:\Users\Princess\projects\Radiata\venv\lib\site-packages\gradio\blocks.py", line 1036, in call_function
prediction = await anyio.to_thread.run_sync(
File "C:\Users\Princess\projects\Radiata\venv\lib\site-packages\anyio\to_thread.py", line 31, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "C:\Users\Princess\projects\Radiata\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 937, in run_sync_in_worker_thread
return await future
File "C:\Users\Princess\projects\Radiata\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 867, in run
result = context.run(func, *args)
File "C:\Users\Princess\projects\Radiata\venv\lib\site-packages\gradio\utils.py", line 488, in async_iteration
return next(iterator)
File "C:\Users\Princess\projects\Radiata\modules\tabs\tensorrt.py", line 173, in build_engine
builder.build()
File "C:\Users\Princess\projects\Radiata\modules\acceleration\tensorrt\engine.py", line 99, in build
build_engine(
File "C:\Users\Princess\projects\Radiata\lib\tensorrt\utilities.py", line 402, in build_engine
engine.build(
File "C:\Users\Princess\projects\Radiata\lib\tensorrt\utilities.py", line 241, in build
engine = engine_from_network(
File "<string>", line 3, in func_impl
File "C:\Users\Princess\projects\Radiata\venv\lib\site-packages\polygraphy\backend\base\loader.py", line 42, in __call__
return self.call_impl(*args, **kwargs)
File "C:\Users\Princess\projects\Radiata\venv\lib\site-packages\polygraphy\backend\trt\loader.py", line 526, in call_impl
return engine_from_bytes(super().call_impl)
File "<string>", line 3, in func_impl
File "C:\Users\Princess\projects\Radiata\venv\lib\site-packages\polygraphy\backend\base\loader.py", line 42, in __call__
return self.call_impl(*args, **kwargs)
File "C:\Users\Princess\projects\Radiata\venv\lib\site-packages\polygraphy\backend\trt\loader.py", line 550, in call_impl
buffer, owns_buffer = util.invoke_if_callable(self._serialized_engine)
File "C:\Users\Princess\projects\Radiata\venv\lib\site-packages\polygraphy\util\util.py", line 661, in invoke_if_callable
ret = func(*args, **kwargs)
File "C:\Users\Princess\projects\Radiata\venv\lib\site-packages\polygraphy\backend\trt\loader.py", line 484, in call_impl
G_LOGGER.critical("Invalid Engine. Please ensure the engine was built correctly")
File "C:\Users\Princess\projects\Radiata\venv\lib\site-packages\polygraphy\logger\logger.py", line 597, in critical
raise PolygraphyException(message) from None
Describe the bug
I'm getting the following error when trying to build TensorRT engine with dynamic shape flag enabled. Tried combination with other flags, but it doesn't seem to matter and dynamic shape is probably the reason. Builds fine without it, but I need dynamic shape to work to use Highres fix.
Reproduction
Fresh clean install
Expected behavior
Building engine without an error
System Info
Additional context
No response
Validations