demo/DemoDiffusion could not work: polygraphy.exception.exception.PolygraphyException: Could not deserialize engine.

[artetaout]

• configs
• 8 * NVIDIA A100-SXM4-80GB
• Driver Version: 470.161.03
• completely follow steps from https://github.com/NVIDIA/TensorRT/tree/release/8.6/demo/Diffusion
• the logs

  /workspace/demo/Diffusion# python3 demo_txt2img.py "a beautiful photograph of Mt. Fuji during cherry blossom" --hf-token=$HF_TOKEN -v
  [I] Initializing StableDiffusion txt2img demo using TensorRT
  Downloading (…)tokenizer/vocab.json: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1.06M/1.06M [00:00<00:00, 1.07MB/s]
  Downloading (…)tokenizer/merges.txt: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 525k/525k [00:00<00:00, 700kB/s]
  Downloading (…)cial_tokens_map.json: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 460/460 [00:00<00:00, 412kB/s]
  Downloading (…)okenizer_config.json: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 824/824 [00:00<00:00, 798kB/s]
  Building TensorRT engine for onnx/vae.opt.onnx: engine/vae.plan
  [W] onnx2trt_utils.cpp:374: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
  [I]     Configuring with profiles: [Profile().add('latent', min=(1, 4, 64, 64), opt=(1, 4, 64, 64), max=(16, 4, 64, 64))]
  [I] Building engine with configuration:
  Flags                  | [FP16]
  Engine Capability      | EngineCapability.DEFAULT
  Memory Pools           | [WORKSPACE: 76738.19 MiB, TACTIC_DRAM: 81251.19 MiB]
  Tactic Sources         | []
  Profiling Verbosity    | ProfilingVerbosity.DETAILED
  Preview Features       | [DISABLE_EXTERNAL_TACTIC_SOURCES_FOR_CORE_0805]
  [W] kFASTER_DYNAMIC_SHAPES_0805 preview feature is disabled.
  [W] Myelin graph with multiple dynamic values may have poor performance if they differ.
  [W] TensorRT encountered issues when converting weights between types and that could affect accuracy.
  [W] If this is not the desired behavior, please modify the weights or retrain with regularization to adjust the magnitude of the weights.
  [W] Check verbose logs for the list of affected weights.
  [W] - 5 weights are affected by this issue: Detected subnormal FP16 values.
  [I] Finished engine building in 1056.613 seconds
  [I] Saving engine to engine/vae.plan
  Loading TensorRT engine: engine/clip.plan
  [I] Loading bytes from engine/clip.plan
  Loading TensorRT engine: engine/unet.plan
  [I] Loading bytes from engine/unet.plan
  [E] 1: [runtime.cpp::parsePlan::315] Error Code 1: Serialization (Serialization assertion plan->header.size == blobSize failed.)
  [!] Could not deserialize engine. See log for details.
  Traceback (most recent call last):
  File "demo_txt2img.py", line 76, in <module>
  demo.loadEngines(args.engine_dir, args.onnx_dir, args.onnx_opset,
  File "/workspace/demo/Diffusion/stable_diffusion_pipeline.py", line 306, in loadEngines
  engine.load()
  File "/workspace/demo/Diffusion/utilities.py", line 220, in load
  self.engine = engine_from_bytes(bytes_from_path(self.engine_path))
  File "<string>", line 3, in engine_from_bytes
  File "/usr/local/lib/python3.8/dist-packages/polygraphy/backend/base/loader.py", line 42, in __call__
  return self.call_impl(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/polygraphy/backend/trt/loader.py", line 568, in call_impl
  G_LOGGER.critical("Could not deserialize engine. See log for details.")
  File "/usr/local/lib/python3.8/dist-packages/polygraphy/logger/logger.py", line 597, in critical
  raise PolygraphyException(message) from None
  polygraphy.exception.exception.PolygraphyException: Could not deserialize engine. See log for details.

[jettan]

It kind of looks like it built the clip and vae engines, but not the unet one... can you try to see that unet.plan has been built?

[zerollzeng]

[E] 1: [runtime.cpp::parsePlan::315] Error Code 1: Serialization (Serialization assertion plan->header.size == blobSize failed.)

Here is why it fails.

[zerollzeng]

I don't think I saw this error before, can you try load the engine with trtexec to see if it work?

[ttyio]

Closing since no activity for more than 3 weeks, pls reopen if you still have question, thanks!

[demuxin]

I had a similar problem and the reason was that the disk was full.

You can refer to it.