comfyanonymous / ComfyUI_TensorRT

MIT License
513 stars 34 forks source link

Add TensorRT `IErrorRecorder` Implementation #54

Open maedtb opened 4 months ago

maedtb commented 4 months ago
maedtb commented 4 months ago

I ran into several errors using TensorRT, and it was unclear why these errors were happening. Adding the TensorRT error reporter cleared up the issues for me entirely. The error messages TensorRT report to us look something like this:

[defaultAllocator.cpp::allocate::31] Error Code 1: Cuda Runtime (out of memory)
[executionContext.cpp::ExecutionContext::565] Error Code 2: OutOfMemory (Requested size was 30152807424 bytes.)

It's not the most user friendly, but it's a lot more helpful of an error than a 'NoneType' object has no attribute 'set_input_shape' :^)

mcmonkey4eva commented 4 months ago

Tested this - it works properly for converting and generating, but an error during generation while it logs properly doesn't raise an exception image

so it just generates a black image while spamming console with a new error every step

comfyanonymous commented 4 months ago
[07/15/2024-23:30:09] [TRT] [I] [MemUsageStats] Peak memory usage during Engine building and serialization: CPU: 13075 MiB
[07/15/2024-23:30:09] [TRT] [I] Serialized 16605 bytes of code generator cache.
[07/15/2024-23:30:09] [TRT] [I] Serialized 4704604 bytes of compilation cache.
[07/15/2024-23:30:09] [TRT] [I] Serialized 2218 timing cache entries
Segmentation fault

This PR gives me a segfault when I try to convert SD1.5 image

maedtb commented 3 months ago

Just leaving an note here that I'm planning on investigating these issues this weekend and I haven't abandoned this. Going to pull it into Draft in the mean time.