Open maedtb opened 4 months ago
I ran into several errors using TensorRT, and it was unclear why these errors were happening. Adding the TensorRT error reporter cleared up the issues for me entirely. The error messages TensorRT report to us look something like this:
[defaultAllocator.cpp::allocate::31] Error Code 1: Cuda Runtime (out of memory)
[executionContext.cpp::ExecutionContext::565] Error Code 2: OutOfMemory (Requested size was 30152807424 bytes.)
It's not the most user friendly, but it's a lot more helpful of an error than a 'NoneType' object has no attribute 'set_input_shape'
:^)
Tested this - it works properly for converting and generating, but an error during generation while it logs properly doesn't raise an exception
so it just generates a black image while spamming console with a new error every step
[07/15/2024-23:30:09] [TRT] [I] [MemUsageStats] Peak memory usage during Engine building and serialization: CPU: 13075 MiB
[07/15/2024-23:30:09] [TRT] [I] Serialized 16605 bytes of code generator cache.
[07/15/2024-23:30:09] [TRT] [I] Serialized 4704604 bytes of compilation cache.
[07/15/2024-23:30:09] [TRT] [I] Serialized 2218 timing cache entries
Segmentation fault
This PR gives me a segfault when I try to convert SD1.5
Just leaving an note here that I'm planning on investigating these issues this weekend and I haven't abandoned this. Going to pull it into Draft in the mean time.
TrTErrorRecorder
which implements the TensorRTIErrorRecorder
interface. This class captures errors to display to the user, and can optionally terminate TensorRT processing when errors occur.error_recorder
field on the TensorRTtensorrt.Builder
andtensorrt.Runtime
classes to an instance ofTrTErrorRecorder
.