search

NVIDIA / Stable-Diffusion-WebUI-TensorRT

TensorRT Extension for Stable Diffusion Web UI
MIT License
1.9k stars 145 forks source link

Enabling PyTorch fallback as no engine was found. #254

Open Nightnightlight opened 8 months ago

Nightnightlight commented 8 months ago

Just keeps saying "Enabling PyTorch fallback as no engine was found" when I try and use a TensorRT engine for a SDXL model. Tried reinstalling the extension, deleting venv and starting over and rebuilding the engines several times. Its SDXL 1024x1024 min and max, 1 batch size min/max, 75 min and 750 max prompt length. Every extension is turned off except for TensorRT. SD Unet is set to automatic though I also tried selecting the model itself which still did not work. Every other setting is default on a fresh automatic1111 install. Profile Settings prompt

Note: After much testing it seems like TensorRT for SDXL simply can not support higher than a 75 token max period. If you make one with a higher max token count than the default 75 it refuses to use that model.

maxxrox commented 8 months ago

Think this is the same for SD1.5 ONNX/TRT models as of the 0.2.0 release of the NVidia TensorRT extension in Automatic1111. I'm not sure what breaking bugs were introduced, but it's been a pretty painful release.

See these two issues with similar results: [(https://github.com/NVIDIA/Stable-Diffusion-WebUI-TensorRT/issues/230)], [(https://github.com/NVIDIA/Stable-Diffusion-WebUI-TensorRT/issues/248)]

MC-dusk commented 8 months ago

I'm using sd1.5 model and having same problem. No matter what image resolution I set, like static 768x768 or dynamic 512~1024 with opt=768, it always says Enabling PyTorch fallback as no engine was found and use normal way to generate. However, default mode can work. That's really strange. My picture output setting remains unchanged at 768x768.

I've tried a couple of arguments and find that it may caused by max batch size 4 which I changed to 1 before and opt token 75 which I changed to 150 before. After keeping those two arguments as default I can change resolution to 768 and 1024 while TensorRT works fine.

FurkanGozukara commented 8 months ago

I'm using sd1.5 model and having same problem. No matter what image resolution I set, like static 768x768 or dynamic 512~1024 with opt=768, it always says Enabling PyTorch fallback as no engine was found and use normal way to generate. However, default mode can work. That's really strange. My picture output setting remains unchanged at 768x768.

I've tried a couple of arguments and find that it may caused by max batch size 4 which I changed to 1 before and opt token 75 which I changed to 150 before. After keeping those two arguments as default I can change resolution to 768 and 1024 while TensorRT works fine.

so max batch size should be 4 and opt token must be 75?

MC-dusk commented 8 months ago

I'm using sd1.5 model and having same problem. No matter what image resolution I set, like static 768x768 or dynamic 512~1024 with opt=768, it always says Enabling PyTorch fallback as no engine was found and use normal way to generate. However, default mode can work. That's really strange. My picture output setting remains unchanged at 768x768. I've tried a couple of arguments and find that it may caused by max batch size 4 which I changed to 1 before and opt token 75 which I changed to 150 before. After keeping those two arguments as default I can change resolution to 768 and 1024 while TensorRT works fine.

so max batch size should be 4 and opt token must be 75?

That's a speculation. I didn't try each combination cause that's too time wasting. Maybe problem is one of them or some bad combinations of them. Anyway, if you are not curious about which combination is fine and just want to have a larger scale, I recommend 512, 768, 960, 1024, 1152, 1536, these are tested by me which are nice multiples of 64, can generate better picture I feel, and easy to pass through.

FurkanGozukara commented 8 months ago

I'm using sd1.5 model and having same problem. No matter what image resolution I set, like static 768x768 or dynamic 512~1024 with opt=768, it always says Enabling PyTorch fallback as no engine was found and use normal way to generate. However, default mode can work. That's really strange. My picture output setting remains unchanged at 768x768. I've tried a couple of arguments and find that it may caused by max batch size 4 which I changed to 1 before and opt token 75 which I changed to 150 before. After keeping those two arguments as default I can change resolution to 768 and 1024 while TensorRT works fine.

so max batch size should be 4 and opt token must be 75?

That's a speculation. I didn't try each combination cause that's too time wasting. Maybe problem is one of them or some bad combinations of them. Anyway, if you are not curious about which combination is fine and just want to have a larger scale, I recommend 512, 768, 960, 1024, 1152, 1536, these are tested by me which are nice multiples of 64, can generate better picture I feel, and easy to pass through.

batch size 4 and opt token 75 also worked for me. definitely a bug going on

Ce-daros commented 5 months ago

same problem

yogesh030 commented 2 months ago

is there any update to this?

Atmic commented 2 months ago

I'm also having this issue as well.

100640002 commented 1 month ago

Warning Enabling PyTorch fallback as no engine was found.

trsh3r commented 1 month ago

same here. still no answer from NVIDIA??