comfyanonymous / ComfyUI_TensorRT

MIT License
504 stars 34 forks source link

i need LORA support #42

Open sev7eny opened 4 months ago

sev7eny commented 4 months ago

please add lora ande controlnet support!!!

gseth commented 4 months ago

LoRA support is cheesed in. When building the engine, connect it with a Lora. Now, the engine model will have the Lora baked in. In the final workflow, you won't be able to change the Lora Strength, though.

BigPeng commented 3 months ago

LoRA support is cheesed in. When building the engine, connect it with a Lora. Now, the engine model will have the Lora baked in. In the final workflow, you won't be able to change the Lora Strength, though.

it works, but the result is worse

BigPeng commented 3 months ago

LoRA support is cheesed in. When building the engine, connect it with a Lora. Now, the engine model will have the Lora baked in. In the final workflow, you won't be able to change the Lora Strength, though.

it works, but the result is worse

I found that tensorrt only supports 77 tokens. If it exceeds the limit, an error will be reported. The exceeded prompt part is invalid, resulting in inconsistent picture results. Is there any way to support more than 77 tokens?

Error Code 3: API Usage Error (Parameter check failed, condition: engineDims.d[i] == dims.d[i]. Static dimension mismatch while setting input shape for context. Set dimensions are [1,154,2048]. Expected dimensions are [1,77,2048].)

comfyanonymous commented 3 months ago

Is there any way to support more than 77 tokens?

Yes, use the node to build a dynamic engine and increase context_max to more than 1. The max amount of tokens is context_max * 77

catboxanon commented 1 week ago

FYI, on-the-fly LoRA support for TensorRT has been available in the web UI since October 2023. https://github.com/NVIDIA/Stable-Diffusion-WebUI-TensorRT