[Request] Example workflows for SD3 models.

[syddharth]

Are there some specific settings to get the SD3 medium models to work? I tried both dynamic and static methods but the compiled engines produce only noise or black images.

[al-swaiti]

same here !

[yusing]

I just made a pull request for it.

sd3-TensorRT-empty_prompt.json Here you go

Or

https://github.com/yusing/ComfyUI_TensorRT/tree/master/workflows

[gseth]

I am still getting black images using the same workflow.

[yusing]

I am still getting black images using the same workflow.

Multiple possible reasons

1. You didn't enter anything in the prompts
2. Loading the wrong CLIP? Use those from SD3, not SDXL.
3. Loading the wrong model, should be plain SD3 Medium, not the one with CLIP.
4. Loading the wrong vae. Do not use vae from SDXL. Dump vae first, then select it in the "load vae" node.

[geroldmeisinger]

I am still getting black images using the same workflow.

try it again (I don't know why but it works sometime).

[geroldmeisinger]

@yusing Thank you so much!

1. how to create the TensorRT engines (missing workflow)? I just used the SDXL one.
2. is the TensorRT version supposed to be reproducable, meaning should it create the same image (with same settings) as the non-rt version?
3. how to load the official TensorRT models? https://github.com/comfyanonymous/ComfyUI_TensorRT/issues/23#issuecomment-2164667520
4. btw you have a lora linked in your workflow

[gseth]

I am still getting black images using the same workflow.

Multiple possible reasons

1. You didn't enter anything in the prompts
2. Loading the wrong CLIP? Use those from SD3, not SDXL.
3. Loading the wrong model, should be plain SD3 Medium, not the one with CLIP.
4. Loading the wrong vae. Do not use vae from SDXL. Dump vae first, then select it in the "load vae" node.

Yeah everything is correct, I re trained the engine. With 1024 and Batch 1. Now when I use Dual Clip like in your workflow, I get no command errors, however in queing a batch of 20, 2-3 are noise or black. At that point it gives an error in cmd for that que.

Now when I use the triple clip, I get this in the command prompt:

[TRT] [E] 3: [executionContext.cpp::nvinfer1::rt::ExecutionContext::setInputShape::2068] Error Code 3: API Usage Error (Parameter check failed at: runtime/api/executionContext.cpp::nvinfer1::rt::ExecutionContext::setInputShape::2068, condition: satisfyProfile Runtime dimension does not satisfy any optimization profile.)

However its working and behavior is same like the dual clip. I get 2-3 black images out of 20.

[yusing]

I am still getting black images using the same workflow.

Multiple possible reasons

1. You didn't enter anything in the prompts
2. Loading the wrong CLIP? Use those from SD3, not SDXL.
3. Loading the wrong model, should be plain SD3 Medium, not the one with CLIP.
4. Loading the wrong vae. Do not use vae from SDXL. Dump vae first, then select it in the "load vae" node.

Yeah everything is correct, I re trained the engine. With 1024 and Batch 1. Now when I use Dual Clip like in your workflow, I get no command errors, however in queing a batch of 20, 2-3 are noise or black. At that point it gives an error in cmd for that que.

Now when I use the triple clip, I get this in the command prompt:

[TRT] [E] 3: [executionContext.cpp::nvinfer1::rt::ExecutionContext::setInputShape::2068] Error Code 3: API Usage Error (Parameter check failed at: runtime/api/executionContext.cpp::nvinfer1::rt::ExecutionContext::setInputShape::2068, condition: satisfyProfile Runtime dimension does not satisfy any optimization profile.)

However its working and behavior is same like the dual clip. I get 2-3 black images out of 20.

That may be a bug with the TensorRT extension itself, I see many spaghetti codes inside this repo.

[yusing]

@yusing Thank you so much!

1. how to create the TensorRT engines (missing workflow)? I just used the SDXL one.
2. is the TensorRT version supposed to be reproducable, meaning should it create the same image (with same settings) as the non-rt version?
3. how to load the official TensorRT models? https://github.com/comfyanonymous/ComfyUI_TensorRT/issues/23#issuecomment-2164667520
4. btw you have a lora linked in your workflow

1. Same as SDXL's workflow
2. I think it should, if this extension is implemented correctly. The author may answer you better than me.
3. Idk
4. I have made it "bypassed", once lora support is added, you can activate and use it.

[geroldmeisinger]

25 all SD3 engines generate black images randomly (almost all of the time)

[gseth]

The black issue has been fixed. Works great with SD3, and the workflow given here. Update Comfy and tensor RT nodes.

https://github.com/comfyanonymous/ComfyUI_TensorRT/issues/7#issuecomment-2174013968