search

city96 / ComfyUI_ExtraModels

Support for miscellaneous image models. Currently supports: DiT, PixArt, HunYuanDiT, MiaoBi, and a few VAEs.
Apache License 2.0
337 stars 31 forks source link

Hires fix? #26

Open drphero opened 3 months ago

drphero commented 3 months ago

Is it possible to use the PixArt models with a hires fix as part of the workflow? I'm not sure if special adjustments are necessary, but doing it the usual way of upscaling the image then running it through a second ksampler with lower denoise produces pretty bad results. Is this just not possible with the PixArt models?

city96 commented 3 months ago

These models only work (well) on the resolutions they were trained on as far as I can tell. You can try to use one of the tiled sampling solutions like "UltimateSDUpscale" with the tile size set to fixed, but I'm not sure the results will be all that great.

drphero commented 3 months ago

That's a great idea! Looks like both Ultimate SD Upscale and Tiled Diffusion work quite well.

I noticed that there is a PixArt ControlNet Conditioning node, I can't find any info on how to use it. I was trying to use a tile controlnet during the upscale, but it throws an error with or with out the node (although I'm not sure I was connecting it correctly).

city96 commented 3 months ago

The PixArt team only released one controlnet so far as far as I know, and those conditioning nodes are meant to be used with that one (see issue #13 ). The regular SD controlnet models won't work sadly since they're for a different architecture.

drphero commented 3 months ago

Ah I see now. It looks like it works extremely differently. They don't seem to say anywhere what type of controlnet it is, but I'm guessing it won't work as a tile one.