search

huggingface / diffusers

🤗 Diffusers: State-of-the-art diffusion models for image and audio generation in PyTorch and FLAX.
https://huggingface.co/docs/diffusers
Apache License 2.0
23.96k stars 4.93k forks source link

Open source upscaler: Clarity #7367

Open tin2tin opened 3 months ago

tin2tin commented 3 months ago

Is your feature request related to a problem? Please describe. Diffusers are missing a good image upscaler.

Describe the solution you'd like. Implement the Clarity-upscaler: Code: https://github.com/philz1337x/clarity-upscaler Demo: https://replicate.com/philipp1337x/clarity-upscaler

https://github.com/huggingface/diffusers/assets/1322593/766a5937-43a0-4857-91c3-331b69f07d8d

sayakpaul commented 3 months ago

Another related: https://github.com/huggingface/diffusers/issues/7219.

asomoza commented 3 months ago

Just a clarification, this is not code, a model or is something new, also IMO the term "reverse engineering" that he used is not correct.

That repo just install a preconfigured auto1111 with some params, you can even use it with just this if you have installed:

masterpiece, best quality, highres, <lora:more_details:0.5> <lora:SDXLrender_v2.0:1> Negative prompt: (worst quality, low quality, normal quality:2) JuggernautNegative-neg Steps: 18, Sampler: DPM++ 3M SDE Karras, CFG scale: 6.0, Seed: 1337, Size: 1024x1024, Model hash: 338b85bc4f, Model: juggernaut_reborn, Denoising strength: 0.35, Tiled Diffusion upscaler: 4x-UltraSharp, Tiled Diffusion scale factor: 2, Tiled Diffusion: {"Method": "MultiDiffusion", "Tile tile width": 112, "Tile tile height": 144, "Tile Overlap": 4, "Tile batch size": 8, "Upscaler": "4x-UltraSharp", "Upscale factor": 2, "Keep input size": true}, ControlNet 0: "Module: tile_resample, Model: control_v11f1e_sd15_tile, Weight: 0.6, Resize Mode: 1, Low Vram: False, Processor Res: 512, Threshold A: 1, Threshold B: 1, Guidance Start: 0.0, Guidance End: 1.0, Pixel Perfect: True, Control Mode: 1, Hr Option: HiResFixOption.BOTH, Save Detected Map: False", Lora hashes: "more_details: 3b8aa1d351ef, SDXLrender_v2.0: 3925cf4759af"

Also I have some doubts with the term upscaler, I get that it can upscale but it changes too much of the original image, for example, these are totally different persons:

before upscale uspcaled
image image

I'm glad that the author wants to share his knowledge but I don't see anything that we can add to diffusers with this, since it is basically a technique for auto1111.

I agree in that diffusers lacks a good upscaler solution but hopefully SUPIR will come soon and I also intend to do one with the new tile controlnet.

tin2tin commented 3 months ago

Ah, thanks for investigating this.

OlegRuban-ai commented 3 months ago

How about this: https://github.com/Fanghua-Yu/SUPIR/issues/38 - Supir Upscale Image?

If you can download in 12 GB or even less, then there will be a good indicator. Is this possible to implement in diffusers?

sayakpaul commented 3 months ago

See https://github.com/huggingface/diffusers/issues/7219/

github-actions[bot] commented 2 months ago

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.