[FR] Clip Skip

[Succubyss]

I was going to implement this myself, but I'm having a hard time figuring out the way clip skip works in cog. They apparently added a parameter for it at some point, but I think their implementation makes doing a Clip Skip of 2 (as CIvitAI would call it) equivalent to clip_skip=1? And Clip Skip 1 (which CivitAI would consider default) would equivalent to no value or 0.

But I can't say for certain. And I think the implementation of this into cog-sdxl should use CivitAI's idea of what the clip skip values should be, otherwise people will be putting in a clip skip of 2 but they're actually getting a clip skip of what CivitAI considers a 3.

[LagPixelLOL]

Added in https://github.com/LagPixelLOL/cog-sdxl/commit/3850a3006ecd288f847a071831c5064c543487b5.

I aligned the value to the HuggingFace Diffusers' idea of CLIP skip, which 0 is no skip, 1 is skip 1 layer, and 2 is skip 2 layers, etc..

[Succubyss]

Added in 3850a30.

I aligned the value to the HuggingFace Diffusers' idea of CLIP skip, which 0 is no skip, 1 is skip 1 layer, and 2 is skip 2 layers, etc..

So Clip Skip 2 in something like CivitAI's web interface or AUTOMATIC1111 is equivalent to clip skip 1 here? Can't say I'm a fan of that, it'd be better to go with the most universally used idea of what clip skip is.

But, your call I s'pose.

[LagPixelLOL]

Welp but CLIP skip is basically skipping CLIP layers, I don't think skipping 2 is actually skipping 1 layer is a good idea.

I can dig into the code of vladmandic/automatic and see what he does and align the value to that.

[LagPixelLOL]

https://github.com/vladmandic/automatic/blob/e7ec07f9783701629ca1411ad82aec87232501b9/modules/processing_args.py#L129

Welp vladmandic aligns it to the A1111 value, I guess I need to change it now. Still don't know why tho...

[LagPixelLOL]

https://github.com/LagPixelLOL/cog-sdxl/commit/e8c4b3235040c5765ce6a262c932ead87f8e6aef