Codeformer not working correctly

[ozmix]

Hi, I want to know if there is a solution to Codeformer masking. Every time I use it in a video, the mask always shakes constantly and does not stick to the face at all. Additionally, it occasionally disappears and reappears and sometimes glitches into a white background too.

Furthermore, It appears that during batch processing the image's original face is used and not adjusted to match the new image it just stays static all the time.

I am aware that not everyone might encounter these issues. I just thought I would ask to see if there is a solution or if something needs to be uninstalled or installed.

https://github.com/C0untFloyd/roop-unleashed/assets/88434082/422a6cab-c60d-4500-8405-addbe042d80e

[dnnagy]

I had the same issue with Codeformer, ended up using GFPGAN as DMDNet is too slow.

[Oil3]

The square is not all the time, right? I don't get it, it seems the onnx is buggy, and I found another model -no clue where- that goes literaly 10 times faster for me: the normal codeformer slows down my machine and seems to be unable to tap in the ram.

I uploaded it soon after reading your posts, have a try and let me know ? https://huggingface.co/Oil3/faster_codeformer_onnx https://huggingface.co/Oil3/faster_codeformer_onnx/resolve/main/CodeFormerFaster.onnx Its half the size too. Quality seems 99.9% identical (couple bites differences in filesize from tests) Rename it as CodeFormerv0.1.onnx and swap it with the other one. on the screenshot the ridiculous output is the normal .onnx, it was just a low res photo. very silly

[dnnagy]

@Oil3 I can't download the model from your link.

[ozmix]

The square is not all the time, right? I don't get it, it seems the onnx is buggy, and I found another model -no clue where- that goes literaly 10 times faster for me: the normal codeformer slows down my machine and seems to be unable to tap in the ram.

I uploaded it soon after reading your posts, have a try and let me know. https://huggingface.co/Oil3/faster_codeformer_onnx https://huggingface.co/Oil3/faster_codeformer_onnx/resolve/main/CodeFormerFaster.onnx Its half the size too. Quality seems 99.9% identical (couple bites differences in filesize from tests) Rename it as CodeFormerv0.1.onnx and swap it with the other one. on the screenshot the ridiculous output is the normal .onnx, it was just a low res photo. very silly

okay so I tested it both the original and faster version provided by you oil and its still buggy

https://github.com/C0untFloyd/roop-unleashed/assets/88434082/1210d678-c3d1-4a4d-91c8-ae013469139f

https://github.com/C0untFloyd/roop-unleashed/assets/88434082/e1d8a2c9-127f-4e1f-bb44-2d4ef39d0d79

the faster version is sharper and faster of course which is great. but for whatever reason it is buggy still that mask is an issue.

but yeah have a look a in cases you notice any difference

[lysxelapsed]

I never really used codeformer aside from testing, because it was just too slow. Espescially for longer videos. But 48 seconds for a single photo / frame? It was never that slow in my case, far from it. Can't test it right now as I'm currently running a batch process. But: Have you tried GPEN? Had to play around a little with the blend ratio... I end up mostly around 0.5 on a 720p target video and I'm satisfied with the results. FPS is roughly half of GFPGAN, which is (barely 😆) acceptable for me.

[Oil3]

I never really used codeformer aside from testing, because it was just too slow. Espescially for longer videos. But 48 seconds for a single photo / frame? It was never that slow in my case, far from it. Can't test it right now as I'm currently running a batch process. But: Have you tried GPEN? Had to play around a little with the blend ratio... I end up mostly around 0.5 on a 720p target video and I'm satisfied with the results. FPS is roughly half of GFPGAN, which is (barely 😆) acceptable for me.

with the smaller codeformer it's fine, I don't understand. I'd like to understand. Same for the squares, my squares are different, but they are not unique specific to codeformer: DMDnet also gave me, sometimes, squares. , but different. codeformer gave yellow squares. dmdnet gave "lower exposure" squares, like if the head was inside a shadow.

I think it's not related to the model itself, maybe ... maybe what?

[Oil3]

edit: A fix that worked for me is to add 1 line in Enhance_Codeformer.py

w00t: Enhance_Codeformer.py at the very beginning, insert at line 18: name = None

class Enhance_CodeFormer(): model_codeformer = None name = None devicename = None

edit: ps: "my" (not mine) version is actually the FP16 model.

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[github-actions[bot]]

This issue was closed because it has been stalled for 5 days with no activity.