Infinite Loop using RestoreFormerPP

[jnc-nj]

First, confirm

• [X] I have read the instruction carefully
• [X] I have searched the existing issues
• [X] I have updated the extension to the latest version

What happened?

When I input a video with 26 frames, the following occurs the restore - using hashed face - swap process keeps looping ad infinitum. I'm not sure if this is a problem with the new version or with the restore model used (im certain the previous version did not have this problem), but in either case it's happening with irregularity. CPU and RAM gets filled up with each cycle to the point where I have to force restart my machine (this might be a separate issue).

Steps to reproduce the problem

workflow-26.json

As said previously, it's an irregular occurrence (roughly 5% of all runs using the node?).

Sysinfo

Linux 528659cf7778 6.2.0-39-generic #40-Ubuntu SMP PREEMPT_DYNAMIC Tue Nov 14 14:18:00 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

GPU: A100

Relevant console log

[ReActor] 12:01:14 - STATUS - Restoring with RestoreFormer_PP.onnx
[ReActor] 12:01:15 - STATUS - Restoring 1
...
[ReActor] 12:01:15 - STATUS - Restoring 26
[ReActor] 12:01:40 - STATUS - Working: source face index [0], target face index [0]
[ReActor] 12:01:40 - STATUS - Using Hashed Source Face(s) Model...
[ReActor] 12:01:40 - STATUS - (Image 0) Using Hashed Target Face(s) Model...
...
[ReActor] 12:02:09 - STATUS - (Image 25) Using Hashed Target Face(s) Model...
[ReActor] 12:02:09 - STATUS - Swapping 0...
...
[ReActor] 12:02:11 - STATUS - Swapping 25...
[ReActor] 12:02:11 - STATUS - --Done!--
[ReActor] 12:02:12 - STATUS - Restoring with RestoreFormer_PP.onnx
[ReActor] 12:02:13 - STATUS - Restoring 1
...
[ReActor] 12:02:39 - STATUS - Restoring 26
......

Additional information

No response

[Gourieff]

And if you use another model but not RestoreFormer_PP - is the processing ok? No loop?

[jnc-nj]

And if you use another model but not RestoreFormer_PP - is the processing ok? No loop?

Yes, I haven't seen any issues with GFPGAN. Once again, I can't be 100% sure, because restoreformerPP doesn't trigger the loop every time -- the vast majority of the time it's fine.

Edit:

I see in the issues section a bunch of people asking about the lack of onnxruntime-gpu support, I can confirm that the current codebase is compatible with CUDA 11.8 + onnxruntime-gpu==1.17.1 (onnxruntime must be uninstalled first), and uses GPU as expected for both face swap and face restore nodes. Perhaps this should be headlined in an FAQ section? The issues section of the repo is getting cluttered.

Edit 2:

I can also (kinda) confirm that the loop no longer exists with onnxruntime-gpu==1.17.1 and onnxruntime==1.17.1. I assume this must be a bug internal to the onnxruntime packages.

Edit 3:

With my A100 card, swap and restore speed on GPU is ~2s/img, analysis speed is ~20s/img.

Edit 4:

The reason I didn't see any issues with GFPGAN was probably because it was a .pth model and not a .onnx model.

Edit 5:

Can confirm current settings does not adversely affect other nodes which require specific execution providers for GPU (eg dwpose).