Issues with the Real ESRGAN model

[nklint]

Hi! Really great that you provide these CoreML models. Before I found this project I tried the Portable executable file version of the https://github.com/xinntao/Real-ESRGAN

I used the CoreML version of Real ESRGAN and it kind of worked. but I have some concerns. The output file seems to be locked to a fixed size and aspect-ratio 1:1 It would be really nice if the output size would be derived from the input but 4x. Also it seems to not handle transparency in input images. The transparent parts are instead rendered as black.

Are these issued something that could be resolved with different input to the CoreMLTools?

[john-rocky]

Hi.

I tried make RealESRGAN CoreML model have flexible input and output shape. It was possible, but memory usage reach to sky rockets even 512 * 512 input. I share the flexible shape model that can accept unlimited input and output.

https://drive.google.com/file/d/1SyVkw3mmNLWF1xxRrtXXeRnC2qRtGLzF/view?usp=sharing

Warning: It use 1.7GB memory when I tried with 512 * 512 input.

Or by some chance, when we set some enumeration value of input like [(320,320),(512,512),(1024,1024)], performance may be better.

MLBoy

[faresch31]

hi thank you for the models. today I tried the RealESRGAN model but the memory reach 1.6 gb and I also tried the flexible shape model but same problem I tried 320 320 and 400 400 and 512 512 but same problem it kind work with 200 200 but I reach 500 mb memory . please if you have any fix tell me my hole project is based on this feature and I don't really know python to I can't convert other models to coreml

[messageloop]

“Error Domain=com.apple.CoreML Code=0 "Error in declaring network." UserInfo={NSLocalizedDescription=Error in declaring network.”

hello, This model fails to initialize when used in my project. Does this happen to you? Xcode 12.5.1 Mac M1 chip

[faresch31]

yes Xcode 13.2.1 Mac M1 chip same error memory go to 1.7gb and crashes

[messageloop]

On my side it's not a memory issue, but a problem with initializing this model, is it something that needs to be on an Intel CPU to work?

[faresch31]

my problem is there is no release of memory

[messageloop]

What is your development environment?

[faresch31]

swiftui

[pboudoin]

same problem here. Too high memory consumption, and the memory is not release after processing

[john-rocky]

Some CoreML models use large memory. Try Vision Request with completion handler. It save the memory than running without it.

[pboudoin]

That's what I use. It seems that it's only in debug build, not in release. May be an issue with CoreML?

[john-rocky]

If the model input shape is fixed (320*320), the memory usage is 110MB in my project. I think the flexible input shape make the memory problem.

[hasinur010]

How did you fix the model input shape?

[latishab]

Has anyone solve the the memory with flexible input shape?