[Example] Evaluation and enhancement for Super Resolution

[ibelem]

@GreyZzzzzzXh Could you please add more description and your recent findings or investigations for this issue? :)

[akineeic]

I have tried ESRGAN(enhanced SRGAN) model recently and there is still some image processing work to do. But I found that the inference time of ESRGAN(more than 4000ms) is much more than the existing model(about 600ms). Since this model doesn't seem to perform well, should I abandon the current work and look for other model directly?

[ibelem]

How about the accuracy of this model? How much ramaining work (like image processing) need to do and what's your final inference results can be expected? I am OK to look for other models but there should be a judgement table for comparing SRGAN, ESRGAN and new models from multiple perspectives.

[akineeic]

1. The most commonly used comparison indicator in SR task is PSNR, which can be found in the paper. So I think PSNR and inference time should be compares between different models and a better model should achieve better results in similar or shorter inference time.
2. As to the ESRGAN model, it's output image is obviously darker than the real image, it should be caused by missing of some model-related image preprocessing operations.

[akineeic]

The result is much more smooth than the original visually. But it can only run in the WebGL backend now since it need the support of PReLU op.

• ESRGAN

• SRGAN

[ibelem]

Could you please try some other photos? I remembered that the quality is poor when using some photos for SRGAN. And you are right, the inference time is a big problem but the PRelu is proposed in https://github.com/intel/webml-polyfill/issues/1002. What's the final inference time when leverage PRelu ops or it's already included but still quite slow?

Here is the code for enabling PRelu in WebGL: https://github.com/intel/webml-polyfill/pull/982/files

[akineeic]

I have enable PRelu op in my code, and there is only 2 PRelu ops in the whole model, so PRelu op shouldn't be the cause of this problem. I think it's caused by the leakyrelu which is converted into Mul and Maximum in Tflite format. I'll try to use the test tool to figure out it.

[ibelem]

Artifacts Reduction, paper

Whereby (appear.in) in their Videoconferencing Stack is experimenting with TensorFlow implementation of AR-CNN (https://github.com/ppooiiuuyh/ARCNN-FAST/tree/branch1) on mobile device. Training set: generic images, Validation set: videoconferencing images Artifacts: genereated with VP8 encoder Optimization Metric: PSNR Model size ~ 400 KB In product they used Fast ARCNN ( 5 Conv layers instead of 4, but with less filters) , used QVGA and increased the Stride.

Maybe even in future, use GAN - ARGAN

@NALLEIN @akineeic