search

rlaphoenix / VSGAN

PyTorch-based Super-Resolution and Restoration Image Processing Module for VapourSynth
https://vsgan.phoeniix.dev
MIT License
190 stars 13 forks source link

VSDPIR RGBH -> VSGAN RGBS issue,... #31

Closed Selur closed 1 year ago

Selur commented 1 year ago

I got a strange problem when using vsdpir and vsgan inside the same script:

using both with RGBH:

from vsdpir import dpir as DPIR
# adjusting color space from YUV420P8 to RGBH for vsDPIRDenoise
clip = core.resize.Bicubic(clip=clip, format=vs.RGBH, matrix_in_s="470bg", range_s="limited")
# denoising using DPIRDenoise
clip = DPIR(clip=clip, strength=5.000, task="denoise", device_index=0, trt=True, trt_cache_path="J:/tmp")
# changing range from limited to full range
clip = core.resize.Bicubic(clip, range_in_s="limited", range_s="full")
# Setting color range to PC (full) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=0)
# resizing using VSGAN
from vsgan import ESRGAN
vsgan = ESRGAN(clip=clip,device="cuda")
model = "C:/Users/Selur/Desktop/testing/1x_Dotzilla_Compact_80k_net_g.pth"
vsgan.load(model)
vsgan.apply() # 640x352
clip = vsgan.clip

works.

using both with RGBS:

from vsdpir import dpir as DPIR
# adjusting color space from YUV420P8 to RGBS for vsDPIRDenoise
clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, matrix_in_s="470bg", range_s="limited")
# denoising using DPIRDenoise
clip = DPIR(clip=clip, strength=5.000, task="denoise", device_index=0, trt=True, trt_cache_path="J:/tmp")
# changing range from limited to full range
clip = core.resize.Bicubic(clip, range_in_s="limited", range_s="full")
# Setting color range to PC (full) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=0)
# resizing using VSGAN
from vsgan import ESRGAN
vsgan = ESRGAN(clip=clip,device="cuda")
model = "C:/Users/Selur/Desktop/testing/1x_Dotzilla_Compact_80k_net_g.pth"
vsgan.load(model)
vsgan.apply() # 640x352
clip = vsgan.clip

works.

Using VSDIR with RGBS and VSGAN with RGBH:

from vsdpir import dpir as DPIR
# adjusting color space from YUV420P8 to RGBS for vsDPIRDenoise
clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, matrix_in_s="470bg", range_s="limited")
# denoising using DPIRDenoise
clip = DPIR(clip=clip, strength=5.000, task="denoise", device_index=0, trt=True, trt_cache_path="J:/tmp")
# changing range from limited to full range
clip = core.resize.Bicubic(clip, range_in_s="limited", range_s="full")
# Setting color range to PC (full) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=0)
# adjusting color space from RGBS to RGBH for vsVSGAN
clip = core.resize.Bicubic(clip=clip, format=vs.RGBH, range_s="full", dither_type="error_diffusion")
# resizing using VSGAN
from vsgan import ESRGAN
vsgan = ESRGAN(clip=clip,device="cuda")
model = "C:/Users/Selur/Desktop/testing/1x_Dotzilla_Compact_80k_net_g.pth"
vsgan.load(model)
vsgan.apply() # 640x352
clip = vsgan.clip

works.

Using VSDIR with RGBH and VSGAN with RGBS:

from vsdpir import dpir as DPIR
# adjusting color space from YUV420P8 to RGBH for vsDPIRDenoise
clip = core.resize.Bicubic(clip=clip, format=vs.RGBH, matrix_in_s="470bg", range_s="limited")
# denoising using DPIRDenoise
clip = DPIR(clip=clip, strength=5.000, task="denoise", device_index=0, trt=True, trt_cache_path="J:/tmp")
# changing range from limited to full range
clip = core.resize.Bicubic(clip, range_in_s="limited", range_s="full")
# Setting color range to PC (full) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=0)
# adjusting color space from RGBH to RGBS for vsVSGAN
clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, range_s="full")
# resizing using VSGAN
from vsgan import ESRGAN
vsgan = ESRGAN(clip=clip,device="cuda")
model = "C:/Users/Selur/Desktop/testing/1x_Dotzilla_Compact_80k_net_g.pth"
vsgan.load(model)
vsgan.apply() # 640x352
clip = vsgan.clip

fails with:

Input type (float) and bias type (struct c10::Half) should be the same

Using VSDIR with RGBS and VSGAN with RGB48:

from vsdpir import dpir as DPIR
# adjusting color space from YUV420P8 to RGBS for vsDPIRDenoise
clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, matrix_in_s="470bg", range_s="limited")
# denoising using DPIRDenoise
clip = DPIR(clip=clip, strength=5.000, task="denoise", device_index=0, trt=True, trt_cache_path="J:/tmp")
# changing range from limited to full range
clip = core.resize.Bicubic(clip, range_in_s="limited", range_s="full")
# Setting color range to PC (full) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=0)
# adjusting color space from RGBS to RGB48 for vsVSGAN
clip = core.resize.Bicubic(clip=clip, format=vs.RGB48, range_s="full", dither_type="error_diffusion")
# resizing using VSGAN
from vsgan import ESRGAN
vsgan = ESRGAN(clip=clip,device="cuda")
model = "C:/Users/Selur/Desktop/testing/1x_Dotzilla_Compact_80k_net_g.pth"
vsgan.load(model)
vsgan.apply() # 640x352
clip = vsgan.clip

produces graphic glitches here https://ibb.co/jTCbghX

HolyWu commented 1 year ago

Using VSDIR with RGBH and VSGAN with RGBS:

from vsdpir import dpir as DPIR
# adjusting color space from YUV420P8 to RGBH for vsDPIRDenoise
clip = core.resize.Bicubic(clip=clip, format=vs.RGBH, matrix_in_s="470bg", range_s="limited")
# denoising using DPIRDenoise
clip = DPIR(clip=clip, strength=5.000, task="denoise", device_index=0, trt=True, trt_cache_path="J:/tmp")
# changing range from limited to full range
clip = core.resize.Bicubic(clip, range_in_s="limited", range_s="full")
# Setting color range to PC (full) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=0)
# adjusting color space from RGBH to RGBS for vsVSGAN
clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, range_s="full")
# resizing using VSGAN
from vsgan import ESRGAN
vsgan = ESRGAN(clip=clip,device="cuda")
model = "C:/Users/Selur/Desktop/testing/1x_Dotzilla_Compact_80k_net_g.pth"
vsgan.load(model)
vsgan.apply() # 640x352
clip = vsgan.clip

fails with:

Input type (float) and bias type (struct c10::Half) should be the same

This is not a fault of VSGAN. The default floating point tensor type is changed to float16 in vsdpir when the input is of RGBH format. Add the following lines below dpir to change the type back to float32.

import torch
torch.set_default_tensor_type(torch.FloatTensor)
rlaphoenix commented 1 year ago

This is not a fault of VSGAN. The default floating point tensor type is changed to float16 in vsdpir when the input is of RGBH format. Add the following lines below dpir to change the type back to float32.

import torch
torch.set_default_tensor_type(torch.FloatTensor)

Is there a reason your VS-DPIR sets it globally like that when using it? Since VS-DPIR is your project, do you have any recommendations on changes I could make to have full interoperability with your VS-DPIR project?

Selur commented 1 year ago

Okay, so adding

import torch
torch.set_default_tensor_type(torch.FloatTensor)

whenever I use dpir with 16bit is the way to go for now. Thanks!

Selur commented 1 year ago

I close this since it was fixed through https://github.com/HolyWu/vs-dpir/releases/tag/v3.0.1