Algolzw / daclip-uir

[ICLR 2024] Controlling Vision-Language Models for Universal Image Restoration. 5th place in the NTIRE 2024 Restore Any Image Model in the Wild Challenge.
https://algolzw.github.io/daclip-uir
MIT License
669 stars 34 forks source link

想请问作者一些问题 #86

Open LiJiajun000322 opened 4 days ago

LiJiajun000322 commented 4 days ago

作者您好,我想请问在daclip-sde中,能否配置让daclip不工作呢,就完全是一个sde去工作,这可以在配置文件中实现吗

LiJiajun000322 commented 4 days ago

general settings

name: ir-sde use_tb_logger: true model: denoising distortion: derain gpu_ids: [0]

sde: max_sigma: 50 T: 100 schedule: cosine # linear, cosine eps: 0.005

degradation: # for some synthetic dataset that only have GTs

for denoising

sigma: 25 noise_type: G # Gaussian noise: G

for super-resolution

scale: 4

datasets

datasets: train: name: Train_Dataset mode: LQGT dataroot_GT: /home/image-restoration-sde-main/codes/datasets/low-light/trainH/GT/ dataroot_LQ: /home/image-restoration-sde-main/codes/datasets/low-light/trainH/LQ/

use_shuffle: true
n_workers: 20  # per GPU
batch_size: 16
GT_size: 256
LR_size: 256
use_flip: true
use_rot: true
color: RGB

val: name: Val_Dataset mode: LQGT dataroot_GT: /home/image-restoration-sde-main/codes/datasets/low-light/testH/GT/ dataroot_LQ: /home/image-restoration-sde-main/codes/datasets/low-light/testH/LQ/

network structures

network_G: which_model_G: ConditionalUNet setting: in_nc: 3 out_nc: 3 nf: 64 depth: 4 ch_mult: [ 1, 2, 4, 8 ] context_dim: 512 use_degra_context: true use_image_context: true

path

path: pretrain_model_G: ~ strict_load: true resume_state: ~

training settings: learning rate scheme, loss

train: optimizer: AdamW # Adam, AdamW, Lion lr_G: !!float 2e-4 lr_scheme: MultiStepLR beta1: 0.9 beta2: 0.99 niter: 700000 warmup_iter: -1 # no warm up lr_steps: [2000, 4000, 6000] lr_gamma: 0.5 eta_min: !!float 1e-6

criterion

is_weighted: False loss_type: l1 weight: 1.0

manual_seed: 3471 val_freq: !!float 1e4

logger

logger: print_freq: 200 save_checkpoint_freq: !!float 1e4

我按如上配置文件去训练了一个lowlight的ir-sde,数据集就是LOL数据集。感觉效果不好,请作者指点一下配置参数

LiJiajun000322 commented 4 days ago

还有如果是低光任务的话,应该选IR-SDE还是Denoising SDE呢,我注意到IRSDE的项目里有两种SDE定义

LiJiajun000322 commented 4 days ago

Q3:daclip-sde文件做推理时,没有GT的情况应该怎么设置呢,我设置dataroot_GT:None会报错。 datasets: test1: name: Test mode: LQGT dataroot_GT: None dataroot_LQ: datasets/universal/deg_type/LQ

是否时要修改mode:LQ?我这样试过也不行

Algolzw commented 4 days ago

低光任务的话就用ir-sde就好。不用daclip的话这两个参数:use_degra_context: true, use_image_context: true要设置为false。在推理时,你尝试下同时修改mode: LQ 和去掉dataroot_GT这一行呢(不行的话可以dataroot_GT也设置为LQ路径试试)。

LiJiajun000322 commented 2 days ago

train: optimizer: AdamW # Adam, AdamW, Lion lr_G: !!float 2e-4 lr_scheme: MultiStepLR beta1: 0.9 beta2: 0.99 niter: 700000 warmup_iter: -1 # no warm up lr_steps: [2000, 4000, 6000] lr_gamma: 0.5 eta_min: !!float 1e-6

logger: print_freq: 200 save_checkpoint_freq: !!float 1e4

这部分的话迭代次数也需要这么多?我在LOL数据集上跑,发现打印的psnr几乎没变过,都是33

LiJiajun000322 commented 2 days ago

不用daclip的话这两个参数:use_degra_context: true, use_image_context: true要设置为false。

我这样做了以后发现无法进行低光恢复

LiJiajun000322 commented 2 days ago

use_degra_context: true use_image_context: false 后可以进行低光恢复 如果use_degra_context设置为false,图像仍然是很暗的

Algolzw commented 1 day ago

我还以为要重新训练。用pretrained weights的话确实是必须设置这两个参数的。