ZhexinLiang / CLIP-LIT

[ICCV 2023, Oral] Iterative Prompt Learning for Unsupervised Backlit Image Enhancement
https://zhexinliang.github.io/CLIP_LIT_page/
269 stars 23 forks source link

prompt fine-tuning阶段中,关于pr_semi_path的指向数据集的困惑? #19

Open QQ2263836119 opened 7 months ago

QQ2263836119 commented 7 months ago
  1. clip_score.py里面的class four_margin_loss(nn.Module):类(如下面代码),结合论文公式,可以得出 $inp=Ib$ , $smi1=I{t-1}$ , $smi=I_{t}$ , $ref=I_w$ 。
        elif num==4:
            loss_inp_semi1=self.margin_loss_L(tensor0,tensor_mid[0],labels)
            loss_semi1_semi2=self.margin_loss_S(tensor_mid[0],tensor_mid[1],labels)
            loss_semi2_ref=self.margin_loss_S(tensor_mid[1],tensor3,labels)
            return loss_inp_ref+loss_inp_semi1+loss_semi1_semi2+loss_semi2_ref
  2. 但是train.py里面,有如下代码可得,semi_path[1-pr_semipath]指的是 $I{t}$ ,semi_path[pr_semipath]指的是 $I{t-1}$。即
    
                #prompt fine-tuning
                if cur_iteration+1==train_thre+1:
                    if total_iteration+1>config.num_clip_pretrained_iters:
                        pr_semi_path=1-pr_semi_path# 此时pr_semi_path指向下一轮的存储地址,或者说前一轮的地址 I_{t-1}
                    if semi_path[0]!=' ' and  semi_path[1]!=' ':
                        print(semi_path)
                        L_margin_loss = clip_score.four_margin_loss(0.9,0.1)
    # semi_path[1-pr_semi_path]指的是I_{t} ,semi_path[pr_semi_path]指的是I_{t-1} 
                        prompt_train_dataset = dataloader_prompt_margin.lowlight_loader(config.lowlight_images_path,config.normallight_images_path,semi_path[1-pr_semi_path],semi_path[pr_semi_path])#,config.overlight_images_path)        
3. 结合下面代码,可知:semi1_path= $I_{t}$ ,semi2_path= $I_{t-1}$。 但是这就与 ‘1’中:“  $inp=I_b$ , $smi1=I_{t-1}$ , $smi=I_{t}$ , $ref=I_w$  ”矛盾。麻烦可以解释下吗?或者是我哪里理解错误了。

class lowlight_loader(data.Dataset): def init(self, lowlight_images_path,normallight_images_path,semi1_path=None,semi2_path=None):