How to train use IQA-PyTorch? or LIVEC?

[h3clikejava]

I want to retrain, but I see the training script uses the kadis700 dataset, which is too large. I want to train based on LIVEC or kadid10k, but I don't know how to write a training script. The variables in the script are fixed:

            inputs_A_orig = batch["img_A_orig"].to(device=device, non_blocking=True)
            inputs_A_ds = batch["img_A_ds"].to(device=device, non_blocking=True)
            inputs_A = torch.cat((inputs_A_orig, inputs_A_ds), dim=0)
            inputs_B_orig = batch["img_B_orig"].to(device=device, non_blocking=True)
            inputs_B_ds = batch["img_B_ds"].to(device=device, non_blocking=True)
            inputs_B = torch.cat((inputs_B_orig, inputs_B_ds), dim=0)
            img_A_name = batch["img_A_name"]
            img_B_name = batch["img_B_name"]

So, I want to use IQA-PyTorch for training, as it has already processed the LIVEC data, but the author didn’t provide a training script either. Do you know how to use LIVEC to train Arniqa?"

[LorenzoAgnolucci]

Hi @h3clikejava

Do you want to pre-train the encoder on LIVEC or just the regressor?

[h3clikejava]

Hello, thank you for your response. I'm a beginner in machine learning, and to be honest, I didn’t fully understand what you mentioned. My requirement is actually to fine-tune the arniqa-clive model in iqa-pytorch. My idea is to use the LIVEC dataset to run ARNIQA's training code and then replace LIVEC with my dataset to eventually generate the 'regressor_clive.pth' file for use with pyiqa. So, I assume what I want to train is what you referred to as the regressor. However, I don’t know how to get started, as the project currently uses the KADIS700Dataset for training. thx

[LorenzoAgnolucci]

Hi,

I suggest you start by reading our paper carefully to fully understand the difference between the pre-trained encoder and the regressor. Briefly, we pre-train the encoder using self-supervised learning on KADIS700 without using any labels. Then, we train a different regressor for each IQA dataset (such as LIVEC) using the MOS. Therefore, assuming that your dataset contains MOS annotations, you can simply start from our pre-trained encoder and just train the regressor. Is my assumption correct? If that's the case, I can help you with that.

[h3clikejava]

Yeah, you are right and thanks for your response. I’ve read through the paper, but since my English isn’t very strong, I didn’t fully understand its contents. I am communicating with you through ChatGPT.

From what I understand, I need to follow the instructions in the paper: "I need to freeze the encoder and train a linear regressor on top of it to obtain the quality score of an image from its representation." However, I’m not sure where to start. I also looked at #3 in the Issues, but I’m still uncertain about how to train a regressor with LIVEC. There are no LIVEC splits in the ./datasets directory. Could you advise me on how to begin? Thank you very much!

[LorenzoAgnolucci]

But what is your end goal? Do you just need a regressor trained on CLIVE? Because that is already available on IQA-PyTorch.

[h3clikejava]

I want to train regressor-myself.pth using my own dataset, but I don’t know how. So I’d like to first go through the training process of regressor-clive and then replace the CLIVE data with my own.

The more detailed reason is that my scoring system is similar to CLIVE, but I would prefer photos with full faces or group photos to have higher scores. Therefore, I need to fine-tune based on CLIVE, or directly replace CLIVE with my own dataset.