How to test to reproduce the numbers in the paper?

[ckkelvinchan]

We use the MATLAB functions (e.g. BRISQUE).

To save computational cost, we compute the metrics on the first, middle, and last frames.

[Ha0Tang]

@ckkelvinchan Thanks, which videos (i.e., test set) are used to reproduce the numbers?

[ckkelvinchan]

All 50 videos in our VideoLQ dataset are used.

[Ha0Tang]

@ckkelvinchan Did you compute the metrics on the "00000000.png", "00000049.png", "00000099.png"?

[ckkelvinchan]

I just checked. It should be 00000000.png, 00000050.png, 00000099.png

[mehranjeelani]

@ckkelvinchan Could you tell me the frame numbers of the vid4 dataset used to calculate the metrics as the total frames in those videos is less than 100?

[ckkelvinchan]

Sorry for not presenting the details clearly. For the Vid4 clips:

030: 00000000.png, 00000020.png, 00000040.png 031: 00000000.png, 00000017.png, 00000033.png 032: 00000000.png, 00000024.png, 00000048.png 033: 00000000.png, 00000023.png, 00000046.png

[mehranjeelani]

@ckkelvinchan Thanks for the fast response. Just one more thing, I could not find the matlab function for PI metric. Could you direct me towards some resource to calculate the same

[ckkelvinchan]

@ckkelvinchan Thanks for the fast response. Just one more thing, I could not find the matlab function for PI metric. Could you direct me towards some resource to calculate the same

If I remember correctly, it should be here.

[mehranjeelani]

Thanks it worked.... Do you by any chance have x2 model?

[ckkelvinchan]

Thanks it worked.... Do you by any chance have x2 model?

Recently I don't have enough resources for the x2 model. We are working on that and will upload it when it is ready. Stay tuned!

[mehranjeelani]

When can I expect it? Any estimated date in mind?

[ckkelvinchan]

When can I expect it? Any estimated date in mind?

I cannot be sure :( Maybe I can spend some time later to update the model so that it supports 2x.

[CrissyHoo]

why you only choose 3 frames for evaluation? why not testing on the whole video frames?

[ckkelvinchan]

why you only choose 3 frames for evaluation? why not testing on the whole video frames?

Since computing NRQM is very time-consuming. We cannot afford calculating almost 5000 frames for 9 methods (= about 45000 frames).

[CrissyHoo]

Thanks for the response！

[CrissyHoo]

@ckkelvinchan Thanks for the fast response. Just one more thing, I could not find the matlab function for PI metric. Could you direct me towards some resource to calculate the same

If I remember correctly, it should be here.

When calculating PI, the GT images are needed, but there is not gt in VideoLQ. Could you tell me what did you use as the gt when calculating PI?

[CrissyHoo]

I found an interesting thing. I tested the bicubic images with brisque and got the same result as in the paper. But when I tested the same bicubic data with niqe, I got 7.9805, which is higher than 5.2762 in the paper. Maybe there is some problem with my niqe code. [http://live.ece.utexas.edu/research/quality/niqe_release.zip](niqe code) I just find it in the directed niqe paper. Could you give me some information about how you calculate niqe?

[ckkelvinchan]

@ckkelvinchan Thanks for the fast response. Just one more thing, I could not find the matlab function for PI metric. Could you direct me towards some resource to calculate the same

If I remember correctly, it should be here.

When calculating PI, the GT images are needed, but there is not gt in VideoLQ. Could you tell me what did you use as the gt when calculating PI?

PI is a weighted sum of NIQE and NRQM. I think no GT is needed.

[ckkelvinchan]

I found an interesting thing. I tested the bicubic images with brisque and got the same result as in the paper. But when I tested the same bicubic data with niqe, I got 7.9805, which is higher than 5.2762 in the paper. Maybe there is some problem with my niqe code. [http://live.ece.utexas.edu/research/quality/niqe_release.zip](niqe code) I just find it in the directed niqe paper. Could you give me some information about how you calculate niqe?

Yes you are correct. In the original paper I computed the NIQE using the MATLAB built-in function, which is different from the one you are using.

I have updated the results using the official code on the Y-channel (which is the normal practice). I will release the updated version soon.

[CrissyHoo]

but the code you provided need the GT, which should be applied in "self_validation_HR" folder.

I found an interesting thing. I tested the bicubic images with brisque and got the same result as in the paper. But when I tested the same bicubic data with niqe, I got 7.9805, which is higher than 5.2762 in the paper. Maybe there is some problem with my niqe code. [http://live.ece.utexas.edu/research/quality/niqe_release.zip](niqe code) I just find it in the directed niqe paper. Could you give me some information about how you calculate niqe?

Yes you are correct. In the original paper I computed the NIQE using the MATLAB built-in function, which is different from the one you are using.

I have updated the results using the official code on the Y-channel (which is the normal practice). I will release the updated version soon.

Thanks!!

[ckkelvinchan]

It is used to calculate RMSE. For NIQE, NRQM, and PI, we don't need GT.

[CrissyHoo]

hello~Could you give me some info about which function you used for video compression?

[ckkelvinchan]

hello~Could you give me some info about which function you used for video compression?

Please refer to here for the details.