search

yhlscut / C4

The code for AAAI 2020 paper "Cascading Convolutional Color Constancy"
48 stars 9 forks source link

Performance report may be problematic in the paper. #1

Open howardyclo opened 4 years ago

howardyclo commented 4 years ago

Hi, I would like to clarify the mismatch of performance on color checker (i.e., gehler) dataset I found in your paper. If I am wrong or misunderstand something, please correct me.

In your paper, I found that the reported mean angular errors of FC4 on the color checker dataset table are 2.40, while in the FC4 original paper, they report more lower mean angular errors, 1.65 (which is lower than your model 2.28). Also not to mention that there is also mismatch between your reported FFCC and the original one (since your paper does not clarify what type of FFCC you used, lets just focus on FC4).

I apologize if I misunderstand something, but the reported results seem to be problematic for me. Could you please explain such performance mismatch? Thanks.

yhlscut commented 4 years ago

Hi.

  1. The reported mean angular '2.40' is the result from table 2 whose exp setting is different from FC4 original paper, please read the paper for more details. The exp setting of table 1 (mean angular error on the color checker dataset table is 1.35) is consistent with the table 4 (mean angular error on the color checker dataset table is 1.65) of FC4 original paper.
  2. Reading the paper FFCC further, you could find that model N, O, P in tabel 1 of FFCC original paper use additional info. So we report the result of model M which achieves the best result in FFCC without using additional info. I apologize for not pointing it out further.