search

ZhihaoPENG-CityU / TCSVT22---DAGC

Accepted by IEEE TCSVT
11 stars 2 forks source link

Need Assistance with Reproducing Results in Paper #2

Open zr-swu opened 1 year ago

zr-swu commented 1 year ago

Dear ZhihaoPENG,

I have been unable to reproduce the results mentioned in your paper accurately. On each dataset I have experimented with, my results consistently fall short by approximately 0.5 points. Is there any difference between the open source code and the code you used to get the results of your paper?

Thank you very much for your time and consideration. Looking forward to your response.

ZhihaoPENG-CityU commented 1 year ago

Thank you for your interest in our work on DAGC. Regarding your question, I can confirm the open-source code is identical to what was used to generate the results in our paper. Based on the discrepancy you are facing, I have a few suggestions that may help: 1- double-check the data preprocessing, hyperparameter values, random seeds, and other factors against what is outlined in the paper to ensure they match. 2- Use the same experimental environment for evaluation as described in our work, e.g., Python 3.6.12, PyTorch 1.9.0+cu102, and GPUs including GeForce RTX 2080 Ti, RTX 3090, and Quadro RTX 8000. 3- Repeat each experiment 10 times and calculate the mean values and standard deviations.

Best regards, Zhihao PENG

zr-swu commented 1 year ago

Thanks for your reply. According to your suggestion, I conducted the experiment again using the same environment and equipment, but the results in the paper still could not be obtained. May I ask whether you fixed the random seeds (which were not mentioned in the paper and public code) during the experiment ?

Original

From:"Zhihao PENG"< @.*** >;

Date:2023/9/8 14:54

To:"ZhihaoPENG-CityU/TCSVT22---DAGC"< @.*** >;

CC:"zr-swu"< @. >;"Author"< @. >;

Subject:Re: [ZhihaoPENG-CityU/TCSVT22---DAGC] Need Assistance withReproducing Results in Paper (Issue #2)

Thank you for your interest in our work on DAGC. Regarding your question, I can confirm the open-source code is identical to what was used to generate the results in our paper. Based on the discrepancy you are facing, I have a few suggestions that may help: 1- double-check the data preprocessing, hyperparameter values, random seeds, and other factors against what is outlined in the paper to ensure they match. 2- Use the same experimental environment for evaluation as described in our work, e.g., Python 3.6.12, PyTorch 1.9.0+cu102, and GPUs including GeForce RTX 2080 Ti, RTX 3090, and Quadro RTX 8000. 3- Repeat each experiment 10 times and calculate the mean values and standard deviations.

Best regards, Zhihao PENG

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>