Results not reproducible

[balajiselvaraj1601]

Hi @pkuCactus ,

We tried to train model with help of vgg model shared in the https://drive.google.com/file/d/1CmDMypSlLM6EAvOt5yjwUQ7O5w-xCm1n/view?usp=sharing.

We are able to get ODS=0.798 OIS=0.813 at 40k iteration

Can you mention the reason for the large deviation in the results? As per your paper, it must be ODS =0.806

We have used BSDS data alone for training the model.

Can you mention some tips or ideas to fix the issue. We have used your code as such with minor modifications to support new version of torch.

[Jinqi-Cheng]

Hi,

I encountered the same problem. I trained the model by using the dataset from http://mftp.mmcheng.net/liuyun/rcf/data/HED-BSDS.tar.gz, and I used the pretrained VGG model as well, but I only got ODS=0.59 at 40k iteration, which is not even close to your result. I wonder why this happened.

Could you tell me how you did the BSR benchmarks? Because mine was acting a bit weird. When I tried to run the benchmarks, the PR curve didn't show up in the end. Instead, a single dot indicating a single value pair of Precision and Recall is shown. Also, is there a restriction on the input format when running the benchmarks? My results were stored as a mat file with [0-255], does it have to be a mat file with [0-1]?

I would really appreciate it if you can help me on this.

[yun-liu]

@balajiselvaraj1601 @Jinqi-Cheng @pkuCactus Have you solved this problem? Using the released model for evaluation, we can only get the ODS F-measure of 0.815. When training using the mix of BSDS500 and PASCAL Context datasets with default settings, we can only get the ODS F-measure of 0.803. Multi-scale testing is enabled in my experiments.

[ForawardStar]

@yun-liu @balajiselvaraj1601 @Jinqi-Cheng Do you solve this problem. Recently, I meet the same problem, I only get the ODS-F score of 0.798, which is far from the results reported in the paper.

[CookiC]

@ForawardStar 你最后复现成功了吗？