search

javiribera / locating-objects-without-bboxes

PyTorch code for "Locating objects without bounding boxes" - Loss function and trained models
Other
251 stars 51 forks source link

Question about Result of Metrics #38

Open hustcc19860606 opened 4 years ago

hustcc19860606 commented 4 years ago

I use the command: CUDA_VISIBLE_DEVICES=1 python -m object-locator.locate --imgsize 256X256 --dataset mall_dataset/frames/test/ --out mall_dataset/output/ --model checkpoints/mall\,lambdaa\=1\,BS\=32\,Adam\,LR1e-4.ckpt --evaluate, and the test set takes pictures from No.seq_001801.jpg to No.seq_002000.jpg. However, the results have large difference from your paper. Can you give me some suggestions? @javiribera bmm_stats fscore_vs_tau precision_vs_th recall_vs_tau

javiribera commented 4 years ago

The "results" you are trying to reproduce are these two plots? You seem to have posted the same plot repeated 3 times, by the way.

However, the take-away from the two plots is the same from the ones of the paper: (1) there are two clusters of pixels with tau values of 0 and 1, very close to the boundaries, (2) BMM does better than Otsu most of the time in terms of the F-score, and (3) optimizing for tau is not going to be significantly better than BMM or Otsu.

hustcc19860606 commented 4 years ago

@javiribera Vertical coordinates of three plots are different. The F-score in the first plot only can achieve at about 22.5. However，the figure 6 in your paper can achieve at over 80. And the precision in the following plot is also much lower than the value in your paper. However，the estimated_maps are similar to the paper. I didn't modify your code, all plots are generated automatically by the code. Can your give me some suggestions to reproduce the values in your paper ? precision_and_recall_vs_r,_tau=0 9444 seq_001801