search

nqanh / affordance-net

AffordanceNet - Multiclass Instance Segmentation Framework - ICRA 2018
Other
121 stars 54 forks source link

way to calculate F score #31

Open fujenchu opened 6 years ago

fujenchu commented 6 years ago

Hi Dr. Nguyen,

It is a really nice work! Thank you. Would you mind to provide script to compute the F score in the paper? I would like to evaluate the performance for my own dataset set. Or it is in the repo and I overlooked it? It would be great if I could borrow the code from you.

Thank you very much.

nqanh commented 6 years ago

Hi, please see https://github.com/nqanh/affordance-net/issues/19 and http://cgm.technion.ac.il/Computer-Graphics-Multimedia/Software/FGEval/

fujenchu commented 6 years ago

Cool! Thanks for your reply. I just got it working.

fujenchu commented 6 years ago

Hi Dr. Nguyen,

I just realized that previous works evaluated their approaches by first cropping images with ground truth mask for bounding boxes. I am wondering if you follow the same procedure (cropping the region of interest) or you pass the whole images for F score to get the results described in the paper? Thank you very much.

nqanh commented 6 years ago

We pass the whole image and evaluate F score. Please see paper for more details.

fujenchu commented 6 years ago

I see. Thank you very much! I've read the paper and we are wondering how many epochs you have trained on UMD dataset for getting the results? It seems the hyperparameters in the repo are for IIT. Would you mind to provide the hyperparameters for UMD if possible? It seems stepsize of 150k in solver.prototxt is large for UMD given the dataset size. Thank you in advance!

nqanh commented 6 years ago

You can just train then stop when the lost doesn't decrease anymore :)