search

nqanh / affordance-net

AffordanceNet - Multiclass Instance Segmentation Framework - ICRA 2018
Other
121 stars 54 forks source link

how to measure the performance. #18

Open ambl2357 opened 6 years ago

ambl2357 commented 6 years ago

I do not know how to measure the performance. Could you tell me a file or a method?

nqanh commented 6 years ago

You just need to merge all the outputted affordance maps into one map (please see the paper for details). Then use the Weighted F-measure code from here to evaluate the results with the groundtruths.

ambl2357 commented 6 years ago

Clearly resolved. thank you and have a good day

-----Original Message----- From: "Anh Nguyen"notifications@github.com To: "nqanh/affordance-net"affordance-net@noreply.github.com; Cc: "ambl2357"ambl2357@naver.com; "Author"author@noreply.github.com; Sent: 2018-02-22 (목) 17:42:31 Subject: Re: [nqanh/affordance-net] how to measure the performance. (#18)

You just need to merge all the outputted affordance maps into one map (please see the paper for details). Then use the Weighted F-measure code from here to evaluate the results with the groundtruths. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

ClaireTun commented 6 years ago

HI, what does mean by"merge all the outputted affordance maps into one map"? How to calculate one kind of affordance like 'grasp'? There are more than one affordances on one pic? How to calculate F measure separately?

nqanh commented 6 years ago

The network outputs each affordance map for each detected object (as in the demo). Since the image may has many objects --> we need to merge them into "one predicted" image. Then the F measure will calculate the accuracy for each affordance (on this "merged" output).

ClaireTun commented 6 years ago

HI， Nqanh~ about the Fmeasure for each affordance, I still have questions. In the Fmeasure code , the type of GT is logical, which means the background and the foreground. But in foreground, we have different objects and affordances. Do I have to seperate all the affordances and make new GTs?

nqanh commented 6 years ago

We can save all the affordances in 1 map. Then, for each affordance, we select its id and the background --> it becomes logical when compares with the GT.

ClaireTun commented 6 years ago

Well,ok! Thank you for your answer and patience! But I am still a little confused. Would you plz share this part code,like how to save in 1 map and select id?

nqanh commented 6 years ago

Please see https://github.com/nqanh/affordance-net/issues/19 (also please use the search function, most of the problems we have already be answered. Thanks!)

ClaireTun commented 6 years ago

Wow~ Thank you!!

chaundm commented 5 years ago

The network outputs each affordance map for each detected object (as in the demo). Since the image may has many objects --> we need to merge them into "one predicted" image. Then the F measure will calculate the accuracy for each affordance (on this "merged" output).

Hi nqanh, Please help me by teaching me how to "merge affordance map" into one map. Could you tell me more details about that? For example, we have 3 file .pnp (contains affordance maps), and 3 file .sm, so which extension file (sm or pnp) we will use to merge and how to merge it, please?

ambl2357 commented 5 years ago

im sorry, I studied so long ago and now I'm doing something different so I can't answer the question. One thing I could answer was that there was no phenomenon corresponding to the question. -----Original Message----- From: "chaundm"notifications@github.com To: "nqanh/affordance-net"affordance-net@noreply.github.com; Cc: "jay-sik"ambl2357@naver.com; "Author"author@noreply.github.com; Sent: 2019-08-21 (수) 22:18:36 (GMT+09:00) Subject: Re: [nqanh/affordance-net] how to measure the performance. (#18)

The network outputs each affordance map for each detected object (as in the demo). Since the image may has many objects --> we need to merge them into "one predicted" image. Then the F measure will calculate the accuracy for each affordance (on this "merged" output).

Hi nqanh, Please help me by teaching me how to "merge affordance map" into one map. Could you tell me more details about that? For example, we have 3 file .pnp (contains affordance maps), and 3 file .sm, so which extension file (sm or pnp) we will use to merge and how to merge it, please? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

chaundm commented 5 years ago

Wow~ Thank you!!

Hi ClaireTun, I have the same trouble as you. Could you tell me how to "merge all of affordances into a map"? And which extension file is used for F_measure code (sm or png). I found that that the ground truth is sm file with black and white, while the prediction in demo_img.py code outputs the png file with each predicted masks with affordance colour. How to combine these files to run the F_measure code in matlab, please?

ClaireTun commented 5 years ago

Well, I finished the project half a year ago. Some details,  I have already forgot. Next week,I will return to school .I can review the code and try to answer your questions 

------------------ Original ------------------ From: chaundm <notifications@github.com> Date: Fri,Aug 30,2019 2:54 PM To: nqanh/affordance-net <affordance-net@noreply.github.com> Cc: ClaireTun <1719473569@qq.com>, Comment <comment@noreply.github.com> Subject: Re: [nqanh/affordance-net] how to measure the performance. (#18)

Wow~ Thank you!!

Hi ClaireTun, I have the same trouble as you. Could you tell me how to "merge all of affordances into a map"? And which extension file is used for F_measure code (sm or png). I found that that the ground truth is sm file with black and white, while the prediction in demo_img.py code outputs the png file with each predicted masks with affordance colour. How to combine these files to run the F_measure code in matlab, please?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

Rechardgu commented 4 years ago

Wow~ Thank you!!

Hi ClaireTun, I have the same trouble as you. Could you tell me how to "merge all of affordances into a map"? And which extension file is used for F_measure code (sm or png). I found that that the ground truth is sm file with black and white, while the prediction in demo_img.py code outputs the png file with each predicted masks with affordance colour. How to combine these files to run the F_measure code in matlab, please? Hi chaundm, did you have solved the problem?

Rechardgu commented 4 years ago

Well, I finished the project half a year ago. Some details,  I have already forgot. Next week,I will return to school .I can review the code and try to answer your questions  … ------------------ Original ------------------ From: chaundm <notifications@github.com> Date: Fri,Aug 30,2019 2:54 PM To: nqanh/affordance-net <affordance-net@noreply.github.com> Cc: ClaireTun <1719473569@qq.com>, Comment <comment@noreply.github.com> Subject: Re: [nqanh/affordance-net] how to measure the performance. (#18) Wow~ Thank you!! Hi ClaireTun, I have the same trouble as you. Could you tell me how to "merge all of affordances into a map"? And which extension file is used for F_measure code (sm or png). I found that that the ground truth is sm file with black and white, while the prediction in demo_img.py code outputs the png file with each predicted masks with affordance colour. How to combine these files to run the F_measure code in matlab, please? — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

Hi ClaireTun, my ground truth and predicted map are png files with each masks with affordance color. How to use the weighted F_measure?