search

NVlabs / HANDAL

HANDAL Dataset and Pipeline
https://nvlabs.github.io/HANDAL/
Other
65 stars 2 forks source link

Applying for more evalution statistics #4

Closed SiyuanHuang95 closed 7 months ago

SiyuanHuang95 commented 9 months ago

Hi,

Thanks for the sharing of your great projects. @sbirchfield @swtyree

I am following this setting and using your dataset in my own research project and would like to take your output as a strong baseline. In the need to make a more holistic comparison, I would like to apply for more evaluation statistics, especially for the 2D Box AP at different IoU thresholds per class. If it is not so convenient for you to opensource this statistic, please send me the log to my email: anthonyhuang4de@gmail.com. Any help is welcome.

Bests,

SiyuanHuang95 commented 9 months ago

BTW, in the Table 2 of the paper, the box and segmentation mask is for the affordance parts or the whole object?

swtyree commented 9 months ago

Hi @SiyuanHuang95, I'm looking for the full evaluation stats. They weren't in the folder that I expected, so I might need to regenerate them from the saved models...

Regarding Table 2, the 2D metrics are for the whole object (specifically, the modal/visible portion of the whole object).

SiyuanHuang95 commented 9 months ago

Thanks for your reply.So is there the result about the affordance result? Either 2D box or segmentation?在 2023年11月30日，06:55，swtyree @.***> 写道： Hi @SiyuanHuang95, I'm looking for the full evaluation stats. They weren't in the folder that I expected, so I might need to regenerate them from the saved models... Regarding Table 2, the 2D metrics are for the whole object (specifically, the modal/visible portion of the whole object).

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>

SiyuanHuang95 commented 9 months ago

Hi @SiyuanHuang95, I'm looking for the full evaluation stats. They weren't in the folder that I expected, so I might need to regenerate them from the saved models...

Regarding Table 2, the 2D metrics are for the whole object (specifically, the modal/visible portion of the whole object).

@swtyree If you are going to re-generate the results, is it also possible to evaluate the performance on the affordance e.g. part-level detection performance?

SiyuanHuang95 commented 8 months ago

Any update? @swtyree

SiyuanHuang95 commented 7 months ago

Hi @SiyuanHuang95, I'm looking for the full evaluation stats. They weren't in the folder that I expected, so I might need to regenerate them from the saved models...

Regarding Table 2, the 2D metrics are for the whole object (specifically, the modal/visible portion of the whole object).

Hi @swtyree,

Would like to know would there be any update? Since I am planning to submit my paper in the near future?

andrewyguo commented 7 months ago

Hi @SiyuanHuang95 ,

Unfortunately, we couldn't find the full set of results from our evaluation. We would need to re-train these but don't have a timeline yet.

SiyuanHuang95 commented 7 months ago

h @swtyree,

thanks for your feedback and information.