search

ymgw55 / segment-anything-edge-detection

Unofficial edge detection implementation using the Automatic Mask Generation (AMG) of the Segment Anything Model (SAM).
53 stars 5 forks source link

Bad performance on ODS and AP #4

Open cyh9696 opened 1 year ago

cyh9696 commented 1 year ago

@ymgw55 Following your script, I set nms threshold as 0.7, and even higher as 0.8 or 0.85, but still got the evaluation result like "ODS: 0.746, OIS: 0.776 AP: 0.769 R50: 0.925". A large gap between it and the result in Table 3, especially at ODS and AP. Do you have any idea? Could you share your reproduced result in the setting as described in https://github.com/facebookresearch/segment-anything/issues/226#issuecomment-1511736967

ymgw55 commented 1 year ago

@cyh9696

Thanks for using my code.

Since the edge detection code used in the original paper has not been published, I don't know the specifics of the experiments performed (such as how edge-NMS was performed). However, I believe that this repository implements the procedures as described.

If you make any changes to the procedures or packages that improve performance, I would appreciate a PR.

cyh9696 commented 1 year ago

@cyh9696

Thanks for using my code.

Since the edge detection code used in the original paper has not been published, I don't know the specifics of the experiments performed (such as how edge-NMS was performed). However, I believe that this repository implements the procedures as described.

If you make any changes to the procedures or packages that improve performance, I would appreciate a PR.

Thanks for replying.

I believe that the AMG setting in this repository is almost perfect. Indeed, the details of edge NMS hasn't been released, but why edgeNMS function of Structed Forrest was implemented? Is it based on Canny Edge NMS?

BTW. Have you furthered the TODO part on faster evaluation by method such as "multiprocessing"?

ymgw55 commented 11 months ago

@cyh9696 Sorry for the late reply.

why edgeNMS function of Structed Forrest was implemented? Is it based on Canny Edge NMS?

We have implemented the Canny edge NMS ourselves, but have not been able to reproduce the edges as shown in the paper. As a result, we are currently using the edgeNMS function of Structured Forest.

We think this part needs further investigation and improvement, and would appreciate any recommendations for effective implementations or packages in Python.

BTW. Have you furthered the TODO part on faster evaluation by method such as "multiprocessing"?

Yes, we have achieved faster performance. It hasn't been updated yet, but we plan to incorporate these changes soon.

ymgw55 commented 11 months ago

As for the difference of edge NMS, I opened a new issue #5, so let's separate it from the discussion here.

ymgw55 commented 6 months ago

@cyh9696 Sorry for late update. I added parallel evaluation code.