search

wenxi-yue / SurgicalSAM

[AAAI2024] Official implementation of SurgicalSAM
MIT License
70 stars 9 forks source link

Results json not published #10

Closed Kunaldargan closed 7 months ago

Kunaldargan commented 7 months ago

Dear Authors,

Thank you for providing the code. Can you also publish the result JSON files generated from the best checkpoint model? This is essential to recreate the experiments and validate the results.

Regards

wenxi-yue commented 7 months ago

Hi,

Thanks for your interest in our work. Sorry for the late reply.

Our results should be reproducible by running inference.py using the provided checkpoints. Could you please specify which JSON file is needed?

wenxi-yue commented 7 months ago

Hi,

I have just noticed that the previous checkpoint link has expired. I have updated the link, so you can now use the provided checkpoints to reproduce the results by running Inference.py. Thank you!

Kunaldargan commented 7 months ago

Hello @wenxi-yue , Thank you for your reply. Can you please provide the results used to evaluate the model? Typically in the COCO JSON format. Or if you can provide output masks which were used to calculate mcIOU and mIOU that will be equally helpful.

Regards

wenxi-yue commented 7 months ago

Hi,

We did not use the COCO JSON format for evaluation due to the specialised evaluation metrics of the EndoVis datasets. Instead, we read the predicted and ground-truth masks and calculated their IoUs using the evaluation code provided by previous works (without employing the COCO JSON format).

To obtain our output masks, you may run inference.py at here, and simply print endovis_masks here. endovis_masks is a dictionary that stores all our output masks. And you may generate the COCO JSON files from these masks.

Hope this helps.

Kunaldargan commented 7 months ago

Closing the issue, Thank you for your time @wenxi-yue