COCO evaluation results do not match

[samiarshad]

Hi,

Thanks for sharing your work. I have executed the validation code from here on COCO validation set and received the results below. Could you please confirm if this is the expected output?

Thanks

------------------------------------------------------------- COCO Evaluation Results ----------------------------------------------- loading annotations into memory... Done (t=1.38s) creating index... index created! Loading and preparing results... DONE (t=7.31s) creating index... index created! Running per image evaluation... Evaluate annotation type bbox DONE (t=51.15s). Accumulating evaluation results... DONE (t=11.30s). Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.002 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.006 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.004 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.007 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.008 Running per image evaluation... Evaluate annotation type segm DONE (t=60.37s). Accumulating evaluation results... DONE (t=10.75s). Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.002 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.005 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.004 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.006 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.006

[berry-ding]

hi, could you provide the json file

[samiarshad]

Hello,

Please find the JSON file attached.

Thanks

predictions.json

[MMa321]

Hello, thank you for your sharing. I also want to evaluate this model on the coco dataset. May I ask how did you deploy it and how could you get the result

[Armadillooutfitter]

Dear MMa321,

I deployed the model using Docker and ran it on the COCO dataset. I obtained the results by running the evaluation script provided with the model.

Best regards, Armadillo Outfitter

On Sun, Feb 25, 2024 at 11:01 PM MMa321 @.***> wrote:

Hello, thank you for your sharing. I also want to evaluate this model on the coco dataset. May I ask how did you deploy it and how could you get the result

— Reply to this email directly, view it on GitHub https://github.com/CASIA-IVA-Lab/FastSAM/issues/183#issuecomment-1963277386, or unsubscribe https://github.com/notifications/unsubscribe-auth/BA3Y3ZYWOL7YYC6RTSFCCBLYVQCIBAVCNFSM6AAAAAA7AHQ46SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRTGI3TOMZYGY . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[MMa321]

尊敬的MMa321， 我使用 Docker 部署了模型，并在 COCO 数据集上运行了它。我 通过运行随 型。 此致敬意 犰狳装备 ... 2024 年 2 月 25 日星期日晚上 11：01 MMa321 @.>写道： 您好，谢谢您的分享。我还想评估这个模型 Coco 数据集。请问您是如何部署它的，以及如何获得 结果 — 直接回复此邮件，在 GitHub 上查看 <#183（评论）>， 或取消订阅 https://github.com/notifications/unsubscribe-auth/BA3Y3ZYWOL7YYC6RTSFCCBLYVQCIBAVCNFSM6AAAAAA7AHQ46SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRTGI3TOMZYGY . 您收到此消息是因为您订阅了此线程。消息 ID： @.>

Hello. Thank you for your reply. Could you please provide the specific code？

[MMa321]

晉語

感谢您分享您的作品。我已经在COCO验证集上执行了验证代码，并收到了以下结果。您能否确认这是否是预期的输出？

谢谢

------------------------------------------------------------- COCO 评估结果-----------------------------------------------将注释加载到内存中...完成 （t=1.38s） 创建索引...索引已创建！加载和准备结果...完成 （t=7.31s） 正在创建索引...索引已创建！正在按图像评估运行...评估注释类型 bbox DONE （t=51.15s）。累积评估结果...完成 （t=11.30s）。平均精度 （AP） @[ IoU=0.50：0.95 | area= all | maxDets=100 ] = 0.000 平均精度 （AP） @[ IoU=0.50 | area= all | maxDets=100 ] = 0.000 平均精度 （AP） @[ IoU=0.75 | area= all | maxDets=100 ] = 0.000 平均精度 （AP） @[ IoU=0.50：0.95 | area= small | maxDets=100 ] = 0.000 平均精度 （AP） @[ IoU=0.50：0.95 | area=medium | maxDets=100 ] = 0.000平均精度 （AP） @[ IoU=0.50：0.95 | area= large | maxDets=100 ] = 0.000 平均召回率 （AR） @[ IoU=0.50：0.95 | area= all | maxDets= 1 ] = 0.000 平均召回率 （AR） @[ IoU=0.50：0.95 | area= all | maxDets= 10 ] = 0.002 平均召回率 （AR） @[ IoU=0.50：0.95 | area= all | maxDets=100 ] = 0.006 平均召回率 （AR） @[ IoU=0.50：0.95 | area= small | maxDets=100 ] = 0.004平均召回率 （AR） @[ IoU=0.50：0.95 | area=medium | maxDets=100 ] = 0.007 平均召回率 （AR） @[ IoU=0.50：0.95 | area= large | maxDets=100 ] = 0.008 每个图像评估正在运行...计算注释类型 segm DONE （t=60.37s）。累积评估结果...完成 （t=10.75s）。平均精度 （AP） @[ IoU=0.50：0.95 | area= all | maxDets=100 ] = 0.000 平均精度 （AP） @[ IoU=0.50 | area= all | maxDets=100 ] = 0.000 平均精度 （AP） @[ IoU=0.75 | area= all | maxDets=100 ] = 0.000 平均精度 （AP） @[ IoU=0.50：0.95 | area= small | maxDets=100 ] = 0.000 平均精度 （AP） @[ IoU=0.50：0.95 | area=medium | maxDets=100 ] = 0.000平均精度 （AP） @[ IoU=0.50：0.95 | area= 大 | maxDets=100 ] = 0.000 平均召回率 （AR） @[ IoU=0.50：0.95 | area= all | maxDets= 1 ] = 0.000 平均召回率 （AR） @[ IoU=0.50：0.95 | area= all | maxDets= 10 ] = 0.002 平均召回率 （AR） @[ IoU=0.50：0.95 | area= all | maxDets=100 ] = 0.005 平均召回率 （AR） @[ IoU=0.50：0.95 | area= small | maxDets=100 ] = 0.004平均召回率 （AR） @[ IoU=0.50：0.95 | area=medium | maxDets=100 ] = 0.006 平均召回率 （AR） @[ IoU=0.50：0.95 | area= large | maxDets=100 ] = 0.006

Hello, I always report errors in the verification process, is it convenient for you to share your verification code