Closed dxlong2000 closed 4 years ago
ppwwyyxx
commented
4 years ago cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.5
Config parameters should not be changed if you want to get the same results.
Cannot help on "method 2" because the details of "method 2" are not provided following the issue template.
dxlong2000
commented
4 years ago cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.5
Config parameters should not be changed if you want to get the same results.
Cannot help on "method 2" because the details of "method 2" are not provided following the issue template.
Sorry I don't understand your point I didn't change this threshold. Could you please tell me how I can get the same result I think that all the steps above are clear. If I had any wrong set-up hyperparameters, please tell me where and I can edit something to get the same result as the tutorial.
Thanks in advance!
ppwwyyxx
commented
4 years ago cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.5 should be removed.
Dear all,
This is an evaluation problem again. I have tried 2 methods of evaluation on the dataset: Val2017 from COCO Datasets: http://images.cocodataset.org/zips/val2017.zip on the pre-trained model: Faster R-CNN: X101. I got 2 same answers. However, I find out that the result I got is not the same as
boxAP: 43.0in theMODEL ZOO:https://github.com/facebookresearch/detectron2/blob/master/MODEL_ZOO.md. Could anyone tell me why?git diff) Method 1: Using inference on datasetfrom detectron2.modeling import build_model model = build_model(cfg) from detectron2.checkpoint import DetectionCheckpointer DetectionCheckpointer(model).load(cfg.MODEL.WEIGHTS)
from detectron2.data import DatasetCatalog from detectron2.data.datasets import register_coco_instances register_coco_instances('/content/datasets/coco/val2017', {}, '/content/datasets/coco/annotations/instances_val2017.json', '/content/datasets/coco/val2017') dataset_dicts = DatasetCatalog.get('/content/datasets/coco/val2017') val2017_metadata = MetadataCatalog.get('/content/datasets/coco/val2017')
from detectron2.data import build_detection_test_loader from detectron2.evaluation import COCOEvaluator, inference_on_dataset evaluator = COCOEvaluator('/content/datasets/coco/val2017', cfg, True, output_dir='./output') val_loader = build_detection_test_loader(cfg, '/content/datasets/coco/val2017') inference_on_dataset(model, val_loader, evaluator)
OrderedDict([('bbox', {'AP': 39.623126911232085, 'AP-airplane': 63.35367597935957, 'AP-apple': 20.01435656659263, 'AP-backpack': 14.692231204208763, 'AP-banana': 18.21074950234784, 'AP-baseball bat': 29.935608422472416, 'AP-baseball glove': 37.62812019014162, 'AP-bear': 69.95245293321169, 'AP-bed': 38.132203723292406, 'AP-bench': 24.497013962494776, 'AP-bicycle': 31.18113068420435, 'AP-bird': 36.34963891886067, 'AP-boat': 26.363982212997954, 'AP-book': 10.767954775427475, 'AP-bottle': 36.55367585828511, 'AP-bowl': 37.866392539334655, 'AP-broccoli': 20.20642967228571, 'AP-bus': 64.27183047453165, 'AP-cake': 32.87370279440155, 'AP-car': 42.99110432681394, 'AP-carrot': 18.978856904842818, 'AP-cat': 66.32943678498886, 'AP-cell phone': 35.193831641694594, 'AP-chair': 25.825456930100017, 'AP-clock': 47.56479370787892, 'AP-couch': 38.9112546976581, 'AP-cow': 52.81053997588185, 'AP-cup': 40.14237586365595, 'AP-dining table': 25.053810126538682, 'AP-dog': 60.65887341800889, 'AP-donut': 38.759712841029184, 'AP-elephant': 61.12713920385456, 'AP-fire hydrant': 62.71991648748373, 'AP-fork': 36.118347843307916, 'AP-frisbee': 63.03606420187224, 'AP-giraffe': 64.26543621089753, 'AP-hair drier': 5.247524752475248, 'AP-handbag': 13.640549876230706, 'AP-horse': 57.241396359342566, 'AP-hot dog': 28.860722544139044, 'AP-keyboard': 49.381347621636564, 'AP-kite': 38.212616998741304, 'AP-knife': 18.67689978984454, 'AP-laptop': 58.17296712322765, 'AP-microwave': 54.76485076277412, 'AP-motorcycle': 41.83626319111957, 'AP-mouse': 60.294095724915785, 'AP-orange': 25.546145743522942, 'AP-oven': 30.56599233826392, 'AP-parking meter': 44.91155755086583, 'AP-person': 53.056360232207425, 'AP-pizza': 49.146909167865175, 'AP-potted plant': 24.897419394223704, 'AP-refrigerator': 53.19459830344534, 'AP-remote': 31.396883025298866, 'AP-sandwich': 29.8211363582134, 'AP-scissors': 26.6994728044233, 'AP-sheep': 49.13193728379734, 'AP-sink': 34.497415182121046, 'AP-skateboard': 53.44728573897266, 'AP-skis': 23.400320833059173, 'AP-snowboard': 35.302859845012684, 'AP-spoon': 18.63207863850262, 'AP-sports ball': 46.649827221359736, 'AP-stop sign': 65.39332050023295, 'AP-suitcase': 39.6054943938063, 'AP-surfboard': 38.34079201663481, 'AP-teddy bear': 43.39972099820209, 'AP-tennis racket': 48.210838264868684, 'AP-tie': 33.22232684519771, 'AP-toaster': 30.495049504950494, 'AP-toilet': 55.67390632071584, 'AP-toothbrush': 23.114731048344016, 'AP-traffic light': 25.80666772735204, 'AP-train': 60.65353211852693, 'AP-truck': 31.527304981042793, 'AP-tv': 54.78741680933443, 'AP-umbrella': 36.56158986369249, 'AP-vase': 34.07200384110469, 'AP-wine glass': 34.597125541067186, 'AP-zebra': 64.42279613693252, 'AP50': 56.98681652707609, 'AP75': 43.867850744159405, 'APl': 52.10633013210152, 'APm': 42.94432972137188, 'APs': 22.578027356979838})])