Evaluation with FreeSOLO_R101_30k_pl.pth

[fawnliu]

Hello, thank you very much for releasing the source code. When I run bash test.sh FreeSOLO_R101_30k_pl.pth, I get the following results. The AP on person is 0.903, but all other categories are indeed 0. Am I missing any key experimental settings?

[04/22 13:53:59] d2.evaluation.evaluator INFO: Inference done 4991/5000. Dataloading: 0.0046 s/iter. Inference: 0.0928 s/iter. Eval: 0.0549 s/iter. Total: 0.1524 s/iter. ETA=0:00:01
[04/22 13:54:00] d2.evaluation.evaluator INFO: Total inference time: 0:12:40.989735 (0.152350 s / iter per device, on 1 devices)
[04/22 13:54:00] d2.evaluation.evaluator INFO: Total inference pure compute time: 0:07:43 (0.092807 s / iter per device, on 1 devices)
[04/22 13:54:05] d2.evaluation.coco_evaluation INFO: Preparing results for COCO format ...
[04/22 13:54:05] d2.evaluation.coco_evaluation INFO: Saving results to training_dir/FreeSOLO_pl/inference/coco_instances_results.json
[04/22 13:54:09] d2.evaluation.coco_evaluation INFO: Evaluating predictions with official COCO API...
[04/22 13:54:55] d2.evaluation.coco_evaluation INFO: Evaluation results for bbox: 
|  AP   |  AP50  |  AP75  |  APs  |  APm  |  APl  |
|:-----:|:------:|:------:|:-----:|:-----:|:-----:|
| 0.011 | 0.028  | 0.009  | 0.004 | 0.010 | 0.031 |
[04/22 13:54:55] d2.evaluation.coco_evaluation INFO: Per-category bbox AP: 
| category      | AP    | category     | AP    | category       | AP    |
|:--------------|:------|:-------------|:------|:---------------|:------|
| person        | 0.903 | bicycle      | 0.000 | car            | 0.000 |
| motorcycle    | 0.000 | airplane     | 0.000 | bus            | 0.000 |
| train         | 0.000 | truck        | 0.000 | boat           | 0.000 |
| traffic light | 0.000 | fire hydrant | 0.000 | stop sign      | 0.000 |
| parking meter | 0.000 | bench        | 0.000 | bird           | 0.000 |
| cat           | 0.000 | dog          | 0.000 | horse          | 0.000 |
| sheep         | 0.000 | cow          | 0.000 | elephant       | 0.000 |
| bear          | 0.000 | zebra        | 0.000 | giraffe        | 0.000 |
| backpack      | 0.000 | umbrella     | 0.000 | handbag        | 0.000 |
| tie           | 0.000 | suitcase     | 0.000 | frisbee        | 0.000 |
| skis          | 0.000 | snowboard    | 0.000 | sports ball    | 0.000 |
| kite          | 0.000 | baseball bat | 0.000 | baseball glove | 0.000 |
| skateboard    | 0.000 | surfboard    | 0.000 | tennis racket  | 0.000 |
| bottle        | 0.000 | wine glass   | 0.000 | cup            | 0.000 |
| fork          | 0.000 | knife        | 0.000 | spoon          | 0.000 |
| bowl          | 0.000 | banana       | 0.000 | apple          | 0.000 |
| sandwich      | 0.000 | orange       | 0.000 | broccoli       | 0.000 |
| carrot        | 0.000 | hot dog      | 0.000 | pizza          | 0.000 |
| donut         | 0.000 | cake         | 0.000 | chair          | 0.000 |
| couch         | 0.000 | potted plant | 0.000 | bed            | 0.000 |
| dining table  | 0.000 | toilet       | 0.000 | tv             | 0.000 |
| laptop        | 0.000 | mouse        | 0.000 | remote         | 0.000 |
| keyboard      | 0.000 | cell phone   | 0.000 | microwave      | 0.000 |
| oven          | 0.000 | toaster      | 0.000 | sink           | 0.000 |
| refrigerator  | 0.000 | book         | 0.000 | clock          | 0.000 |
| vase          | 0.000 | scissors     | 0.000 | teddy bear     | 0.000 |
| hair drier    | 0.000 | toothbrush   | 0.000 |                |       |
[04/22 13:56:08] d2.evaluation.coco_evaluation INFO: Evaluation results for segm: 
|  AP   |  AP50  |  AP75  |  APs  |  APm  |  APl  |
|:-----:|:------:|:------:|:-----:|:-----:|:-----:|
| 0.012 | 0.030  | 0.008  | 0.001 | 0.005 | 0.036 |
[04/22 13:56:08] d2.evaluation.coco_evaluation INFO: Per-category segm AP: 
| category      | AP    | category     | AP    | category       | AP    |
|:--------------|:------|:-------------|:------|:---------------|:------|
| person        | 0.926 | bicycle      | 0.000 | car            | 0.000 |
| motorcycle    | 0.000 | airplane     | 0.000 | bus            | 0.000 |
| train         | 0.000 | truck        | 0.000 | boat           | 0.000 |
| traffic light | 0.000 | fire hydrant | 0.000 | stop sign      | 0.000 |
| parking meter | 0.000 | bench        | 0.000 | bird           | 0.000 |
| cat           | 0.000 | dog          | 0.000 | horse          | 0.000 |
| sheep         | 0.000 | cow          | 0.000 | elephant       | 0.000 |
| bear          | 0.000 | zebra        | 0.000 | giraffe        | 0.000 |
| backpack      | 0.000 | umbrella     | 0.000 | handbag        | 0.000 |
| tie           | 0.000 | suitcase     | 0.000 | frisbee        | 0.000 |
| skis          | 0.000 | snowboard    | 0.000 | sports ball    | 0.000 |
| kite          | 0.000 | baseball bat | 0.000 | baseball glove | 0.000 |
| skateboard    | 0.000 | surfboard    | 0.000 | tennis racket  | 0.000 |
| bottle        | 0.000 | wine glass   | 0.000 | cup            | 0.000 |
| fork          | 0.000 | knife        | 0.000 | spoon          | 0.000 |
| bowl          | 0.000 | banana       | 0.000 | apple          | 0.000 |
| sandwich      | 0.000 | orange       | 0.000 | broccoli       | 0.000 |
| carrot        | 0.000 | hot dog      | 0.000 | pizza          | 0.000 |
| donut         | 0.000 | cake         | 0.000 | chair          | 0.000 |
| couch         | 0.000 | potted plant | 0.000 | bed            | 0.000 |
| dining table  | 0.000 | toilet       | 0.000 | tv             | 0.000 |
| laptop        | 0.000 | mouse        | 0.000 | remote         | 0.000 |
| keyboard      | 0.000 | cell phone   | 0.000 | microwave      | 0.000 |
| oven          | 0.000 | toaster      | 0.000 | sink           | 0.000 |
| refrigerator  | 0.000 | book         | 0.000 | clock          | 0.000 |
| vase          | 0.000 | scissors     | 0.000 | teddy bear     | 0.000 |
| hair drier    | 0.000 | toothbrush   | 0.000 |                |       |
[04/22 13:56:09] d2.engine.defaults INFO: Evaluation results for coco_2017_val in csv format:
[04/22 13:56:09] d2.evaluation.testing INFO: copypaste: Task: bbox
[04/22 13:56:09] d2.evaluation.testing INFO: copypaste: AP,AP50,AP75,APs,APm,APl
[04/22 13:56:09] d2.evaluation.testing INFO: copypaste: 0.0113,0.0283,0.0085,0.0045,0.0101,0.0311
[04/22 13:56:09] d2.evaluation.testing INFO: copypaste: Task: segm
[04/22 13:56:09] d2.evaluation.testing INFO: copypaste: AP,AP50,AP75,APs,APm,APl
[04/22 13:56:09] d2.evaluation.testing INFO: copypaste: 0.0116,0.0296,0.0079,0.0005,0.0055,0.0362

[WXinlong]

Hi @scarleatt, your pasted result is for class-aware instance segmentation, which does not suit our case. The class-agnostic instance segmentation results will be printed after that. You just need to wait a few more seconds to get the output of the python tools/eval_cocoapi.py in test.sh.

[fawnliu]

Thanks for your reply.

[lcl-git-3d]

Hello,i can not download FreeSOLO_R101_30k_pl.pth now.Can you send to me,please?