How to detect only one class (person) from the coco pre trained model

[anki92]

Can any one tell me how to select only one class which is 'person' in my case from coco data for instance segmentation. Pre trained model. (mask_rcnn_R_50_FPN_3x.yaml)

I want to detect 'person' from the given image only.

[ppwwyyxx]

The results contain the labels so you can select results that are "person". https://detectron2.readthedocs.io/modules/structures.html#detectron2.structures.Instances

[ppwwyyxx]

It doesn't look like your issue is related to the original, and from your abstract description I cannot understand clearly what you did. If the code is not doing what you expected, please include details about the problem following the issue template.

[bamwani]

@ppwwyyxx Is there a way to choose the classes to detect in pre-trained coco model? Like from coco's 80 classes, I just want to detect 3 classes. As you said we can select this from results. I'm unable to find the "Results". Code is running perfectly fine. I just want to filter the classes I want to detect in pre-trained coco model.

I'm running it through demo.py file.

[ppwwyyxx]

"results" means the model's outputs, whose format is documented in https://detectron2.readthedocs.io/tutorials/models.html#model-output-format.

For demo the results are produced in demo/predictor.py.

[bamwani]

@ppwwyyxx I can get the list of classes from predictions['instances'].pred_classes.tolist() I dont want to check the classes AFTER detections, I want to put a filter BEFORE detections. I do not want to see bounding box of any class apart from "mobile"

Is there a way to do this in demo?

[ppwwyyxx]

No

[bamwani]

so, At least is it possible if I use a builtin model, and provide input as mentioned above? Instead of demo? or what is the alternative?

[ppwwyyxx]

https://github.com/facebookresearch/detectron2/issues/556#issuecomment-574229540 applies to any builtin models.

[bamwani]

and also for pretrained models?

[ppwwyyxx]

Yes

[MahmoudKounyahli]

Hello, I have the following problem:

I want to detect only one class with the pretrained models, e.g. "person", because I only want to detect one class in the image, I want to ignore other classes in the image, if I let the Detectorn2 detect all classes of COCO (81 classes), runs super slow (5 sec per image), I think, if the Detectorn2 detects only one class in the image, it might run faster, because I need it for real time detection.

or as you say @ppwwyyxx or can someone tell me how to use the parameter "gt_classes" at Model Input Format, I can't find any example for this.

At GETTINGSTARTED.md on Colab Notebook I found this `MetadataCatalog.get("balloon" + d).set(thing_classes=["balloon"]` but that's only possible if i want to train my own model, and i don't want that, i want to use pretrained model .

Thanks in advance !

[pvti]

@MahmoudKounyahli Personally I think inference time depends on both number of classes and instances in each class. In your case, just one class ''person'. I think you may retrain model for 1 class on your datasets. The pre-trained model in MODEL ZOO is training for multi classes (e.g 80 classes on COCO). You should use it as a initial weights and train your personal model.

[maryamasa]

Hi, I have another question, is it possible just add a class to those classes in coco? like 5 more and keep weight for 80 classes and add other weights just for that 5 new classes, without training model again for all classes? Thanks

[ManuFU]

@maryamasa check the chapter "train on custom dataset" https://colab.research.google.com/drive/16jcaJoc6bCFAQ96jDe2HwtXj7BMD_-m5

[Mouiad-JRA]

The results contain the labels so you can select results that are "person".

plz can u tell me how get person label??

[amrahsmaytas]

Hey Everyone, If anyone still facing the issue of how to select the custom classes for eg: person class only in this issue then, you can check my colab notebook where i have written a small function which will help to select any custom class.

The colab link is: https://colab.research.google.com/drive/1x-eMvFQTLBTr7ho9ZlYkHF0NmyUyAlxT?usp=sharing

[amrahsmaytas]

The results contain the labels so you can select results that are "person".

plz can u tell me how get person label??

you can check my colab notebook , it will do the job.

[miguelbedia]

Hi @amrahsmaytas , I've implemented the code you suggested in the colab link and i've experimenting an AssertionError in my detectron2. The base of the detectron2 is the same as the demo.py. This assertion error appears me in the line 642 of the script visualizer.py, in the following code: if labels in not None assert len(labels) == num_instances.

I dont know why this is happening but maybe you know why.

Maybe you know how to help me :( Thank you very must for your suggestion!! :D

[amrahsmaytas]

Can you share your output['instances']

Maybe the person class is not detected by the model!

Please check, Thanks

P.s: you need to input the class index of the class you want to select. Make sure you have done that, if you want to keep other class than person

[miguelbedia]

Hi @amrahsmaytas I've upload the image. As you can see the classes are output in the vector (print(classes)).

[amrahsmaytas]

Send me the modified loc, you have done!

[miguelbedia]

This is just the code. The insides of the visualizer script remains the same as the original

[miguelbedia]

Hey man, you want this???

[miguelbedia]

in my case semantic detection is not used. Anything you need tell me.

[amrahsmaytas]

The code is optimised for only semantic, I will share a code which also consider BBoxes, sooner.

Thanks

[miguelbedia]

The code is optimised for only semantic, I will share a code which also consider BBoxes, sooner.

Thanks

@amrahsmaytas Thank you very much for your help!!! Nice to have people like you in the community!!! I'll be waiting for that code then. Thanks!!!

[amrahsmaytas]

The code is optimised for only semantic, I will share a code which also consider BBoxes, sooner.

Thanks

@amrahsmaytas Thank you very much for your help!!! Nice to have people like you in the community!!! I'll be waiting for that code then. Thanks!!!

Pleasure

[amrahsmaytas]

The code is optimised for only semantic, I will share a code which also consider BBoxes, sooner. Thanks

@amrahsmaytas Thank you very much for your help!!! Nice to have people like you in the community!!! I'll be waiting for that code then. Thanks!!!

I have updated the colab notebook w.r.t to detection

You can find the code in the cell "(For detection) Prediction of only 1 class using Detectron2"

Thanks

[ManuFU]

If you want a shorter solution. You can also just filter your output by the class you want.

outputs["instances"] -> outputs['instances'][outputs['instances'].pred_classes == 0]

0 is the index for persons.

Before:

After:

[miguelbedia]

Thatbk you very much for your answer. I really apreciate it. I'll check it!!! But as you showed, it seems pretty clear.

Thank you very much!! :)

El mié., 7 abr. 2021 13:35, ManuFU @.***> escribió:

If you want a shorter solution. You can also just filter your output by the class you want.

outputs["instances"] -> outputs['instances'][outputs['instances'].pred_classes == 0]

0 is the index for persons.

Before: [image: image] https://user-images.githubusercontent.com/15906404/113860024-d8f0f480-97a5-11eb-9c36-3c96bd0bdde6.png

After: [image: image] https://user-images.githubusercontent.com/15906404/113860066-e7d7a700-97a5-11eb-9a32-909f7dd48beb.png

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/facebookresearch/detectron2/issues/556#issuecomment-814841618, or unsubscribe https://github.com/notifications/unsubscribe-auth/ATQ5FXFZ2HLULWNL7XM77SDTHQ7OJANCNFSM4J5NV5FA .

[miguelbedia]

If you want a shorter solution. You can also just filter your output by the class you want.

outputs["instances"] -> outputs['instances'][outputs['instances'].pred_classes == 0]

0 is the index for persons.

Before:

After:

This solution is perfect too. Shorter and fast. Thank you very much!! :D

[codeperfectplus]

It will detect only class 0 (person).

rects = outputs["instances"][outputs["instances"].pred_classes == 0].pred_boxes.tensor.cpu().numpy()
scores = outputs["instances"][outputs["instances"].pred_classes == 0].scores.tolist()
classes = outputs["instances"][outputs["instances"].pred_classes == 0].pred_classes.tolist()

[DishantMewada]

Hi,

I want to use the specific class as mentioned here for getting the inference without training the model. I have the annotated dataset and want to evaluate the pre-trained model available in Detectron2. I can evaluate my model trained on a custom dataset but just wanted to know how to incorporate it for the pre-built model.

My code for training the model is as below. I just want to evaluate the pre-trained model on my custom dataset containing a specific class and ignore all other classes in the pre-trained model for evaluation (here in my dataset: evaluate for only 'plastic bag' which is present in LVIS as well) In the code, I have registered the 'train' and 'val' datasets before running the cell.

from detectron2.engine import DefaultTrainer
from detectron2.config import get_cfg

from detectron2.evaluation import COCOEvaluator, inference_on_dataset
from detectron2.data import build_detection_test_loader

import os

cfg = get_cfg()
cfg.merge_from_file("./detectron2_repo/configs/LVISv0.5-InstanceSegmentation/mask_rcnn_X_101_32x8d_FPN_1x.yaml")
cfg.DATASETS.TRAIN = ("train",)
cfg.DATASETS.TEST = ()  
cfg.DATALOADER.NUM_WORKERS = 4
cfg.MODEL.WEIGHTS = "detectron2://LVISv0.5-InstanceSegmentation/mask_rcnn_X_101_32x8d_FPN_1x/144219108/model_final_5e3439.pkl" 
cfg.SOLVER.IMS_PER_BATCH = 4
cfg.SOLVER.BASE_LR = 0.02
cfg.SOLVER.MAX_ITER = 30000   
cfg.MODEL.ROI_HEADS.BATCH_SIZE_PER_IMAGE = 128  
cfg.MODEL.ROI_HEADS.NUM_CLASSES = 1

os.makedirs(cfg.OUTPUT_DIR, exist_ok=True)
trainer = DefaultTrainer(cfg)
trainer.resume_or_load(resume=False)
trainer.train()

cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.5   
cfg.DATASETS.TEST = ( )
predictor = DefaultPredictor(cfg)

evaluator = COCOEvaluator("val", output_dir= "./output")
val_loader = build_detection_test_loader(cfg, "val")
print(inference_on_dataset(predictor.model, val_loader, evaluator))

[innat]

@ppwwyyxx In the doc, as you pointed out, it can be done

category_3_detections = instances[instances.pred_classes == 3]
confident_detections = instances[instances.scores > 0.9]

However, I think it would be a nice addition if there is any built-in functionality for this. It's quite needed often. HERE is the discussion (before I saw this post). Something like

cfg = get_cfg()
cfg.merge_from_file(model_zoo.get_config_file("COCO-InstanceSegmentation/..."))
cfg.MODEL.WEIGHTS = model_zoo.get_checkpoint_url("COCO-InstanceSegmentation/...")

# HERE, 
cfg.CLASS_INDICES = [1, 5, 20, 70] # Or, 
cfg.CLASS_STRING = ['person',  'airplane',  'cow',  'oven']

predictor = DefaultPredictor(cfg)
outputs = predictor(im)