search

facebookresearch / detectron2

Detectron2 is a platform for object detection, segmentation and other visual recognition tasks.
https://detectron2.readthedocs.io/en/latest/
Apache License 2.0
29.97k stars 7.41k forks source link

Let DefaultPredictor support HSV or LAB color schemes #3144

Open spatiallysaying opened 3 years ago

spatiallysaying commented 3 years ago

Instructions To Reproduce the 🐛 Bug:

  1. Full runnable code or full changes you made: 
    
#We are importing our own Trainer Module here to use the COCO validation evaluation during training. Otherwise no validation eval occurs.
from detectron2.engine import DefaultTrainer
from detectron2.evaluation import COCOEvaluator

class CocoTrainer(DefaultTrainer): @classmethod def build_evaluator(cls, cfg, dataset_name, output_folder=None): cfg.INPUT.FORMAT = "HSV" if output_folder is None: os.makedirs("coco_eval", exist_ok=True) output_folder = "coco_eval"

return COCOEvaluator(dataset_name, cfg, False, output_folder)

from detectron2.engine import DefaultTrainer from detectron2.config import get_cfg import os

cfg = get_cfg() cfg.merge_from_file("./detectron2_repo/configs/COCO-Detection/faster_rcnn_R_101_FPN_3x.yaml") cfg.DATASETS.TRAIN = ("rwythr",) cfg.DATASETS.TEST = () # no metrics implemented for this dataset cfg.TEST.EVAL_PERIOD = 100 cfg.DATALOADER.NUM_WORKERS = 2 cfg.MODEL.WEIGHTS = "detectron2://COCO-Detection/faster_rcnn_R_101_FPN_3x/137851257/model_final_f6e8b1.pkl" cfg.SOLVER.IMS_PER_BATCH = 2 cfg.SOLVER.BASE_LR = 0.0002 cfg.SOLVER.MAX_ITER = 10 # 300 iterations seems good enough, but you can certainly train longer cfg.MODEL.ROI_HEADS.BATCH_SIZE_PER_IMAGE = 128 # faster, and good enough for this toy dataset cfg.MODEL.ROI_HEADS.NUM_CLASSES = 1 cfg.MODEL.DEVICE = "cpu" cfg.INPUT.RANDOM_FLIP = "horizontal"

Cropping type. See documentation of detectron2.data.transforms.RandomCrop for explanation.

cfg.INPUT.CROP.TYPE = "relative_range"

Size of crop in range (0, 1] if CROP.TYPE is "relative" or "relative_range" and in number of

pixels if CROP.TYPE is "absolute"

cfg.INPUT.CROP.SIZE = [0.9, 0.9] cfg.INPUT.FORMAT = "HSV"

os.makedirs(cfg.OUTPUT_DIR, exist_ok=True) If making changes to the project itself, please use output of the following command: git rev-parse HEAD; git diff

``` 2. What exact command you run: #test evaluation from detectron2.data import DatasetCatalog, MetadataCatalog, build_detection_test_loader from detectron2.evaluation import COCOEvaluator, inference_on_dataset cfg.MODEL.WEIGHTS = os.path.join(cfg.OUTPUT_DIR, "model_final.pth") cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.50 predictor = DefaultPredictor(cfg) evaluator = COCOEvaluator("rwythr", cfg, False, output_dir="./output/") val_loader = build_detection_test_loader(cfg, "rwythr") inference_on_dataset(trainer.model, val_loader, evaluator) 3. __Full logs__ or other relevant observations: ``` --------------------------------------------------------------------------- AssertionError Traceback (most recent call last) in () 5 cfg.MODEL.WEIGHTS = os.path.join(cfg.OUTPUT_DIR, "model_final.pth") 6 cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.50 ----> 7 predictor = DefaultPredictor(cfg) 8 evaluator = COCOEvaluator("rwythr", cfg, False, output_dir="./output/") 9 val_loader = build_detection_test_loader(cfg, "rwythr") /content/detectron2_repo/detectron2/engine/defaults.py in __init__(self, cfg) 294 295 self.input_format = cfg.INPUT.FORMAT --> 296 assert self.input_format in ["RGB", "BGR"], self.input_format 297 298 def __call__(self, original_image): AssertionError: HSV ``` 4. please simplify the steps as much as possible so they do not require additional resources to run, such as a private dataset. ## Expected behavior: I assumed HSV shall work directly but failing in evaluation. Detectron configuration [documentation ](url)mentions that we can use HSV like `_C.INPUT.FORMAT = "HSV"` ## Environment: Provide your environment information using the following command: ``` wget -nc -q https://github.com/facebookresearch/detectron2/raw/master/detectron2/utils/collect_env.py && python collect_env.py ``` No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda' --------------------- --------------------------------------------------------------- sys.platform linux Python 3.7.10 (default, May 3 2021, 02:48:31) [GCC 7.5.0] numpy 1.19.5 detectron2 0.4.1 @/content/detectron2_repo/detectron2 Compiler GCC 7.5 CUDA compiler not available DETECTRON2_ENV_MODULE PyTorch 1.8.1+cu101 @/usr/local/lib/python3.7/dist-packages/torch PyTorch debug build False GPU available False Pillow 7.1.2 torchvision 0.9.1+cu101 @/usr/local/lib/python3.7/dist-packages/torchvision fvcore 0.1.5 iopath 0.1.8 cv2 4.1.2 --------------------- --------------------------------------------------------------- PyTorch built with: - GCC 7.3 - C++ Version: 201402 - Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel(R) 64 architecture applications - Intel(R) MKL-DNN v1.7.0 (Git Hash 7aed236906b1f7a05c0917e5257a1af05e9ff683) - OpenMP 201511 (a.k.a. OpenMP 4.5) - NNPACK is enabled - CPU capability usage: AVX2 - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=10.1, CUDNN_VERSION=7.6.3, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.8.1, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, If your issue looks like an installation issue / environment issue, please first try to solve it yourself with the instructions in https://detectron2.readthedocs.io/tutorials/install.html#common-installation-issues
ppwwyyxx commented 3 years ago

DefaultPredictor is just for simple demo, so it currently does not support extra color schemes.

As its documentation https://detectron2.readthedocs.io/en/latest/modules/engine.html#detectron2.engine.defaults.DefaultPredictor says:

If you’d like to do anything more fancy, please refer to its source code as examples to build and use the model manually.

spatiallysaying commented 3 years ago

@ppwwyyxx, any suggestion on how to go about it? I am assuming that multiple color formats are already supported per the link. https://github.com/facebookresearch/detectron2/issues/url . I am just thinking the if training and inference are correctly put, this should work.

smothry commented 9 months ago

Has any further work been done on using different color schemes with Detectron2 DefaultPredictor? HSV in particular?