Possibility to test on one single image

an99990 commented 2 years ago

Hi, I would like to test the instance segmentation model on a single image. Is it possible. In test.py i dont see any input argument to test on our own images. Here is the command i did

python tools/test.py configs/semask_swin/coco_stuff10k/semfpn_semask_swin_large_patch4_window12_640x640_80k_coco10k.py semask_large_fpn_coco10k.pth --eval mIoU --show-dir visuals

Thank you

praeclarumjj3 commented 2 years ago

Hi @an99990, our codebase for Semask-FPN is based on the MMSegmentation library. You may refer to the demo.py file in the parent mmseg repo for running inference on a single image.

I will add a demo file to this repo when I get some free time.

an99990 commented 2 years ago

hi @praeclarumjj3, thank you for your answer. So i tried simply using their demo_image and give the path like so :

python image_demo.py ../../../../images/person_bike.jpg configs/semask_swin/coco_stuff10k/semfpn_semask_swin_large_patch4_window12_640x640_80k_coco10k.py semask_large_fpn_coco10k.pth

but i got this AssertionError: MMCV==1.4.4 is used but incompatible. Please install mmcv>=[1, 1, 4], <=[1, 3, 0]. which version of mmcv version are you guys using ? I see in init that you guys use a version between MMCV_MIN = '1.1.4' MMCV_MAX = '1.3.0' . Are you considering upgrading the versions ? would it involve a lot of changes ? Can i do something to allow the use of mvcc 1.4.4 ?

praeclarumjj3 commented 2 years ago

I think if you change the MMCV_MAX on line 6 to allow for 1.4.4, the code should work. https://github.com/Picsart-AI-Research/SeMask-Segmentation/blob/786f395fab4e156970628134cb49eb3547d7287b/SeMask-FPN/mmseg/__init__.py#L6

If that does not work, you could use an appropriate mmcv version between the specified range.

an99990 commented 2 years ago

thanks i confirm that it works changing that line

an99990 commented 2 years ago

by compiling the demo_image from mmsegmentation with semask config, i get errors : KeyError: "EncoderDecoder: 'SeMaskSwinTransformer is not in the models registry'" do you know what i can do ? thank you

praeclarumjj3 commented 2 years ago

Try running pip install -e . inside SeMask-FPN directory.

an99990 commented 2 years ago

thank you for your prompt answer. I dont have the error anymore, will test after installing everything. Also theres is a typo in the requirements/runtime . its suppoer to be opencv-python i believe :). will update after installation

https://github.com/Picsart-AI-Research/SeMask-Segmentation/blob/786f395fab4e156970628134cb49eb3547d7287b/SeMask-FPN/requirements/runtime.txt#L8

praeclarumjj3 commented 2 years ago

Thanks for pointing that out, @an99990! I will correct that in a later commit.

an99990 commented 2 years ago

updates : i still get the same issue. It was complaining about mmcv for a moment so i reinstalled it, then it complained about open-cv. Will dig into more, please let me know if you think of anything

praeclarumjj3 commented 2 years ago

Did you install opencv-python?

an99990 commented 2 years ago

yes i did install opencv-python

praeclarumjj3 commented 2 years ago

What's the exact error?

an99990 commented 2 years ago

root@0d2c6aa6ee79:/opt/halodi/halodi-segmentation/halodi_segmentation# python image_demo.py
Traceback (most recent call last):
  File "/opt/halodi/halodi-segmentation/halodi_segmentation/models/mmcv/mmcv/utils/registry.py", line 52, in build_from_cfg
    return obj_cls(**args)
  File "/opt/halodi/halodi-segmentation/halodi_segmentation/models/mmsegmentation/mmseg/models/segmentors/encoder_decoder.py", line 36, in __init__
    self.backbone = builder.build_backbone(backbone)
  File "/opt/halodi/halodi-segmentation/halodi_segmentation/models/mmsegmentation/mmseg/models/builder.py", line 20, in build_backbone
    return BACKBONES.build(cfg)
  File "/opt/halodi/halodi-segmentation/halodi_segmentation/models/mmcv/mmcv/utils/registry.py", line 212, in build
    return self.build_func(*args, **kwargs, registry=self)
  File "/opt/halodi/halodi-segmentation/halodi_segmentation/models/mmcv/mmcv/cnn/builder.py", line 27, in build_model_from_cfg
    return build_from_cfg(cfg, registry, default_args)
  File "/opt/halodi/halodi-segmentation/halodi_segmentation/models/mmcv/mmcv/utils/registry.py", line 44, in build_from_cfg
    raise KeyError(
KeyError: 'SeMaskSwinTransformer is not in the models registry'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "image_demo.py", line 49, in <module>
    main()
  File "image_demo.py", line 36, in main
    model = init_segmentor(args[0], args[-1], device='cuda:0')
  File "/opt/halodi/halodi-segmentation/halodi_segmentation/models/mmsegmentation/mmseg/apis/inference.py", line 32, in init_segmentor
    model = build_segmentor(config.model, test_cfg=config.get('test_cfg'))
  File "/opt/halodi/halodi-segmentation/halodi_segmentation/models/mmsegmentation/mmseg/models/builder.py", line 48, in build_segmentor
    return SEGMENTORS.build(
  File "/opt/halodi/halodi-segmentation/halodi_segmentation/models/mmcv/mmcv/utils/registry.py", line 212, in build
    return self.build_func(*args, **kwargs, registry=self)
  File "/opt/halodi/halodi-segmentation/halodi_segmentation/models/mmcv/mmcv/cnn/builder.py", line 27, in build_model_from_cfg
    return build_from_cfg(cfg, registry, default_args)
  File "/opt/halodi/halodi-segmentation/halodi_segmentation/models/mmcv/mmcv/utils/registry.py", line 55, in build_from_cfg
    raise type(e)(f'{obj_cls.__name__}: {e}')
KeyError: "EncoderDecoder: 'SeMaskSwinTransformer is not in the models registry'"
root@0d2c6aa6ee79:/opt/halodi/halodi-segmentation/halodi_segmentation# python
Python 3.8.12 | packaged by conda-forge | (default, Oct 12 2021, 21:59:51)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cv2

praeclarumjj3 commented 2 years ago

After reinstalling mmcv, did you re-run pip install -e .? If not, then you need to rebuild mmsegmentation after a new mmcv installation.

an99990 commented 2 years ago

root@0d2c6aa6ee79:/opt/halodi/halodi-segmentation/halodi_segmentation/models/mmsegmentation# pip install -e .
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Obtaining file:///opt/halodi/halodi-segmentation/halodi_segmentation/models/mmsegmentation
Requirement already satisfied: matplotlib in /opt/conda/lib/python3.8/site-packages (from mmsegmentation==0.21.1) (3.5.1)
Requirement already satisfied: numpy in /opt/conda/lib/python3.8/site-packages (from mmsegmentation==0.21.1) (1.22.0)
Requirement already satisfied: packaging in /opt/conda/lib/python3.8/site-packages (from mmsegmentation==0.21.1) (21.3)
Requirement already satisfied: prettytable in /opt/conda/lib/python3.8/site-packages (from mmsegmentation==0.21.1) (3.0.0)
Requirement already satisfied: pyparsing>=2.2.1 in /opt/conda/lib/python3.8/site-packages (from matplotlib->mmsegmentation==0.21.1) (3.0.6)
Requirement already satisfied: python-dateutil>=2.7 in /opt/conda/lib/python3.8/site-packages (from matplotlib->mmsegmentation==0.21.1) (2.8.2)
Requirement already satisfied: cycler>=0.10 in /opt/conda/lib/python3.8/site-packages (from matplotlib->mmsegmentation==0.21.1) (0.11.0)
Requirement already satisfied: fonttools>=4.22.0 in /opt/conda/lib/python3.8/site-packages (from matplotlib->mmsegmentation==0.21.1) (4.29.0)
Requirement already satisfied: kiwisolver>=1.0.1 in /opt/conda/lib/python3.8/site-packages (from matplotlib->mmsegmentation==0.21.1) (1.3.2)
Requirement already satisfied: pillow>=6.2.0 in /opt/conda/lib/python3.8/site-packages (from matplotlib->mmsegmentation==0.21.1) (8.2.0)
Requirement already satisfied: six>=1.5 in /opt/conda/lib/python3.8/site-packages (from python-dateutil>=2.7->matplotlib->mmsegmentation==0.21.1) (1.16.0)
Requirement already satisfied: wcwidth in /opt/conda/lib/python3.8/site-packages (from prettytable->mmsegmentation==0.21.1) (0.2.5)
Installing collected packages: mmsegmentation
  Attempting uninstall: mmsegmentation
    Found existing installation: mmsegmentation 0.21.1
    Uninstalling mmsegmentation-0.21.1:
      Successfully uninstalled mmsegmentation-0.21.1
  Running setup.py develop for mmsegmentation
Successfully installed mmsegmentation-0.21.1
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
root@0d2c6aa6ee79:/opt/halodi/halodi-segmentation/halodi_segmentation/models/mmsegmentation# cd ..
root@0d2c6aa6ee79:/opt/halodi/halodi-segmentation/halodi_segmentation/models# cd ..
root@0d2c6aa6ee79:/opt/halodi/halodi-segmentation/halodi_segmentation# python image_demo.py
Traceback (most recent call last):
  File "/opt/halodi/halodi-segmentation/halodi_segmentation/models/mmcv/mmcv/utils/registry.py", line 52, in build_from_cfg
    return obj_cls(**args)
  File "/opt/halodi/halodi-segmentation/halodi_segmentation/models/mmsegmentation/mmseg/models/segmentors/encoder_decoder.py", line 36, in __init__
    self.backbone = builder.build_backbone(backbone)
  File "/opt/halodi/halodi-segmentation/halodi_segmentation/models/mmsegmentation/mmseg/models/builder.py", line 20, in build_backbone
    return BACKBONES.build(cfg)
  File "/opt/halodi/halodi-segmentation/halodi_segmentation/models/mmcv/mmcv/utils/registry.py", line 212, in build
    return self.build_func(*args, **kwargs, registry=self)
  File "/opt/halodi/halodi-segmentation/halodi_segmentation/models/mmcv/mmcv/cnn/builder.py", line 27, in build_model_from_cfg
    return build_from_cfg(cfg, registry, default_args)
  File "/opt/halodi/halodi-segmentation/halodi_segmentation/models/mmcv/mmcv/utils/registry.py", line 44, in build_from_cfg
    raise KeyError(
KeyError: 'SeMaskSwinTransformer is not in the models registry'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "image_demo.py", line 49, in <module>
    main()
  File "image_demo.py", line 36, in main
    model = init_segmentor(args[0], args[-1], device='cuda:0')
  File "/opt/halodi/halodi-segmentation/halodi_segmentation/models/mmsegmentation/mmseg/apis/inference.py", line 32, in init_segmentor
    model = build_segmentor(config.model, test_cfg=config.get('test_cfg'))
  File "/opt/halodi/halodi-segmentation/halodi_segmentation/models/mmsegmentation/mmseg/models/builder.py", line 48, in build_segmentor
    return SEGMENTORS.build(
  File "/opt/halodi/halodi-segmentation/halodi_segmentation/models/mmcv/mmcv/utils/registry.py", line 212, in build
    return self.build_func(*args, **kwargs, registry=self)
  File "/opt/halodi/halodi-segmentation/halodi_segmentation/models/mmcv/mmcv/cnn/builder.py", line 27, in build_model_from_cfg
    return build_from_cfg(cfg, registry, default_args)
  File "/opt/halodi/halodi-segmentation/halodi_segmentation/models/mmcv/mmcv/utils/registry.py", line 55, in build_from_cfg
    raise type(e)(f'{obj_cls.__name__}: {e}')
KeyError: "EncoderDecoder: 'SeMaskSwinTransformer is not in the models registry'"

an99990 commented 2 years ago

Complete log

root@0d2c6aa6ee79:/opt/halodi/halodi-segmentation/halodi_segmentation/models/mmcv# MMCV_WITH_OPS=1 pip install -e .
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Obtaining file:///opt/halodi/halodi-segmentation/halodi_segmentation/models/mmcv
Requirement already satisfied: addict in /opt/conda/lib/python3.8/site-packages (from mmcv-full==1.4.4) (2.4.0)
Requirement already satisfied: numpy in /opt/conda/lib/python3.8/site-packages (from mmcv-full==1.4.4) (1.22.0)
Requirement already satisfied: packaging in /opt/conda/lib/python3.8/site-packages (from mmcv-full==1.4.4) (21.3)
Requirement already satisfied: Pillow in /opt/conda/lib/python3.8/site-packages (from mmcv-full==1.4.4) (8.2.0)
Requirement already satisfied: pyyaml in /opt/conda/lib/python3.8/site-packages (from mmcv-full==1.4.4) (6.0)
Requirement already satisfied: yapf in /opt/conda/lib/python3.8/site-packages (from mmcv-full==1.4.4) (0.32.0)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /opt/conda/lib/python3.8/site-packages (from packaging->mmcv-full==1.4.4) (3.0.6)
Installing collected packages: mmcv-full
  Attempting uninstall: mmcv-full
    Found existing installation: mmcv-full 1.4.4
    Uninstalling mmcv-full-1.4.4:
      Successfully uninstalled mmcv-full-1.4.4
  Running setup.py develop for mmcv-full
Successfully installed mmcv-full-1.4.4
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
root@0d2c6aa6ee79:/opt/halodi/halodi-segmentation/halodi_segmentation/models/mmcv# cd ..
root@0d2c6aa6ee79:/opt/halodi/halodi-segmentation/halodi_segmentation/models# cd mmsegmentation/
root@0d2c6aa6ee79:/opt/halodi/halodi-segmentation/halodi_segmentation/models/mmsegmentation# pip install -e .
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Obtaining file:///opt/halodi/halodi-segmentation/halodi_segmentation/models/mmsegmentation
Requirement already satisfied: matplotlib in /opt/conda/lib/python3.8/site-packages (from mmsegmentation==0.21.1) (3.5.1)
Requirement already satisfied: numpy in /opt/conda/lib/python3.8/site-packages (from mmsegmentation==0.21.1) (1.22.0)
Requirement already satisfied: packaging in /opt/conda/lib/python3.8/site-packages (from mmsegmentation==0.21.1) (21.3)
Requirement already satisfied: prettytable in /opt/conda/lib/python3.8/site-packages (from mmsegmentation==0.21.1) (3.0.0)
Requirement already satisfied: pillow>=6.2.0 in /opt/conda/lib/python3.8/site-packages (from matplotlib->mmsegmentation==0.21.1) (8.2.0)
Requirement already satisfied: python-dateutil>=2.7 in /opt/conda/lib/python3.8/site-packages (from matplotlib->mmsegmentation==0.21.1) (2.8.2)
Requirement already satisfied: fonttools>=4.22.0 in /opt/conda/lib/python3.8/site-packages (from matplotlib->mmsegmentation==0.21.1) (4.29.0)
Requirement already satisfied: cycler>=0.10 in /opt/conda/lib/python3.8/site-packages (from matplotlib->mmsegmentation==0.21.1) (0.11.0)
Requirement already satisfied: kiwisolver>=1.0.1 in /opt/conda/lib/python3.8/site-packages (from matplotlib->mmsegmentation==0.21.1) (1.3.2)
Requirement already satisfied: pyparsing>=2.2.1 in /opt/conda/lib/python3.8/site-packages (from matplotlib->mmsegmentation==0.21.1) (3.0.6)
Requirement already satisfied: six>=1.5 in /opt/conda/lib/python3.8/site-packages (from python-dateutil>=2.7->matplotlib->mmsegmentation==0.21.1) (1.16.0)
Requirement already satisfied: wcwidth in /opt/conda/lib/python3.8/site-packages (from prettytable->mmsegmentation==0.21.1) (0.2.5)
Installing collected packages: mmsegmentation
  Attempting uninstall: mmsegmentation
    Found existing installation: mmsegmentation 0.21.1
    Uninstalling mmsegmentation-0.21.1:
      Successfully uninstalled mmsegmentation-0.21.1
  Running setup.py develop for mmsegmentation
Successfully installed mmsegmentation-0.21.1
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
root@0d2c6aa6ee79:/opt/halodi/halodi-segmentation/halodi_segmentation/models/mmsegmentation# cd ..
root@0d2c6aa6ee79:/opt/halodi/halodi-segmentation/halodi_segmentation/models# cd ..
root@0d2c6aa6ee79:/opt/halodi/halodi-segmentation/halodi_segmentation# python image_demo.py
Traceback (most recent call last):
  File "/opt/halodi/halodi-segmentation/halodi_segmentation/models/mmcv/mmcv/utils/registry.py", line 52, in build_from_cfg
    return obj_cls(**args)
  File "/opt/halodi/halodi-segmentation/halodi_segmentation/models/mmsegmentation/mmseg/models/segmentors/encoder_decoder.py", line 36, in __init__
    self.backbone = builder.build_backbone(backbone)
  File "/opt/halodi/halodi-segmentation/halodi_segmentation/models/mmsegmentation/mmseg/models/builder.py", line 20, in build_backbone
    return BACKBONES.build(cfg)
  File "/opt/halodi/halodi-segmentation/halodi_segmentation/models/mmcv/mmcv/utils/registry.py", line 212, in build
    return self.build_func(*args, **kwargs, registry=self)
  File "/opt/halodi/halodi-segmentation/halodi_segmentation/models/mmcv/mmcv/cnn/builder.py", line 27, in build_model_from_cfg
    return build_from_cfg(cfg, registry, default_args)
  File "/opt/halodi/halodi-segmentation/halodi_segmentation/models/mmcv/mmcv/utils/registry.py", line 44, in build_from_cfg
    raise KeyError(
KeyError: 'SeMaskSwinTransformer is not in the models registry'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "image_demo.py", line 49, in <module>
    main()
  File "image_demo.py", line 36, in main
    model = init_segmentor(args[0], args[-1], device='cuda:0')
  File "/opt/halodi/halodi-segmentation/halodi_segmentation/models/mmsegmentation/mmseg/apis/inference.py", line 32, in init_segmentor
    model = build_segmentor(config.model, test_cfg=config.get('test_cfg'))
  File "/opt/halodi/halodi-segmentation/halodi_segmentation/models/mmsegmentation/mmseg/models/builder.py", line 48, in build_segmentor
    return SEGMENTORS.build(
  File "/opt/halodi/halodi-segmentation/halodi_segmentation/models/mmcv/mmcv/utils/registry.py", line 212, in build
    return self.build_func(*args, **kwargs, registry=self)
  File "/opt/halodi/halodi-segmentation/halodi_segmentation/models/mmcv/mmcv/cnn/builder.py", line 27, in build_model_from_cfg
    return build_from_cfg(cfg, registry, default_args)
  File "/opt/halodi/halodi-segmentation/halodi_segmentation/models/mmcv/mmcv/utils/registry.py", line 55, in build_from_cfg
    raise type(e)(f'{obj_cls.__name__}: {e}')
KeyError: "EncoderDecoder: 'SeMaskSwinTransformer is not in the models registry'"

praeclarumjj3 commented 2 years ago

Wait, you need to build the segmentation inside the SeMask-FPN directory in our repo. We add the code for registering SeMask models in our repo files. It hasn't been incorporated into the original mmsegmentation repo that you seem to be trying to run.

You will have to use the image_demo.py inside our repo and place it inside a folder with the name demo. Then it will work.

an99990 commented 2 years ago

how do i build mmsegmentation inside your SeMask-FPN folder? i see a folder names mmseg, but theres no setup.py ? do i need to build mmcv inside that folder too ? The get_started page in that folder does not work...

praeclarumjj3 commented 2 years ago

Here is the setup.py file: https://github.com/Picsart-AI-Research/SeMask-Segmentation/blob/main/SeMask-FPN/setup.py

You only need to build mmsegmentation. The mmseg contains all the code for different modules. You can build it using the command pip install -e . when your working directory is SeMask-Segmentation/SeMask-FPN.

praeclarumjj3 commented 2 years ago

I have updated the instructions. You may refer to this page: https://github.com/Picsart-AI-Research/SeMask-Segmentation/blob/main/SeMask-FPN/docs/get_started.md#installation.

an99990 commented 2 years ago

it works :) thank you

sahamitul commented 2 years ago

Hi @praeclarumjj3, I am getting this crash:-

python image_demo.py --img test.jpg --config configs/semask_swin/coco_stuff10k/semfpn_semask_swin_large_patch4_window12_640x640_80k_coco10k.py --checkpoint semask_large_fpn_coco10k.pth

Traceback (most recent call last): File "image_demo.py", line 4, in from mmseg.apis import inference_segmentor, init_segmentor, show_result_pyplot File "/nfs/opt-fs-05/home/msaha/segmentation/SeMask-Segmentation/SeMask-FPN/mmseg/apis/init.py", line 1, in from .inference import inference_segmentor, init_segmentor, show_result_pyplot File "/nfs/opt-fs-05/home/msaha/segmentation/SeMask-Segmentation/SeMask-FPN/mmseg/apis/inference.py", line 8, in from mmseg.models import build_segmentor File "/nfs/opt-fs-05/home/msaha/segmentation/SeMask-Segmentation/SeMask-FPN/mmseg/models/init.py", line 1, in from .backbones import # noqa: F401,F403 File "/nfs/opt-fs-05/home/msaha/segmentation/SeMask-Segmentation/SeMask-FPN/mmseg/models/backbones/init.py", line 1, in from .swin_transformer import SwinTransformer File "/nfs/opt-fs-05/home/msaha/segmentation/SeMask-Segmentation/SeMask-FPN/mmseg/models/backbones/swin_transformer.py", line 13, in from timm.models.layers import DropPath, to_2tuple, truncnormal File "/home/msaha/.local/lib/python3.6/site-packages/timm/init.py", line 2, in from .models import create_model, list_models, is_model, list_modules, model_entrypoint, \ File "/home/msaha/.local/lib/python3.6/site-packages/timm/models/init.py", line 2, in from .byoanet import File "/home/msaha/.local/lib/python3.6/site-packages/timm/models/byoanet.py", line 16, in from .byobnet import ByoBlockCfg, ByoModelCfg, ByobNet, interleave_blocks File "/home/msaha/.local/lib/python3.6/site-packages/timm/models/byobnet.py", line 28, in from dataclasses import dataclass, field, replace ModuleNotFoundError: No module named 'dataclasses'

Any clues? Thanks!

praeclarumjj3 commented 2 years ago

It looks like you are using python 3.6. Try using python 3.7: https://forums.fast.ai/t/modulenotfounderror-no-module-named-dataclasses/24499

Picsart-AI-Research / SeMask-Segmentation

Possibility to test on one single image #11