Use standalone segmentors

[claverru]

Hello,

First of all, thank you for your great work.

I want to migrate from another semantic segmentation framework that I used integrated with Pytorch Lightning for the training process. I've been diving into the code and I see that models and training process are quite coupled.

I wonder what is the best practice if I want to use your models implementation and keep using Pytorch Lightning's training engine. This would bring me your models' implementation power and at the same time I would keep the flexibility that Pytorch Lightning brings me.

More concretely, to instantiate a Segmentor, I need to pass backbone, head and neck configs, which makes it weird to use as a standalone module. Moreover because in the config I need to specify training and test information which I won't use.

In this issue https://github.com/open-mmlab/mmsegmentation/issues/2408 they don't use the Segmentor class. It's the cleanest way I've found so far but still I would have to re-implement some of the functions written in the Segmentor class.

Thank you,

Claudio

[xiexinch]

Hi @claverru, The use of mmseg as a package is also our most recommended usage, currently instantiating a model necessarily requires a configuration, which can be a dict in the code or a config file. I think the simplest way to do it is as follows:

from mmseg.registry import MODELS

model_cfg = dict(
    type='EncoderDecoder', 
    backbone=dict(type='xxx', ...),
    decode_head=dict(type='xxx', ...))
model = MODELS.build(model_cfg)

If you would like to make your code clean, might use the config file:

from mmengine import Config
from mmseg.registry import MODELS

cfg_path = 'xxx.py'
model = MODELS.build(Config.from_file(cfg_path).model)

config.py:

model = dict(
    type='EncoderDecoder', 
    backbone=dict(type='xxx', ...),
    decode_head=dict(type='xxx', ...))

[claverru]

Hello @xiexinch , thanks for your response.

I guess it makes sense. What about if I want to use your preconfigurations (pretrained models)? I had this:

from mmseg.apis import MMSegInferencer

model = MMSegInferencer(
    model="unet-s5-d16_fcn_4xb4-ce-1.0-dice-3.0-40k_chase-db1-128x128"
).model

Config can only receive a real path to a file, making it hard to reuse your preconfigurations. I was walking around your code and I found MMSegInferencer resolved the full path of a config with only the name. Otherwise I guess I have to clone the full repo or resolve the paths under bin/ myself.

[xiexinch]

Hello @xiexinch , thanks for your response.

I guess it makes sense. What about if I want to use your preconfigurations (pretrained models)? I had this:

from mmseg.apis import MMSegInferencer

model = MMSegInferencer(
    model="unet-s5-d16_fcn_4xb4-ce-1.0-dice-3.0-40k_chase-db1-128x128"
).model

Config can only receive a real path to a file, making it hard to reuse your preconfigurations. I was walking around your code and I found MMSegInferencer resolved the full path of a config with only the name. Otherwise I guess I have to clone the full repo or resolve the paths under bin/ myself.

Using MMSegInferencer to build a model is also a solution. If only setting the model name to the API, it will automatically download the pre-trained weights. To find the correct model name, you might install the CLI tool mim to help you. Or you can find them at configs/xxx/metafile.yaml.

pip install openmim
mim search mmsegmentation

https://github.com/open-mmlab/mmsegmentation/blob/main/configs/ann/metafile.yaml

[claverru]

Thanks, @xiexinch

What about if I wanted to finetune one of those models with a different number of classes? Any idea on how I can change the number of output channels and reset those weights in a smart way that can be applied for every segmentor?