This repository implements Semantic Instance Segmentation with a Discriminative Loss Function with some enhancements.
In prediction phase, network inputs an image and outputs a semantic segmentation mask, the number of instances and embeddings for all pixels in the image. Then, foreground embeddings (which correspond to instances) are selected using semantic segmentation mask and foreground embeddings are clustered into "the number of instances" groups via clustering.
git clone --recursive https://github.com/Wizaron/instance-segmentation-pytorch.git
sudo apt install imagemagick
conda env create -f instance-segmentation-pytorch/code/conda_environment.yml
CVPPP2017_LSC_training.zip
) to instance-segmentation-pytorch/data/raw/CVPPP/
processed/CVPPP/semantic-annotations/
and processed/CVPPP/instance-annotations
.imagemagick
should be installed.).metadata/CVPPP/image_shapes.txt
.metadata/CVPPP/number_of_instances.txt
.metadata/CVPPP/training_image_paths.txt
, metadata/CVPPP/validation_image_paths.txt
processed/CVPPP/lmdb/
.
Data should be prepared prior to training and evaluation.
source activate ins-seg-pytorch
or conda activate ins-seg-pytorch
instance-segmentation-pytorch/data/raw/CVPPP/
. Hence, raw dataset should be found at instance-segmentation-pytorch/data/raw/CVPPP/CVPPP2017_LSC_training/
.instance-segmentation-pytorch/data/scripts/CVPPP/
and run sh prepare.sh
.Start a Visdom server in a screen
or tmux
.
Activate previously created conda environment : source activate ins-seg-pytorch
or conda activate ins-seg-pytorch
Start visdom server : python -m visdom.server
We can access visdom server using http://localhost:8097
Activate previously created conda environment : source activate ins-seg-pytorch
or conda activate ins-seg-pytorch
Go to instance-segmentation-pytorch/code/
and run train.py
.
usage: train.py [-h] [--model MODEL] [--usegpu] [--nepochs NEPOCHS]
[--batchsize BATCHSIZE] [--debug] [--nworkers NWORKERS]
--dataset DATASET
optional arguments:
-h, --help show this help message and exit
--model MODEL Filepath of trained model (to continue training)
[Default: '']
--usegpu Enables cuda to train on gpu [Default: False]
--nepochs NEPOCHS Number of epochs to train for [Default: 600]
--batchsize BATCHSIZE
Batch size [Default: 2]
--debug Activates debug mode [Default: False]
--nworkers NWORKERS Number of workers for data loading (0 to do it using
main process) [Default : 2]
--dataset DATASET Name of the dataset which is "CVPPP"
Debug mode plots pixel embeddings to visdom, it reduces size of the embeddings to two-dimensions using TSNE. Hence, it slows training down.
As training continues, models are saved to instance-segmentation-pytorch/models/CVPPP
.
After training is completed, we can make predictions.
Activate previously created conda environment : source activate ins-seg-pytorch
or conda activate ins-seg-pytorch
Go to instance-segmentation-pytorch/code/
.
Run pred_list.py
.
usage: pred_list.py [-h] --lst LST --model MODEL [--usegpu]
[--n_workers N_WORKERS] --dataset DATASET
optional arguments:
-h, --help show this help message and exit
--lst LST Text file that contains image paths
--model MODEL Path of the model
--usegpu Enables cuda to predict on gpu
--dataset DATASET Name of the dataset which is "CVPPP"
For example: python pred_list.py --lst ../data/metadata/CVPPP/validation_image_paths.txt --model ../models/CVPPP/2018-3-4_16-15_jcmaxwell_29-937494/model_155_0.123682662845.pth --usegpu --n_workers 4 --dataset CVPPP
outputs
directory.evaluate.py
. It prints output metrics to the stdout.usage: evaluate.py [-h] --pred_dir PRED_DIR --dataset DATASET
optional arguments:
-h, --help show this help message and exit
--pred_dir PRED_DIR Prediction directory
--dataset DATASET Name of the dataset which is "CVPPP"
For example: python evaluate.py --pred_dir ../outputs/CVPPP/2018-3-4_16-15_jcmaxwell_29-937494-model_155_0.123682662845/validation/ --dataset CVPPP
After training is complete, we can make predictions. We can use pred.py
to make predictions for a single image.
Activate previously created conda environment : source activate ins-seg-pytorch
or conda activate ins-seg-pytorch
Go to instance-segmentation-pytorch/code/
.
Run pred.py
.
usage: pred.py [-h] --image IMAGE --model MODEL [--usegpu] --output OUTPUT
[--n_workers N_WORKERS] --dataset DATASET
optional arguments:
-h, --help show this help message and exit
--image IMAGE Path of the image
--model MODEL Path of the model
--usegpu Enables cuda to predict on gpu
--output OUTPUT Path of the output directory
--dataset DATASET Name of the dataset which is "CVPPP"
SBD | \ | DiC\ | Foreground Dice | |
---|---|---|---|---|
87.9 | 0.5 | 96.8 |