Add segmentation models

[mitmul]

I'd like to add some semantic segmentation models. Possible candidates:

• [ ] Wider or Deeper: Revisiting the ResNet Model for Visual Recognition
  • code: https://github.com/itijyou/ademxapp
• [ ] RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentation
  • code: https://github.com/guosheng/refinenet
• [x] SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation
  • code: https://github.com/alexgkendall/SegNet-Tutorial
• [ ] U-Net: Convolutional Networks for Biomedical Image Segmentation
  • code: https://lmb.informatik.uni-freiburg.de/people/ronneber/u-net/
• [ ] Multi-Scale Context Aggregation by Dilated Convolutions
  • code: https://github.com/fyu/dilation
• [ ] DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
  • code: https://bitbucket.org/aquariusjay/deeplab-public-ver2

[yuyu2172]

Thumbs up on this issue. :+1:

I would prefer a model which we can compare performance to the reported ones.

[23pointsNorth]

Another addition would be

• [ ] Mask R-CNN. code: https://github.com/CharlesShang/FastMaskRCNN

• [ ] FC DenseNet The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation code: https://github.com/SimJeg/FC-DenseNet

[zori]

@23pointsNorth Just to mention Mask R-CNN is for instance segmentation and cannot produce a semantic segmentation. Otherwise, certainly a good to have one!

[souravsingh]

I am interested in working on the issue

[yuyu2172]

Cool! Which model are you interested in?

[souravsingh]

@yuyu2172 I am interested in adding U-Net and FC DenseNet.

[yuyu2172]

Nice. I would love to have those added. It would be nice if the PR has the similar structure as this one https://github.com/chainer/chainercv/pull/757

[kumasento]

Hi @mitmul , sorry for bothering you, but I'm really curious about how to reproduce the result you've got for SegNet on CamVid. I tried the script with its default setting, but the best evaluation mIoU I can get is just 0.3368. Maybe you ran it for longer time than 16k iterations? Thanks!

[mitmul]

@kumasento I haven’t confirmed the reproduction with newer version of ChainerCV code by myself for a long while, so let me check the result using the latest version again. I’ll run this example: https://github.com/chainer/chainercv/tree/master/examples/segnet

[kumasento]

@mitmul Thank you very much! That's the exact script I've run before.

[mitmul]

@kumasento I ran the script and got the following result (I removed log lines containing only training error information):

$ python calc_weight.py
$ MPLBACKEND=Agg python train.py --gpu 0
epoch       iteration   elapsed_time  lr          main/loss   validation/main/miou  validation/main/mean_class_accuracy  validation/main/pixel_accuracy
65          2000        1117.47       0.1         0.187626    0.569284              0.772247                             0.855193
130         4000        2229.86       0.1         0.105407    0.543893              0.701374                             0.855025
196         6000        3340          0.1         0.110691    0.361815              0.59967                              0.650387
261         8000        4448.21       0.1         0.10473     0.500844              0.703283                             0.821186
326         10000       5555.72       0.1         0.341374    0.27762               0.541763                             0.551853
392         12000       6666.44       0.1         0.0865148   0.501126              0.657423                             0.830836
457         14000       7776.4        0.1         0.114381    0.534688              0.750539                             0.835903
523         16000       8884.18       0.1         0.101966    0.588422              0.763433                             0.879876

So, the final mIoU reached to 0.588422 that is somehow better than the results reported in the README of this example code. What versions of Python, Chainer, ChainerCV and CuPy did you use? My environment was:

$ python -V
Python 3.7.4
$ python -c 'import chainer; print(chainer.print_runtime_info())'
Platform: Linux-4.15.0-58-generic-x86_64-with-debian-buster-sid
Chainer: 7.0.0
ChainerX: Not Available
NumPy: 1.17.4
CuPy:
  CuPy Version          : 7.0.0
  CUDA Root             : /usr/local/cuda
  CUDA Build Version    : 10010
  CUDA Driver Version   : 10010
  CUDA Runtime Version  : 10010
  cuBLAS Version        : 10201
  cuFFT Version         : 10101
  cuRAND Version        : 10101
  cuSOLVER Version      : (10, 2, 0)
  cuSPARSE Version      : 10300
  NVRTC Version         : (10, 1)
  cuDNN Build Version   : 7605
  cuDNN Version         : 7605
  NCCL Build Version    : 2402
  NCCL Runtime Version  : 2402
iDeep: Not Available

Could you also give me the log?

[kumasento]

Thanks for your update! I think the result you’ve gotten is validation accuracy, but the one listed in the table means test accuracy, if I understand correctly. You can use the evaluation script to evaluate this snapshot to give a double check :)

On Thu, 9 Jan 2020 at 01:20, Shunta Saito notifications@github.com wrote:

@kumasento https://github.com/kumasento I ran the script and got the following result (I removed log lines containing only training error information):

$ python calc_weight.py $ MPLBACKEND=Agg python train.py --gpu 0 epoch iteration elapsed_time lr main/loss validation/main/miou validation/main/mean_class_accuracy validation/main/pixel_accuracy 65 2000 1117.47 0.1 0.187626 0.569284 0.772247 0.855193 130 4000 2229.86 0.1 0.105407 0.543893 0.701374 0.855025 196 6000 3340 0.1 0.110691 0.361815 0.59967 0.650387 261 8000 4448.21 0.1 0.10473 0.500844 0.703283 0.821186 326 10000 5555.72 0.1 0.341374 0.27762 0.541763 0.551853 392 12000 6666.44 0.1 0.0865148 0.501126 0.657423 0.830836 457 14000 7776.4 0.1 0.114381 0.534688 0.750539 0.835903 523 16000 8884.18 0.1 0.101966 0.588422 0.763433 0.879876

So, the final mIoU reached to 0.588422 that is somehow better than the results reported in the README of this example code https://github.com/chainer/chainercv/blob/master/examples/segnet/README.md. What versions of Python, Chainer, ChainerCV and CuPy did you use? My environment was:

$ python -V Python 3.7.4 $ python -c 'import chainer; print(chainer.print_runtime_info())' Platform: Linux-4.15.0-58-generic-x86_64-with-debian-buster-sid Chainer: 7.0.0 ChainerX: Not Available NumPy: 1.17.4 CuPy: CuPy Version : 7.0.0 CUDA Root : /usr/local/cuda CUDA Build Version : 10010 CUDA Driver Version : 10010 CUDA Runtime Version : 10010 cuBLAS Version : 10201 cuFFT Version : 10101 cuRAND Version : 10101 cuSOLVER Version : (10, 2, 0) cuSPARSE Version : 10300 NVRTC Version : (10, 1) cuDNN Build Version : 7605 cuDNN Version : 7605 NCCL Build Version : 2402 NCCL Runtime Version : 2402 iDeep: Not Available

Could you also give me the log?

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/chainer/chainercv/issues/158?email_source=notifications&email_token=ACC42R3MJHCXKNV6VICAA2DQ4Z3WRA5CNFSM4DKPZMA2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEIOSRXQ#issuecomment-572336350, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACC42R6NXVQUOUITPKSYVMLQ4Z3WRANCNFSM4DKPZMAQ .

[mitmul]

@kumasento OK, thanks for the guide. I'll run the evaluation on the test dataset. Please give me some more time.

[mitmul]

@kumasento I ran the evaluation on the test dataset of CamVid with this script:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import argparse
import json

from chainer import iterators
from chainer.dataset import concat_examples
from chainer.serializers import load_npz
from chainercv.datasets import CamVidDataset, camvid_label_names
from chainercv.evaluations import eval_semantic_segmentation
from chainercv.links import SegNetBasic
from chainercv.utils import apply_to_iterator

if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument('--snapshot', type=str,
                        default='result/snapshot_model.npz')
    parser.add_argument('--batchsize', type=int, default=12)
    parser.add_argument('--gpu', type=int, default=0)
    args = parser.parse_args()

    test = CamVidDataset(split='test')
    test_iter = iterators.SerialIterator(
        test, args.batchsize, shuffle=False, repeat=False)

    model = SegNetBasic(n_class=len(camvid_label_names))
    load_npz(args.snapshot, model)
    model = model.to_gpu(args.gpu)

    in_values, out_values, rest_values = apply_to_iterator(
        model.predict, test_iter)

    del in_values

    pred_labels, = out_values
    gt_labels, = rest_values

    result = eval_semantic_segmentation(pred_labels, gt_labels)

    print(f"miou: {result['miou']}")

and got miou: 0.4834565081914471 that is actually worse than the reported value in the README (49.4 %) but better than the original value of 46.3 %. I used this snapshot: snapshot_model.npz.zip

So, from my experiments, it seems not to succeed to reproduce your result with miou: 0.3368. Could you also let me know the information about your environment?

[kumasento]

@mitmul Thanks! I can achieve the same accuracy a you did with the same snapshot. I will rerun the training and see whether I can get the same level of accuracy.

[kumasento]

BTW my environment:

$ python -V
Python 3.7.3
$ python -c 'import chainer; print(chainer.print_runtime_info())'
Platform: Linux-4.4.0-142-generic-x86_64-with-debian-stretch-sid
Chainer: 6.3.0
NumPy: 1.16.4
CuPy:
  CuPy Version          : 6.3.0
  CUDA Root             : /usr/local/cuda-10.1
  CUDA Build Version    : 10010
  CUDA Driver Version   : 10010
  CUDA Runtime Version  : 10010
  cuDNN Build Version   : 7602
  cuDNN Version         : 7602
  NCCL Build Version    : 2402
  NCCL Runtime Version  : 2402
iDeep: Not Available
None

[kumasento]

@mitmul After running again and using the snapshot that has the highest validation accuracy, I can now get a good test accuracy :) Thanks

[mitmul]

@kumasento Good!