search

ISCAS007 / torchseg

use pytorch to do image semantic segmentation
GNU General Public License v3.0
7 stars 1 forks source link

2018-08-11 edge + the-state-of-art #9

Open yzbx opened 6 years ago

yzbx commented 6 years ago

miou

Network backbone note voc cityscapes ade20k
deeplabv3+ resnet50 - 64-74 - -
deeplabv3+ resnet101 - 68-75 - -
deeplabv3+ resnet101 OS=16 76.66 77.23 -
deeplabv3+ resnet101 OS=8 78.05 77.82 -
deeplabv3+ resnet101 OS=8,MS,Flip 79.35 79.30 -
deeplabv3+ resnet101 coarse - 81.3 -
deeplabv3+ resnet101 crop size=321 67.22 - -
deeplabv3+ resnet101 crop size=513 77.21 - -
deeplabv3+ resnet101 batch size=4 64.43 - -
deeplabv3+ resnet101 batch size=8 75.76 - -
deeplabv3+ resnet101 batch size=12 76.49 - -
deeplabv3+ resnet101 batch size=16 77.21 - -
deeplabv3+ resnet101 duplicate hard classes 85.7 - -
Network backbone note voc cityscapes ade20k
pspnet resnet50 - -
pspnet resnet101 - 82.6 80.2 43.39
pspnet resnet152 - -

multi-scale {0.5,0.8,1.0,1.2,1.5,2.0} testing will help denseASPP miou to 80.6%

Network backbone cityscapes
denseASPP densenet121 76.2
denseASPP densenet169 77.7
denseASPP densenet201 78.9
yzbx commented 6 years ago

https://www.slideshare.net/mitmul/unofficial-pyramid-scene-parsing-network-cvpr-2017

yzbx commented 6 years ago

add regulator l1 and l2 loss

python test/pspnet_test.py --batch_size=4 --optimizer=sgd --learning_rate=0.01 --midnet_name=aspp --backbone_pretrained=True --note='reg'

sgd optimizer + poly scheduler + psp midnet

python test/pspnet_test.py --net_name=pspnet --backbone_name=resnet50 --backbone_pretrained=True --midnet_name=psp --midnet_scale=10 --optimizer=sgd --learning_rate=0.01 --note=sgd
yzbx commented 6 years ago

tensorflow

dataset accuracy
cityscapes 77%
ade20k 40%

pytorch

yzbx commented 6 years ago

abnormal object detection

https://github.com/ISCAS007/keras-yolo3

  1. download and convert weights
  2. demo video and target area, target object detection 
    python yolo_video.py --input=/home/yzbx/Videos/sherbrooke_video.avi
yzbx commented 6 years ago

tensorflow example

yzbx commented 6 years ago

problem

we need use edge, convert the label from tfrecord or from tensor

if use tensor

  1. we need use opencv lib to do edge detection and dilate
  2. opencv only support numpy array
  3. need convert data from tensor to numpy array
  4. need use tf.Session() to accomplish the convert
  5. cannot use default session in tensorflow, no interface found for tf.Session() if use tfrecord
  6. not support dynamic edge_width

ways

  1. [x] convert tensorflow model and weight to pytorch
  2. remove distributed system support, supervisitor support.

convert to pytorch


(new) ➜  train mmconvert -sf tensorflow -in model.ckpt-0.meta -iw model.ckpt-0 --dstNode ResizeBilinear_3 -df pytorch -om tf_to_pytorch.pth
/home/yzbx/bin/miniconda3/envs/new/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
from ._conv import register_converters as _register_converters
Parse file [model.ckpt-0.meta] with binary format successfully.
Tensorflow model file [model.ckpt-0.meta] loaded successfully.
Tensorflow checkpoint file [model.ckpt-0] loaded successfully. [1173] variables loaded.
Tensorflow has not supported operator [NoOp] with name [fifo_queue_Dequeue:1].
Tensorflow has not supported operator [QueueDequeueV2] with name [fifo_queue_Dequeue].
Traceback (most recent call last):
File "/home/yzbx/bin/miniconda3/envs/new/bin/mmconvert", line 11, in <module>
sys.exit(_main())
File "/home/yzbx/bin/miniconda3/envs/new/lib/python3.6/site-packages/mmdnn/conversion/_script/convert.py", line 102, in _main
ret = convertToIR._convert(ir_args)
File "/home/yzbx/bin/miniconda3/envs/new/lib/python3.6/site-packages/mmdnn/conversion/_script/convertToIR.py", line 115, in _convert
parser.run(args.dstPath)
File "/home/yzbx/bin/miniconda3/envs/new/lib/python3.6/site-packages/mmdnn/conversion/common/DataStructure/parser.py", line 22, in run
self.gen_IR()
File "/home/yzbx/bin/miniconda3/envs/new/lib/python3.6/site-packages/mmdnn/conversion/tensorflow/tensorflow_parser.py", line 309, in gen_IR
func(current_node)
File "/home/yzbx/bin/miniconda3/envs/new/lib/python3.6/site-packages/mmdnn/conversion/tensorflow/tensorflow_parser.py", line 661, in rename_FusedBatchNorm
self.set_weight(source_node.name, 'mean', self.ckpt_data[mean.name])
AttributeError: 'NoneType' object has no attribute 'name'
yzbx commented 6 years ago

problem

  1. use official tensorflow preprocessing
  2. use feed_dict
  3. cannot convert a tensor to tensor
        for epoch in range(epoches):
            for i, (images, labels, edges) in enumerate(data_loader):
                tf_images_4d,tf_labels_4d=batch_preprocess_image_and_label(images.numpy(),labels.numpy(),FLAGS,ignore_label,is_training=True)
#                tf_labels_4d = tf.expand_dims(tf_labels_3d, axis=-1)

                print(tf_images_4d.shape,tf_labels_4d)
                sess.run(fetches=[optimizer.minimize(total_loss), total_loss], feed_dict={
                         images: tf_images_4d, labels: tf_labels_4d})

ways

  1. use numpy preprocessing
  2. find a way to feed with tensorflow tensor
  3. input numpy data list, feed to network, preprocess with tf function, then stack them and run session
yzbx commented 6 years ago

todo

  1. FailedPreconditionError (see above for traceback): Attempting to use uninitialized value logits/semantic/biases/Momentum
  2. arbitrary input size
  3. edge preprocess
  4. voc, ade20k dataset suport