About the video data

[LeslieZhoa]

Thanks for the repo！It works well! I want to know how I can use the video data when the mode is predict.

[aserchen]

Hi ,@LeslieZhoa . I want to use the model on the video data ,too .But I trained it on VOC2012 semantic segmentation dataset and only get mIOU = 0.46 without pretrained model. Although I change the parameters '--batch_size' and '--weight_decay' ,the mIOU values are so terrible, too. My GPU is Titan Xp .How about you?

[LeslieZhoa]

Hi, @aserchen , I am glad for your reply. My model is still training and I add BN layer in the ASPP . Until now, the train_mIOU is about 0.45 and the global_step is only 5056. I not use the pretrained model beacause I modified the model. For the video data, I have a very stupid code as follows. You can refer to it. I will put it on my github soon, but it is annotated in Chinese. `

# coding: utf-8
  import os
  import tensorflow as tf
  from utils import config as FLAGS
  from utils import deeplab_model,preprocessing
  import numpy as np
  import cv2

  def main():
      image=tf.placeholder(tf.float32,[None,None,3])
      inputs=preprocessing.mean_image_subtraction(image)
      inputs=tf.expand_dims(inputs,axis=0)
      model=deeplab_model.model_generator(FLAGS.num_classes,
                                       FLAGS.output_stride,
                                       FLAGS.base_architecture,
                                       FLAGS.pre_trained_model,
                                       None,)
      logits=model(inputs,False)

      pred_classes=tf.expand_dims(tf.argmax(logits,axis=3,output_type=tf.int32),axis=3)

      pred_decoded_labels=tf.py_func(preprocessing.decode_labels,
                                  [pred_classes,1,FLAGS.num_classes],
                                  tf.uint8)
      pred_decoded_labels=tf.squeeze(pred_decoded_labels)
      saver=tf.train.Saver()
      sess=tf.Session()
      model_file=tf.train.latest_checkpoint(FLAGS.model_dir)
      saver.restore(sess,model_file)
      if FLAGS.test_mode=='1':
          for filename in os.listdir(FLAGS.pictue):
              x=cv2.imread(FLAGS.pictue+filename)
              x=cv2.cvtColor(x,cv2.COLOR_BGR2RGB)
              out=sess.run(pred_decoded_labels,feed_dict={image:x})
              out=cv2.cvtColor(out,cv2.COLOR_RGB2BGR)
              cv2.imshow('im',out)
              cv2.imwrite(FLAGS.output + filename,out)
          cv2.waitKey(0)
          cv2.destroyAllWindows()

      if FLAGS.test_mode=='2':
          cap=cv2.VideoCapture(0)
          fourcc = cv2.VideoWriter_fourcc(*'XVID')
          out = cv2.VideoWriter(FLAGS.output+'out.mp4' ,fourcc,10,(1280,480))
          while True:
              ret,frame = cap.read()
              if ret == True:
                  frame1=cv2.cvtColor(frame,cv2.COLOR_BGR2RGB)
                  result=sess.run(pred_decoded_labels,feed_dict={image:frame1})
                  result=cv2.cvtColor(result,cv2.COLOR_RGB2BGR)
                  result1=np.concatenate([frame, result], axis=1)
                  a = out.write(result1)
                  cv2.imshow("result", result1)
                  if cv2.waitKey(1) & 0xFF == ord('q'):
                      break
              else:
                  break
          cap.release()
          out.release()
          cv2.destroyAllWindows()
      sess.close()

  if __name__=='__main__':
      main()

`

[DRACOyu]

@LeslieZhoa hello, I meet the same question，as follows: Key Variable not found in checkpoint [[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_INT32], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]] I removed the tf.train.init_checkpoint,this problem still remains, I use inceptionV3 as the baseline, from your answer, checkpoint needs to be updated as well.so how to update the checkpoint? hope your answer,thanks

[LeslieZhoa]

Hi @DRACOyu , I am glad for your question. Do you make sure the checkpoint of the model is in FLAG.model_dir? The model_dir is not the resnet model. And I put the whole code in my project. But it is annotated in Chinese. I hope I can solve your question.

[chenhongyun123]

@LeslieZhoa Hello,i try to run your code,because you say you add BN layer in the ASPP ,but failed.I meet some question ,as follows" INFO:tensorflow:Saving checkpoints for 0 into ./model\model.ckpt.".So i want to know if you meet this question.And how can i solve it.

[LeslieZhoa]

Hi @chenhongyun123 what you put is not error . Could you put all the log?

[DRACOyu]

BN的参数默认是不添加到模型中的，需要在代码里要添加保存均值和方差的部分

------------------ 原始邮件 ------------------ 发件人: "chenhongyun123"notifications@github.com; 发送时间: 2019年8月8日(星期四) 晚上10:35 收件人: "rishizek/tensorflow-deeplab-v3-plus"tensorflow-deeplab-v3-plus@noreply.github.com; 抄送: "1254549726"1254549726@qq.com;"Mention"mention@noreply.github.com; 主题: Re: [rishizek/tensorflow-deeplab-v3-plus] About the video data (#34)

@LeslieZhoa Hello,i try to run your code,because you say you add BN layer in the ASPP ,but failed.I meet some question ,as follows" INFO:tensorflow:Saving checkpoints for 0 into ./model\model.ckpt.".So i want to know if you meet this question.And how can i solve it.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[chenhongyun123]

I am sorry. All the log is follow. D:\gitlab\tensorflow-deeplab_v3_plus，加入BN>python train.py d:\anaconda3\lib\site-packages\tensorflow\python\framework\dtypes.py:523: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecate d; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint8 = np.dtype([("qint8", np.int8, 1)]) d:\anaconda3\lib\site-packages\tensorflow\python\framework\dtypes.py:524: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecate d; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_quint8 = np.dtype([("quint8", np.uint8, 1)]) d:\anaconda3\lib\site-packages\tensorflow\python\framework\dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecate d; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint16 = np.dtype([("qint16", np.int16, 1)]) d:\anaconda3\lib\site-packages\tensorflow\python\framework\dtypes.py:526: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecate d; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_quint16 = np.dtype([("quint16", np.uint16, 1)]) d:\anaconda3\lib\site-packages\tensorflow\python\framework\dtypes.py:527: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecate d; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint32 = np.dtype([("qint32", np.int32, 1)]) d:\anaconda3\lib\site-packages\tensorflow\python\framework\dtypes.py:532: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecate d; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. np_resource = np.dtype([("resource", np.ubyte, 1)]) d:\anaconda3\lib\site-packages\h5py__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from float to np.floating is dep recated. In future, it will be treated as np.float64 == np.dtype(float).type. from ._conv import register_converters as _register_converters INFO:tensorflow:Using config: {'_model_dir': './model', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_ch eckpoints_secs': 100.0, '_session_config': device_count { key: "GPU" value: 1 } , '_keep_checkpoint_max': 3, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_pr otocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_service': None, '_cluster_spec': <tensorflow.python.training.server_lib.Cl usterSpec object at 0x0000015D006F2B00>, '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluationmaster': '', ' is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1} INFO:tensorflow:开始训练里奥 INFO:tensorflow:Calling model_fn. WARNING:tensorflow:From D:\gitlab\tensorflow-deeplab_v3_plus，加入BN\utils\deeplab_model.py:306: calling reduce_mean (from tensorflow.python.ops.math_o ps) with keep_dims is deprecated and will be removed in a future version. Instructions for updating: keep_dims is deprecated, use keepdims instead INFO:tensorflow:Done calling model_fn. INFO:tensorflow:Create CheckpointSaverHook. INFO:tensorflow:Graph was finalized. 2019-08-08 23:04:13.899606: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not com piled to use: AVX AVX2 2019-08-08 23:04:14.493198: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1432] Found device 0 with properties: name: GeForce 940M major: 5 minor: 0 memoryClockRate(GHz): 1.176 pciBusID: 0000:02:00.0 totalMemory: 2.00GiB freeMemory: 1.66GiB 2019-08-08 23:04:14.510420: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1511] Adding visible gpu devices: 0 2019-08-08 23:04:17.967135: I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] Device interconnect StreamExecutor with strength 1 edge matrix: 2019-08-08 23:04:17.977796: I tensorflow/core/common_runtime/gpu/gpu_device.cc:988] 0 2019-08-08 23:04:17.980434: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1001] 0: N 2019-08-08 23:04:17.984728: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device: GPU:0 with 1408 MB memory) -> physical GPU (device: 0, name: GeForce 940M, pci bus id: 0000:02:00.0, compute capability: 5.0) INFO:tensorflow:Restoring parameters from ./model\model.ckpt-0 INFO:tensorflow:Running local_init_op. INFO:tensorflow:Done running local_init_op. INFO:tensorflow:Saving checkpoints for 0 into ./model\model.ckpt.

------------------ 原始邮件 ------------------ 发件人: "LeslieZhao"notifications@github.com; 发送时间: 2019年8月8日(星期四) 晚上10:42 收件人: "rishizek/tensorflow-deeplab-v3-plus"tensorflow-deeplab-v3-plus@noreply.github.com; 抄送: "丶Fat cloud"377199273@qq.com;"Mention"mention@noreply.github.com; 主题: Re: [rishizek/tensorflow-deeplab-v3-plus] About the video data (#34)

Hi @chenhongyun123 what you put is not error . Could you put all the log?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[chenhongyun123]

不是很明白，如果我想在ASPP中添加BN，要如何实现呢？

------------------ 原始邮件 ------------------ 发件人: "DRACOyu"notifications@github.com; 发送时间: 2019年8月8日(星期四) 晚上10:44 收件人: "rishizek/tensorflow-deeplab-v3-plus"tensorflow-deeplab-v3-plus@noreply.github.com; 抄送: "丶Fat cloud"377199273@qq.com;"Mention"mention@noreply.github.com; 主题: Re: [rishizek/tensorflow-deeplab-v3-plus] About the video data (#34)

BN的参数默认是不添加到模型中的，需要在代码里要添加保存均值和方差的部分

------------------ 原始邮件 ------------------ 发件人: "chenhongyun123"notifications@github.com; 发送时间: 2019年8月8日(星期四) 晚上10:35 收件人: "rishizek/tensorflow-deeplab-v3-plus"tensorflow-deeplab-v3-plus@noreply.github.com; 抄送: "1254549726"1254549726@qq.com;"Mention"mention@noreply.github.com; 主题: Re: [rishizek/tensorflow-deeplab-v3-plus] About the video data (#34)

@LeslieZhoa Hello,i try to run your code,because you say you add BN layer in the ASPP ,but failed.I meet some question ,as follows" INFO:tensorflow:Saving checkpoints for 0 into ./model\model.ckpt.".So i want to know if you meet this question.And how can i solve it.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[chenhongyun123]

@LeslieZhoaI am very sorry, the last problem was solved. It was due to the problem of the GPU of my own computer. I changed the value of batch_size and now it runs.

[chenhongyun123]

@LeslieZhoa Thanks for your project.