FCN.py 디버깅 : pdb.set_trace()

FCN.py 실행할때 에러났던거 main()함수 정의한거 안에 있는 sess.run(train_step, feed_dict=feed_dict) 이부분 때문인듯함. 아래는 디버깅 결과, feed_dict 확인해보니까 에러는 input이미지에서 난 거 같음. sess.run자체의 문제는 아닌거같고 train_step이 잘못된거로 보임, 더 해보고 댓글로 추가함

참고로 **** Epochs completed: 1***** 는 main()함수의 train_images, train_annotations = train_dataset_reader.next_batch(FLAGS.batch_size)코드가 끝나면 찍히는거로 확인됨(왜지?)

디버깅 결과

(중략) 2021-05-09 20:09:54.230568: W tensorflow/core/framework/allocator.cc:107] Allocation of 411041792 exceeds 10% of system memory. 2021-05-09 20:09:54.430081: W tensorflow/core/framework/allocator.cc:107] Allocation of 411041792 exceeds 10% of system memory. 2021-05-09 20:09:55.258454: W tensorflow/core/framework/allocator.cc:107] Allocation of 411041792 exceeds 10% of system memory. ** Epochs completed: 1** c:\users\user_\pycharmprojects\capstoneproject2_test\fcn.py(212)main() -> sess.run(train_step, feed_dict=feed_dict) (Pdb) n ValueError: Cannot feed value of shape (0,) for Tensor 'input_image:0', which has shape '(?, 224, 224, 3)' c:\users\user_\pycharmprojects\capstoneproject2_test\fcn.py(212)main() ->sess.run(train_step, feed_dict=feed_dict) (Pdb) print(feed_dict) {<tf.Tensor 'input_image:0' shape=(?, 224, 224, 3) dtype=float32>: array([], dtype=float64), <tf.Tensor 'annotation:0' shape=(?, 224, 224, 1) dtype=int32>: array([], dtype=float64), <tf.Tensor 'keep_probabilty:0' shape= dtype=float32>: 0.85}

이건 sess.run 돌리기 전에 trainstep 찍어본 결과임, 뭔소린지는 모르겠음 c:\users\user\pycharmprojects\capstoneproject2_test\fcn.py(212)main() -> sess.run(train_step, feed_dict=feed_dict) (Pdb) print(train_step) name: "Adam" op: "NoOp" input: "^Adam/update_inference/conv1_1_w/ApplyAdam" input: "^Adam/update_inference/conv1_1_b/ApplyAdam" input: "^Adam/update_inference/conv1_2_w/ApplyAdam" input: "^Adam/update_inference/conv1_2_b/ApplyAdam" input: "^Adam/update_inference/conv2_1_w/ApplyAdam" input: "^Adam/update_inference/conv2_1_b/ApplyAdam" input: "^Adam/update_inference/conv2_2_w/ApplyAdam" input: "^Adam/update_inference/conv2_2_b/ApplyAdam" input: "^Adam/update_inference/conv3_1_w/ApplyAdam" input: "^Adam/update_inference/conv3_1_b/ApplyAdam" input: "^Adam/update_inference/conv3_2_w/ApplyAdam" input: "^Adam/update_inference/conv3_2_b/ApplyAdam" input: "^Adam/update_inference/conv3_3_w/ApplyAdam" input: "^Adam/update_inference/conv3_3_b/ApplyAdam" input: "^Adam/update_inference/conv3_4_w/ApplyAdam" input: "^Adam/update_inference/conv3_4_b/ApplyAdam" input: "^Adam/update_inference/conv4_1_w/ApplyAdam" input: "^Adam/update_inference/conv4_1_b/ApplyAdam" input: "^Adam/update_inference/conv4_2_w/ApplyAdam" input: "^Adam/update_inference/conv4_2_b/ApplyAdam" input: "^Adam/update_inference/conv4_3_w/ApplyAdam" input: "^Adam/update_inference/conv4_3_b/ApplyAdam" input: "^Adam/update_inference/conv4_4_w/ApplyAdam" input: "^Adam/update_inference/conv4_4_b/ApplyAdam" input: "^Adam/update_inference/conv5_1_w/ApplyAdam" input: "^Adam/update_inference/conv5_1_b/ApplyAdam" input: "^Adam/update_inference/conv5_2_w/ApplyAdam" input: "^Adam/update_inference/conv5_2_b/ApplyAdam" input: "^Adam/update_inference/conv5_3_w/ApplyAdam" input: "^Adam/update_inference/conv5_3_b/ApplyAdam" input: "^Adam/update_inference/W6/ApplyAdam" input: "^Adam/update_inference/b6/ApplyAdam" input: "^Adam/update_inference/W7/ApplyAdam" input: "^Adam/update_inference/b7/ApplyAdam" input: "^Adam/update_inference/W8/ApplyAdam" input: "^Adam/update_inference/b8/ApplyAdam" input: "^Adam/update_inference/W_t1/ApplyAdam" input: "^Adam/update_inference/b_t1/ApplyAdam" input: "^Adam/update_inference/W_t2/ApplyAdam" input: "^Adam/update_inference/b_t2/ApplyAdam" input: "^Adam/update_inference/W_t3/ApplyAdam" input: "^Adam/update_inference/b_t3/ApplyAdam" input: "^Adam/Assign" input: "^Adam/Assign_1"

def main(argv=None):
  pdb.set_trace()
  # 인풋 이미지와 타겟 이미지, 드롭아웃 확률을 받을 플레이스홀더를 정의합니다.
  keep_probability = tf.placeholder(tf.float32, name="keep_probabilty")
  image = tf.placeholder(tf.float32, shape=[None, IMAGE_SIZE, IMAGE_SIZE, 3], name="input_image")
  annotation = tf.placeholder(tf.int32, shape=[None, IMAGE_SIZE, IMAGE_SIZE, 1], name="annotation")

  # FCN 그래프를 선언하고 TensorBoard를 위한 summary들을 지정합니다.
  pred_annotation, logits = inference(image, keep_probability)
  tf.summary.image("input_image", image, max_outputs=2)
  tf.summary.image("ground_truth", tf.cast(annotation, tf.uint8), max_outputs=2)
  tf.summary.image("pred_annotation", tf.cast(pred_annotation, tf.uint8), max_outputs=2)

main()함수의 해당 부분 디버깅

(중략) py, it will be understood as (type, (1,)) / '(1,)type'. np_resource = np.dtype([("resource", np.ubyte, 1)]) WARNING:tensorflow:From FCN.py:247: The name tf.app.run is deprecated. Please use tf.compat.v1.app.run instead.

c:\users\user_\pycharmprojects\capstoneproject2_test\fcn.py(153)main() -> keep_probability = tf.placeholder(tf.float32, name="keep_probabilty") (Pdb) n WARNING:tensorflow:From FCN.py:153: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.

W0509 20:47:39.141066 302320 deprecation_wrapper.py:119] From FCN.py:153: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.

c:\users\user_\pycharmprojects\capstoneproject2_test\fcn.py(154)main() ->image = tf.placeholder(tf.float32, shape=[None, IMAGE_SIZE, IMAGE_SIZE, 3], name="input_image") (Pdb) n c:\users\user_\pycharmprojects\capstoneproject2_test\fcn.py(155)main() -> annotation = tf.placeholder(tf.int32, shape=[None, IMAGE_SIZE, IMAGE_SIZE, 1], name="annotation") (Pdb) n c:\users\user_\pycharmprojects\capstoneproject2_test\fcn.py(158)main() -> pred_annotation, logits = inference(image, keep_probability) (Pdb) n setting up vgg initialized conv layers ... WARNING:tensorflow:From FCN.py:92: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.

W0509 20:47:44.351018 302320 deprecation_wrapper.py:119] From FCN.py:92: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.

WARNING:tensorflow:From C:\Users\user_\PycharmProjects\CapstoneProject2_test\TensorflowUtils.py:57: calling Constant.init (from tensorflow.python.ops.initops) with dtype is deprecate d and will be removed in a future version. Instructions for updating: Call initializer instance with the dtype argument instead of passing it to the constructor W0509 20:47:44.351018 302320 deprecation.py:506] From C:\Users\user\PycharmProjects\CapstoneProject2_test\TensorflowUtils.py:57: calling Constant.init (from tensorflow.python.ops.ini tops) with dtype is deprecated and will be removed in a future version. Instructions for updating: Call initializer instance with the dtype argument instead of passing it to the constructor **WARNING:tensorflow:From C:\Users\user\PycharmProjects\CapstoneProject2_test\TensorflowUtils.py:58: The name tf.get_variable is deprecated. Please use tf.compat.v1.get_variable instead.**

W0509 20:47:44.351018 302320 deprecationwrapper.py:119] From C:\Users\user\PycharmProjects\CapstoneProject2_test\TensorflowUtils.py:58: The name tf.get_variable is deprecated. Please us e tf.compat.v1.get_variable instead.

WARNING:tensorflow:From C:\Users\user_\PycharmProjects\CapstoneProject2_test\TensorflowUtils.py:101: The name tf.nn.avg_pool is deprecated. Please use tf.nn.avg_pool2d instead.

W0509 20:47:44.501434 302320 deprecationwrapper.py:119] From C:\Users\user\PycharmProjects\CapstoneProject2_test\TensorflowUtils.py:101: The name tf.nn.avg_pool is deprecated. Please us e tf.nn.avg_pool2d instead.

WARNING:tensorflow:From C:\Users\user_\PycharmProjects\CapstoneProject2_test\TensorflowUtils.py:97: The name tf.nn.max_pool is deprecated. Please use tf.nn.max_pool2d instead.

W0509 20:47:46.256138 302320 deprecationwrapper.py:119] From C:\Users\user\PycharmProjects\CapstoneProject2_test\TensorflowUtils.py:97: The name tf.nn.max_pool is deprecated. Please use tf.nn.max_pool2d instead.

WARNING:tensorflow:From C:\Users\user_\PycharmProjects\CapstoneProject2_test\TensorflowUtils.py:63: The name tf.truncated_normal is deprecated. Please use tf.random.truncated_normal instead.

W0509 20:47:46.261185 302320 deprecationwrapper.py:119] From C:\Users\user\PycharmProjects\CapstoneProject2_test\TensorflowUtils.py:63: The name tf.truncated_normal is deprecated. Pleas e use tf.random.truncated_normal instead.

WARNING:tensorflow:From FCN.py:105: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version. Instructions for updating: Please use rate instead of keep_prob. Rate should be set to rate = 1 - keep_prob. W0509 20:47:46.341412 302320 deprecation.py:506] From FCN.py:105: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.

Instructions for updating: Please use rate instead of keep_prob. Rate should be set to rate = 1 - keep_prob. WARNING:tensorflow:From FCN.py:145: calling argmax (from tensorflow.python.ops.math_ops) with dimension is deprecated and will be removed in a future version. Instructions for updating: Use the axis argument instead W0509 20:47:46.956080 302320 deprecation.py:506] From FCN.py:145: calling argmax (from tensorflow.python.ops.math_ops) with dimension is deprecated and will be removed in a future version . Instructions for updating: Use the axis argument instead WARNING:tensorflow:From C:\anaconda3\lib\site-packages\tensorflow\python\util\dispatch.py:180: calling expand_dims (from tensorflow.python.ops.array_ops) with dim is deprecated and will b e removed in a future version. Instructions for updating: Use the axis argument instead W0509 20:47:46.961135 302320 deprecation.py:506] From C:\anaconda3\lib\site-packages\tensorflow\python\util\dispatch.py:180: calling expand_dims (from tensorflow.python.ops.array_ops) wit h dim is deprecated and will be removed in a future version. Instructions for updating: Use the axis argument instead Instructions for updating: Use the axis argument instead WARNING:tensorflow:From C:\anaconda3\lib\site-packages\tensorflow\python\util\dispatch.py:180: calling expand_dims (from tensorflow.python.ops.array_ops) with dim is deprecated and will b e removed in a future version. Instructions for updating: Use the axis argument instead W0509 20:47:46.961135 302320 deprecation.py:506] From C:\anaconda3\lib\site-packages\tensorflow\python\util\dispatch.py:180: calling expand_dims (from tensorflow.python.ops.arrayops) wit h dim is deprecated and will be removed in a future version. Instructions for updating: Use the axis argument instead c:\users\user\pycharmprojects\capstoneproject2_test\fcn.py(159)main() ->tf.summary.image("input_image", image, max_outputs=2) (Pdb) n WARNING:tensorflow:From FCN.py:159: The name tf.summary.image is deprecated. Please use tf.compat.v1.summary.image instead.

W0509 20:50:38.010888 302320 deprecation_wrapper.py:119] From FCN.py:159: The name tf.summary.image is deprecated. Please use tf.compat.v1.summary.image instead.

c:\users\user_\pycharmprojects\capstoneproject2_test\fcn.py(160)main() ->tf.summary.image("ground_truth", tf.cast(annotation, tf.uint8), max_outputs=2) (Pdb) n c:\users\user_\pycharmprojects\capstoneproject2_test\fcn.py(161)main() -> tf.summary.image("pred_annotation", tf.cast(pred_annotation, tf.uint8), max_outputs=2) (Pdb) n c:\users\user_\pycharmprojects\capstoneproject2_test\fcn.py(164)main() -> loss = tf.reduce_mean((tf.nn.sparse_softmax_cross_entropy_with_logits(logits=logits,labels=tf.squeeze(annotation, squeeze_dims=[3]), name="entropy"))) (Pdb)

굵은글씨로 표시한 부분 보면 (페이지 번역 사용하세요 머리아프니까) 딱히 에러까지는 아닌데 경고메세지로 계속해서 ~~는 더이상 사용되지 않습니다 ~~를 사용해주세요. 라고 뜨네요; 죄다 바꿔야하나.. 일단 계속 해보겠음

  # 위 main()함수 이어서
  # 손실함수를 선언하고 손실함수에 대한 summary를 지정합니다.
  loss = tf.reduce_mean((tf.nn.sparse_softmax_cross_entropy_with_logits(logits=logits,labels=tf.squeeze(annotation, squeeze_dims=[3]), name="entropy")))
  tf.summary.scalar("entropy", loss)

c:\users\user_\pycharmprojects\capstoneproject2_test\fcn.py(164)main() ->loss = tf.reduce_mean((tf.nn.sparse_softmax_cross_entropy_with_logits(logits=logits,labels=tf.squeeze(annotation, squeeze_dims=[3]), name="entropy"))) (Pdb) n WARNING:tensorflow:From C:\anaconda3\lib\site-packages\tensorflow\python\util\dispatch.py:180: calling squeeze (from tensorflow.python.ops.array_ops) with squeeze_dims is deprecated and w ill be removed in a future version. Instructions for updating: Use the axis argument insteadhttps://guides.github.com/features/mastering-markdown/ W0509 21:01:29.026000 302320 deprecation.py:506] From C:\anaconda3\lib\site-packages\tensorflow\python\util\dispatch.py:180: calling squeeze (from tensorflow.python.ops.array_ops) with sq ueezedims is deprecated and will be removed in a future version. Instructions for updating: Use the axis argument instead c:\users\user\pycharmprojects\capstoneproject2_test\fcn.py(165)main() -> tf.summary.scalar("entropy", loss) (Pdb) n WARNING:tensorflow:From FCN.py:165: The name tf.summary.scalar is deprecated. Please use tf.compat.v1.summary.scalar instead.

W0509 21:08:04.730573 302320 deprecation_wrapper.py:119] From FCN.py:165: The name tf.summary.scalar is deprecated. Please use tf.compat.v1.summary.scalar instead.

tf.compat.v1.~~ 참고링크 : https://mylifemystudy.tistory.com/73

아나콘다랑 파이참 둘다 텐서플로 버전 확인해봄;

분명 텐서플로 1.14.0으로 설치되어 있을텐데 2.0 -> 1.x 변환 코드를 쓰라네요; 영문을 모르겠는; ㅋㅋ

ㅋㅋㅋ텐서플로 버전 1.14.0인데??? 흠..

근데 경고메세지라 사실 무시해도 될거같긴 함; 에러는 아니니께

버전이 안맞는걸로 생긴 경고라면 , 가상환경을 써보세용.(경고가 거슬리면!) 거기다가 다시 tensorflow 버전에 맞게 깔아보는건 어때유? 저기 써져있는 코드의 버전이랑 지희컴 버전이랑 안맞는거 아닌가?

ㅇㅋㅇㅋ 일단 디버깅 더 해보고 정 안되면 해보겠음 근데 경고라면 뭐.. ㄱㅊ지 않을까ㅋㅋ

오키오키 나도 해볼게! 너랑 버전 달라서 될 수도 있어.

def main(argv=None):
~ 중략
  print("Setting up image reader...")
  train_records, valid_records = scene_parsing.read_dataset(FLAGS.data_dir)
  print(len(train_records))
  print(len(valid_records))
~ 중략

c:\users\user_\pycharmprojects\capstoneproject2test\fcn.py(177)main() -> print("Setting up image reader...") (Pdb) n Setting up image reader... c:\users\user\pycharmprojects\capstoneproject2_test\fcn.py(178)main() -> train_records, valid_records = scene_parsing.read_dataset(FLAGS.data_dir) (Pdb) n Found pickle file! c:\users\user_\pycharmprojects\capstoneproject2_test\fcn.py(179)main() -> print(len(train_records)) (Pdb) n 0 c:\users\user_\pycharmprojects\capstoneproject2_test\fcn.py(180)main() -> print(len(valid_records)) (Pdb) n 0

main()함수 안에서 training 데이터와 validation 데이터의 개수를 불러오는 부분이 0이라고 찍히네요, 위에 경고 메세지는 무시해도 될 것 같습니다 (아니면 텐서플로 버전문제로 안불러와지는건가; 모르겠음) 쨌든 데이터 자체가 안불러와졌으니 당연하게 sess.run(train_step, feed_dict=feed_dict)에서 에러가 나는듯? 아직 FCN.py 코드 해석 덜 되었으니 이것저것 더 해보고 추가로 달겠음다~

위 문제에서 train_records, valid_records = scene_parsing.read_dataset(FLAGS.data_dir)부분이 근본적인 원인일거라 생각합니다. 사용된 read_dataset()함수는 read_MITSceneParsingData.py 안의 함수로, 코드는 아래와 같음. 귀찮아서 일부만 가져왔으니 jihee_FCNtest 브랜치에 push해둔 코드 확인하거나 지윤님이 올린 코드 확인하세요.

코드 해석 필요합니다 (이따가 함;) FCN.py를 실행시켰을 때 결과는 Found pickle file! 이 뜹니다.

# MIT Scene Parsing 데이터를 다운로드 받을 경로
DATA_URL = 'http://data.csail.mit.edu/places/ADEchallenge/ADEChallengeData2016.zip' 

# 다운받은 MIT Scene Parsing 데이터를 읽습니다.
def read_dataset(data_dir):
  pickle_filename = "MITSceneParsing.pickle"
  pickle_filepath = os.path.join(data_dir, pickle_filename)
  # MITSceneParsing.pickle 파일이 없으면 다운 받은 MITSceneParsing 데이터를 pickle 파일로 저장합니다.
  if not os.path.exists(pickle_filepath):
    utils.maybe_download_and_extract(data_dir, DATA_URL, is_zipfile=True)
    SceneParsing_folder = os.path.splitext(DATA_URL.split("/")[-1])[0]
    result = create_image_lists(os.path.join(data_dir, SceneParsing_folder))
    print ("Pickling ...")
    with open(pickle_filepath, 'wb') as f:
      pickle.dump(result, f, pickle.HIGHEST_PROTOCOL)
  else:
    print ("Found pickle file!")

s명령어로 train_records, valid_records = scene_parsing.read_dataset(FLAGS.data_dir)에서 read_dataset()로 step in

c:\users\user_\pycharmprojects\capstoneproject2_test\ read_mitsceneparsingdata.py(16)read_dataset() -> def read_dataset(data_dir): (Pdb) n c:\users\user_\pycharmprojects\capstoneproject2_test\read_mitsceneparsingdata.py(17)read_dataset() ->pickle_filename = "MITSceneParsing.pickle" (Pdb) n c:\users\user_\pycharmprojects\capstoneproject2_test\read_mitsceneparsingdata.py(18)read_dataset() -> pickle_filepath = os.path.join(data_dir, pickle_filename) (Pdb) n c:\users\user_\pycharmprojects\capstoneproject2_test\read_mitsceneparsingdata.py(20)read_dataset() -> if not os.path.exists(pickle_filepath): (Pdb) s --Call-- c:\anaconda3\lib\genericpath.py(16)exists() -> def exists(path): (Pdb) s c:\anaconda3\lib\genericpath.py(18)exists() -> try: (Pdb) s c:\anaconda3\lib\genericpath.py(19)exists() -> os.stat(path) (Pdb) s c:\anaconda3\lib\genericpath.py(22)exists() -> return True (Pdb) s --Return-- c:\anaconda3\lib\genericpath.py(22)exists()->True -> return True (Pdb) s c:\users\user_\pycharmprojects\capstoneproject2_test\read_mitsceneparsingdata.py(28)read_dataset() -> print ("Found pickle file!") (Pdb)

jjimini98 / CapstoneProject2

FCN.py 디버깅 : pdb.set_trace() #19

디버깅 결과