search

Naver-AI-Hackathon / AI-Vision

67 stars 34 forks source link

nsml load 관련 에러 #142

Open ralasun opened 5 years ago

ralasun commented 5 years ago

기타

안녕하세요, 기존에 모델구조를 변경하지 않고, 모델을 불러와서 재학습시키려 하는데, 여러번 실행할때마다 에러나는 위치가 다릅니다. 코드는 아래와 같습니다.

모델정의후,

if confg.mode == 'train': ... model.compile(loss='categorical_crossentropy', optimizer=opt_rmsprop, metrics=['accuracy'])

    nsml.load(51, session='INLine/ir_ph1_v2/167')

.....

에러원인이 어떤건지 알 수 있을까요?

session 178 실행시 에러

Traceback (most recent call last): File "main_vgg_finetuned.py", line 231, in nsml.load(51, session='INLine/ir_ph1_v2/167') File "/app/nsml/client.py", line 439, in load load_fn(temp_file.name) File "main_vgg_finetuned.py", line 41, in load model.load_weights(file_path) File "/opt/conda/lib/python3.5/site-packages/keras/engine/network.py", line 1166, in load_weights f, self.layers, reshape=reshape) File "/opt/conda/lib/python3.5/site-packages/keras/engine/saving.py", line 1058, in load_weights_from_hdf5_group K.batch_set_value(weight_value_tuples) File "/opt/conda/lib/python3.5/site-packages/keras/backend/tensorflow_backend.py", line 2470, in batch_set_value get_session().run(assign_ops, feed_dict=feed_dict) File "/opt/conda/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 887, in run run_metadata_ptr) File "/opt/conda/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1086, in _run str(subfeed_t.get_shape()))) ValueError: Cannot feed value of shape (512, 256, 3, 3) for Tensor 'Placeholder_12:0', which has shape '(3, 3, 256, 256)'Exception ignored in: <bound method BaseSession.del of <tensorflow.python.client.session.Session object at 0x2b80361abc50>> Traceback (most recent call last): File "/opt/conda/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 696, in delTypeError: 'NoneType' object is not callable User session exited

session 180 실행시 에러

Traceback (most recent call last): File "main_vgg_finetuned.py", line 226, in nsml.load(51, session='INLine/ir_ph1_v2/167') File "/app/nsml/client.py", line 439, in load load_fn(temp_file.name) File "main_vgg_finetuned.py", line 41, in load model.load_weights(file_path) File "/opt/conda/lib/python3.5/site-packages/keras/engine/network.py", line 1166, in load_weights f, self.layers, reshape=reshape) File "/opt/conda/lib/python3.5/site-packages/keras/engine/saving.py", line 1058, in load_weights_from_hdf5_group K.batch_set_value(weight_value_tuples) File "/opt/conda/lib/python3.5/site-packages/keras/backend/tensorflow_backend.py", line 2470, in batch_set_value get_session().run(assign_ops, feed_dict=feed_dict) File "/opt/conda/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 887, in run run_metadata_ptr) File "/opt/conda/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1086, in _run str(subfeed_t.get_shape())))ValueError: Cannot feed value of shape (512,) for Tensor 'Placeholder_5:0', which has shape '(128,)' User session exited

181세션

Traceback (most recent call last): File "main_vgg_finetuned.py", line 226, in nsml.load(checkpoint='51', session='INLine/ir_ph1_v2/167') File "/app/nsml/client.py", line 439, in load load_fn(temp_file.name) File "main_vgg_finetuned.py", line 41, in load model.load_weights(file_path) File "/opt/conda/lib/python3.5/site-packages/keras/engine/network.py", line 1166, in load_weights f, self.layers, reshape=reshape) File "/opt/conda/lib/python3.5/site-packages/keras/engine/saving.py", line 1058, in load_weights_from_hdf5_group K.batch_set_value(weight_value_tuples) File "/opt/conda/lib/python3.5/site-packages/keras/backend/tensorflow_backend.py", line 2470, in batch_set_value get_session().run(assign_ops, feed_dict=feed_dict) File "/opt/conda/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 887, in run run_metadata_ptr) File "/opt/conda/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1086, in _run str(subfeed_t.get_shape())))ValueError: Cannot feed value of shape (64,) for Tensor 'Placeholder_11:0', which has shape '(256,)' User session exited

감사합니다.

Hackoperation commented 5 years ago

안녕하세요.

에러나는 위치는 매번같지만 weight를 load할때의 내용이 달라지는것같습니다.

load하려는 모델의 구조와 코드안에서의 모델의 구조를 다시 한번확인부탁드립니다.

또한 web에서 각코드의 변경사항을 확인해볼수 있습니다. 참고