david-vazquez / mcv-m5

Master in Computer Vision - M5 Visual recognition
13 stars 51 forks source link

Error trying to apply eval_detection_fscore.py #28

Open carlosb1 opened 7 years ago

carlosb1 commented 7 years ago

We just trained our yolo model but we we try to apply the eval_detection_fscore.py with our trained model, we received an error:

  File "/home/master/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1035, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.ResourceExhaustedError: OOM when allocating tensor with shape[1024]
     [[Node: yoloconvolution2d_21_b/Assign = Assign[T=DT_FLOAT, _class=["loc:@yoloconvolution2d_21_b"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/gpu:0"](yoloconvolution2d_21_b, Const_120)]]

Caused by op u'yoloconvolution2d_21_b/Assign', defined at:
  File "eval_detection_fscore.py", line 50, in <module>
    tiny=tiny_yolo)
  File "/home/master/week3/mcv-m5-master/code/models/yolo.py", line 23, in build_yolo
    model = YOLO(input_shape=img_shape, num_classes=n_classes, num_priors=n_priors)
  File "/home/master/week3/mcv-m5-master/code/models/yolo.py", line 138, in YOLO
    epsilon=0.000001))(net['merge7'])
  File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 546, in __call__
    self.build(input_shapes[0])
  File "/home/master/week3/mcv-m5-master/code/layers/yolo_layers.py", line 143, in build
    constraint=self.b_constraint)
  File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 418, in add_weight
    weight = initializer(shape, name=name)
  File "/usr/local/lib/python2.7/dist-packages/keras/initializations.py", line 112, in zero
    return K.zeros(shape, name=name)
  File "/usr/local/lib/python2.7/dist-packages/keras/backend/tensorflow_backend.py", line 492, in zeros
    dtype, name)
  File "/usr/local/lib/python2.7/dist-packages/keras/backend/tensorflow_backend.py", line 259, in variable
    v = tf.Variable(value, dtype=_convert_string_dtype(dtype), name=name)
  File "/home/master/.local/lib/python2.7/site-packages/tensorflow/python/ops/variables.py", line 226, in __init__
    expected_shape=expected_shape)
  File "/home/master/.local/lib/python2.7/site-packages/tensorflow/python/ops/variables.py", line 334, in _init_from_args
    validate_shape=validate_shape).op
  File "/home/master/.local/lib/python2.7/site-packages/tensorflow/python/ops/gen_state_ops.py", line 47, in assign
    use_locking=use_locking, name=name)
  File "/home/master/.local/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 763, in apply_op
    op_def=op_def)
  File "/home/master/.local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2395, in create_op
    original_op=self._default_original_op, op_def=op_def)
  File "/home/master/.local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1264, in __init__
    self._traceback = _extract_stack()

ResourceExhaustedError (see above for traceback): OOM when allocating tensor with shape[1024]
     [[Node: yoloconvolution2d_21_b/Assign = Assign[T=DT_FLOAT, _class=["loc:@yoloconvolution2d_21_b"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/gpu:0"](yoloconvolution2d_21_b, Const_120)]]
lluisgomez commented 7 years ago

@carlosb1 it looks like an Out Of Memory error. Which device are you using?

lfrcoder commented 6 years ago

Hello, I saw your work on the mcv-m5 project on github and I feel very good, but there is only code on github, no data and corresponding keras development kit. Can you pass a complete data such as relevant data and keras development kit? thank you very much!