juglab / n2v

This is the implementation of Noise2Void training.
Other
392 stars 107 forks source link

Error messages while using the 2D example SEM in google colab #16

Closed guijacquemet closed 5 years ago

guijacquemet commented 5 years ago

When running the 2D example SEM example of n2v in google colab I now receive several error messages. It was working fine yesterday.

The main error happens during the training. One Epoch is completed successfully but then it crashes:

W0620 10:18:39.233062 140231502292864 deprecation_wrapper.py:119] From /usr/local/lib/python3.6/dist-packages/keras/optimizers.py:790: The name tf.train.Optimizer is deprecated. Please use tf.compat.v1.train.Optimizer instead.

W0620 10:18:47.475789 140231502292864 lazy_loader.py:50] The TensorFlow contrib module will not be included in TensorFlow 2.0. For more information, please see:

W0620 10:18:47.630948 140231502292864 deprecation_wrapper.py:119] From /usr/local/lib/python3.6/dist-packages/csbdeep/utils/tf.py:240: The name tf.summary.image is deprecated. Please use tf.compat.v1.summary.image instead.

W0620 10:18:48.145319 140231502292864 deprecation_wrapper.py:119] From /usr/local/lib/python3.6/dist-packages/csbdeep/utils/tf.py:268: The name tf.summary.merge is deprecated. Please use tf.compat.v1.summary.merge instead.

W0620 10:18:48.149436 140231502292864 deprecation_wrapper.py:119] From /usr/local/lib/python3.6/dist-packages/csbdeep/utils/tf.py:275: The name tf.summary.FileWriter is deprecated. Please use tf.compat.v1.summary.FileWriter instead.

Epoch 1/10 20/20 [==============================] - 23s 1s/step - loss: 1.2110 - n2v_mse: 1.2110 - n2v_abs: 0.8920 - val_loss: 1.5108 - val_n2v_mse: 1.5108 - val_n2v_abs: 0.9683

InvalidArgumentError Traceback (most recent call last) /usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py in _do_call(self, fn, args) 1355 try: -> 1356 return fn(args) 1357 except errors.OpError as e:

12 frames InvalidArgumentError: You must feed a value for placeholder tensor 'Placeholder' with dtype float and shape [?,?,?,1] [[{{node Placeholder}}]]

During handling of the above exception, another exception occurred:

InvalidArgumentError Traceback (most recent call last) /usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py in _do_call(self, fn, *args) 1368 pass 1369 message = error_interpolation.interpolate(message, self._graph) -> 1370 raise type(e)(node_def, op, message) 1371 1372 def _extend_graph(self):

InvalidArgumentError: You must feed a value for placeholder tensor 'Placeholder' with dtype float and shape [?,?,?,1] [[node Placeholder (defined at /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:517) ]]

Original stack trace for 'Placeholder': File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main "main", mod_spec) File "/usr/lib/python3.6/runpy.py", line 85, in _run_code exec(code, run_globals) File "/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py", line 16, in app.launch_new_instance() File "/usr/local/lib/python3.6/dist-packages/traitlets/config/application.py", line 658, in launch_instance app.start() File "/usr/local/lib/python3.6/dist-packages/ipykernel/kernelapp.py", line 477, in start ioloop.IOLoop.instance().start() File "/usr/local/lib/python3.6/dist-packages/tornado/ioloop.py", line 888, in start handler_func(fd_obj, events) File "/usr/local/lib/python3.6/dist-packages/tornado/stack_context.py", line 277, in null_wrapper return fn(*args, kwargs) File "/usr/local/lib/python3.6/dist-packages/zmq/eventloop/zmqstream.py", line 450, in _handle_events self._handle_recv() File "/usr/local/lib/python3.6/dist-packages/zmq/eventloop/zmqstream.py", line 480, in _handle_recv self._run_callback(callback, msg) File "/usr/local/lib/python3.6/dist-packages/zmq/eventloop/zmqstream.py", line 432, in _run_callback callback(*args, *kwargs) File "/usr/local/lib/python3.6/dist-packages/tornado/stack_context.py", line 277, in null_wrapper return fn(args, kwargs) File "/usr/local/lib/python3.6/dist-packages/ipykernel/kernelbase.py", line 283, in dispatcher return self.dispatch_shell(stream, msg) File "/usr/local/lib/python3.6/dist-packages/ipykernel/kernelbase.py", line 235, in dispatch_shell handler(stream, idents, msg) File "/usr/local/lib/python3.6/dist-packages/ipykernel/kernelbase.py", line 399, in execute_request user_expressions, allow_stdin) File "/usr/local/lib/python3.6/dist-packages/ipykernel/ipkernel.py", line 196, in do_execute res = shell.run_cell(code, store_history=store_history, silent=silent) File "/usr/local/lib/python3.6/dist-packages/ipykernel/zmqshell.py", line 533, in run_cell return super(ZMQInteractiveShell, self).run_cell(*args, kwargs) File "/usr/local/lib/python3.6/dist-packages/IPython/core/interactiveshell.py", line 2718, in run_cell interactivity=interactivity, compiler=compiler, result=result) File "/usr/local/lib/python3.6/dist-packages/IPython/core/interactiveshell.py", line 2822, in run_ast_nodes if self.run_code(code, result): File "/usr/local/lib/python3.6/dist-packages/IPython/core/interactiveshell.py", line 2882, in run_code exec(code_obj, self.user_global_ns, self.user_ns) File "", line 1, in history = model.train(X, X_val) File "/usr/local/lib/python3.6/dist-packages/n2v/models/n2v_standard.py", line 218, in train callbacks=self.callbacks, verbose=1) File "/usr/local/lib/python3.6/dist-packages/keras/legacy/interfaces.py", line 91, in wrapper return func(*args, *kwargs) File "/usr/local/lib/python3.6/dist-packages/keras/engine/training.py", line 1418, in fit_generator initial_epoch=initial_epoch) File "/usr/local/lib/python3.6/dist-packages/keras/engine/training_generator.py", line 94, in fit_generator callbacks.set_model(callback_model) File "/usr/local/lib/python3.6/dist-packages/keras/callbacks.py", line 54, in set_model callback.set_model(model) File "/usr/local/lib/python3.6/dist-packages/csbdeep/utils/tf.py", line 213, in set_model self.gt_outputs = [K.placeholder(shape=_gt_shape(K.int_shape(x))) for x in self.model.outputs] File "/usr/local/lib/python3.6/dist-packages/csbdeep/utils/tf.py", line 213, in self.gt_outputs = [K.placeholder(shape=_gt_shape(K.int_shape(x))) for x in self.model.outputs] File "/usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py", line 517, in placeholder x = tf.placeholder(dtype, shape=shape, name=name) File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/array_ops.py", line 2143, in placeholder return gen_array_ops.placeholder(dtype=dtype, shape=shape, name=name) File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/gen_array_ops.py", line 6262, in placeholder "Placeholder", dtype=dtype, shape=shape, name=name) File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/op_def_library.py", line 788, in _apply_op_helper op_def=op_def) File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/util/deprecation.py", line 507, in new_func return func(args, kwargs) File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py", line 3616, in create_op op_def=op_def) File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py", line 2005, in init self._traceback = tf_stack.extract_stack()

Another issue happened also earlier

a name used to identify the model

model_name = 'n2v_2D'

the base directory in which our model will live

basedir = 'models'

We are now creating our network model.

model = N2V(config, model_name, basedir=basedir)

Errors:

WARNING: Logging before flag parsing goes to stderr. W0620 10:18:26.427209 140231502292864 deprecation_wrapper.py:119] From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:517: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.

W0620 10:18:26.472275 140231502292864 deprecation_wrapper.py:119] From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:4138: The name tf.random_uniform is deprecated. Please use tf.random.uniform instead.

W0620 10:18:26.503654 140231502292864 deprecation_wrapper.py:119] From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:74: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead.

W0620 10:18:26.519927 140231502292864 deprecation_wrapper.py:119] From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:174: The name tf.get_default_session is deprecated. Please use tf.compat.v1.get_default_session instead.

W0620 10:18:26.521225 140231502292864 deprecation_wrapper.py:119] From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:181: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.

W0620 10:18:29.414467 140231502292864 deprecation_wrapper.py:119] From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:1834: The name tf.nn.fused_batch_norm is deprecated. Please use tf.compat.v1.nn.fused_batch_norm instead.

W0620 10:18:29.695360 140231502292864 deprecation_wrapper.py:119] From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:3976: The name tf.nn.max_pool is deprecated. Please use tf.nn.max_pool2d instead.

W0620 10:18:30.005542 140231502292864 deprecation_wrapper.py:119] From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:2018: The name tf.image.resize_nearest_neighbor is deprecated. Please use tf.compat.v1.image.resize_nearest_neighbor instead.

alex-krull commented 5 years ago

Hello guijacquemet, thanks for pointing out the problem! This issue was created because of an incompatibility with the new csbdeep version 0.4.0. I changed the dependency back to 0.3.0. Things are now working again. You might have to uninstall and then reinstall n2v using pip.

guijacquemet commented 5 years ago

Thanks works like a charm now