yu4u / noise2noise

An unofficial and partial Keras implementation of "Noise2Noise: Learning Image Restoration without Clean Data"
MIT License
1.07k stars 237 forks source link

I have GPU GeForce GTX 1050 and 32GB RAM but still issuse of OOM #55

Open haideralifaridkhan opened 3 years ago

haideralifaridkhan commented 3 years ago

199/200 [============================>.] - ETA: 0s - loss: 17700.6033 - PSNR: 10.1373Traceback (most recent call last): File "train.py", line 112, in main() File "train.py", line 106, in main callbacks=callbacks) File "/usr/local/lib64/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 1861, in fit_generator initial_epoch=initial_epoch) File "/usr/local/lib64/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 1141, in fit return_dict=True) File "/usr/local/lib64/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 1389, in evaluate tmp_logs = self.test_function(iterator) File "/usr/local/lib64/python3.6/site-packages/tensorflow/python/eager/def_function.py", line 828, in call result = self._call(*args, kwds) File "/usr/local/lib64/python3.6/site-packages/tensorflow/python/eager/def_function.py", line 871, in _call self._initialize(args, kwds, add_initializers_to=initializers) File "/usr/local/lib64/python3.6/site-packages/tensorflow/python/eager/def_function.py", line 726, in _initialize *args, *kwds)) File "/usr/local/lib64/python3.6/site-packages/tensorflow/python/eager/function.py", line 2969, in _get_concrete_function_internal_garbage_collected graphfunction, = self._maybe_define_function(args, kwargs) File "/usr/local/lib64/python3.6/site-packages/tensorflow/python/eager/function.py", line 3361, in _maybe_define_function graph_function = self._create_graph_function(args, kwargs) File "/usr/local/lib64/python3.6/site-packages/tensorflow/python/eager/function.py", line 3206, in _create_graph_function capture_by_value=self._capture_by_value), File "/usr/local/lib64/python3.6/site-packages/tensorflow/python/framework/func_graph.py", line 990, in func_graph_from_py_func func_outputs = python_func(func_args, func_kwargs) File "/usr/local/lib64/python3.6/site-packages/tensorflow/python/eager/def_function.py", line 634, in wrapped_fn out = weak_wrapped_fn().wrapped(*args, **kwds) File "/usr/local/lib64/python3.6/site-packages/tensorflow/python/framework/func_graph.py", line 977, in wrapper raise e.ag_error_metadata.to_exception(e) ValueError: in user code:

/usr/local/lib64/python3.6/site-packages/tensorflow/python/keras/engine/training.py:1233 test_function  *
    return step_function(self, iterator)
/usr/local/lib64/python3.6/site-packages/tensorflow/python/keras/engine/training.py:1224 step_function  **
    outputs = model.distribute_strategy.run(run_step, args=(data,))
/usr/local/lib64/python3.6/site-packages/tensorflow/python/distribute/distribute_lib.py:1259 run
    return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs)
/usr/local/lib64/python3.6/site-packages/tensorflow/python/distribute/distribute_lib.py:2730 call_for_each_replica
    return self._call_for_each_replica(fn, args, kwargs)
/usr/local/lib64/python3.6/site-packages/tensorflow/python/distribute/distribute_lib.py:3417 _call_for_each_replica
    return fn(*args, **kwargs)
/usr/local/lib64/python3.6/site-packages/tensorflow/python/keras/engine/training.py:1217 run_step  **
    outputs = model.test_step(data)
/usr/local/lib64/python3.6/site-packages/tensorflow/python/keras/engine/training.py:1183 test_step
    y_pred = self(x, training=False)
/usr/local/lib64/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py:998 __call__
    input_spec.assert_input_compatibility(self.input_spec, inputs, self.name)
/usr/local/lib64/python3.6/site-packages/tensorflow/python/keras/engine/input_spec.py:207 assert_input_compatibility
    ' input tensors. Inputs received: ' + str(inputs))

ValueError: Layer model expects 1 input(s), but it received 2 input tensors. Inputs received: [<tf.Tensor 'IteratorGetNext:0' shape=(None, None, None, None) dtype=uint8>, <tf.Tensor 'IteratorGetNext:1' shape=(None, None, None, None) dtype=uint8>]

2165-01-12 17:11:32.737853: W tensorflow/core/kernels/data/generator_dataset_op.cc:107] Error occurred when finalizing GeneratorDataset iterator: Failed precondition: Python interpreter state is not initialized. The process may be terminated. [[{{node PyFunc}}]]

haideralifaridkhan commented 3 years ago

How much minimum specification computer required to performed this code?

Andyshen555 commented 3 years ago

This is not a problem of OOM. You have to change the code in generator py to make the list into a tuple, since now validation set only accepts tuple instead of list