search

lanha / DSen2

Super-Resolution of Sentinel-2 Images: Learning a Globally Applicable Deep Neural Network
GNU General Public License v3.0
235 stars 71 forks source link

UnimplementedError ... The Conv2D op currently only supports the NHWC tensor format on the CPU. #44

Closed guopeng-jiang closed 2 weeks ago

guopeng-jiang commented 6 months ago

I run the following line

%run demoDSen2.py but got the following error as below. Please help.

---------------------------------------------------------------------------
UnimplementedError                        Traceback (most recent call last)
File ~\OneDrive - University of Canterbury\DSen2\testing\demoDSen2.py:47
     45 print('Siberia')
     46 im10, im20, imGT = readh5('S2B_MSIL1C_20170725_T43WFQ.mat', imGT=True)
---> 47 SR20 = DSen2_20(im10, im20)
     48 # Evaluation against the ground truth on the 20m resolution bands (simulated)
     49 print('DSen2:')

File ~\OneDrive - University of Canterbury\DSen2\testing\supres.py:27, in DSen2_20(d10, d20, deep)
     25 test = [p10, p20]
     26 input_shape = ((4, None, None), (6, None, None))
---> 27 prediction = _predict(test, input_shape, deep=deep)
     28 images = recompose_images(prediction, border=border, size=d10.shape)
     29 images *= SCALE

File ~\OneDrive - University of Canterbury\DSen2\testing\supres.py:65, in _predict(test, input_shape, deep, run_60)
     63 model.load_weights(predict_file)
     64 print("Predicting using file: {}".format(predict_file))
---> 65 prediction = model.predict(test, verbose=1)
     66 return prediction

File ~\AppData\Local\anaconda3\Lib\site-packages\keras\src\utils\traceback_utils.py:70, in filter_traceback.<locals>.error_handler(*args, **kwargs)
     67     filtered_tb = _process_traceback_frames(e.__traceback__)
     68     # To get the full stack trace, call:
     69     # `tf.debugging.disable_traceback_filtering()`
---> 70     raise e.with_traceback(filtered_tb) from None
     71 finally:
     72     del filtered_tb

File ~\AppData\Roaming\Python\Python311\site-packages\tensorflow\python\eager\execute.py:53, in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
     51 try:
     52   ctx.ensure_initialized()
---> 53   tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
     54                                       inputs, attrs, num_outputs)
     55 except core._NotOkStatusException as e:
     56   if name is not None:

UnimplementedError: Graph execution error:

Detected at node 'model_1/conv2d_14/Conv2D' defined at (most recent call last):
    File "<frozen runpy>", line 198, in _run_module_as_main
    File "<frozen runpy>", line 88, in _run_code
    File "C:\Users\gji19\AppData\Local\anaconda3\Lib\site-packages\ipykernel_launcher.py", line 17, in <module>
      app.launch_new_instance()
    File "C:\Users\gji19\AppData\Local\anaconda3\Lib\site-packages\traitlets\config\application.py", line 992, in launch_instance
      app.start()
    File "C:\Users\gji19\AppData\Local\anaconda3\Lib\site-packages\ipykernel\kernelapp.py", line 711, in start
      self.io_loop.start()
    File "C:\Users\gji19\AppData\Roaming\Python\Python311\site-packages\tornado\platform\asyncio.py", line 195, in start
      self.asyncio_loop.run_forever()
    File "C:\Users\gji19\AppData\Local\anaconda3\Lib\asyncio\base_events.py", line 607, in run_forever
      self._run_once()
    File "C:\Users\gji19\AppData\Local\anaconda3\Lib\asyncio\base_events.py", line 1922, in _run_once
      handle._run()
    File "C:\Users\gji19\AppData\Local\anaconda3\Lib\asyncio\events.py", line 80, in _run
      self._context.run(self._callback, *self._args)
    File "C:\Users\gji19\AppData\Local\anaconda3\Lib\site-packages\ipykernel\kernelbase.py", line 510, in dispatch_queue
      await self.process_one()
    File "C:\Users\gji19\AppData\Local\anaconda3\Lib\site-packages\ipykernel\kernelbase.py", line 499, in process_one
      await dispatch(*args)
    File "C:\Users\gji19\AppData\Local\anaconda3\Lib\site-packages\ipykernel\kernelbase.py", line 406, in dispatch_shell
      await result
    File "C:\Users\gji19\AppData\Local\anaconda3\Lib\site-packages\ipykernel\kernelbase.py", line 729, in execute_request
      reply_content = await reply_content
    File "C:\Users\gji19\AppData\Local\anaconda3\Lib\site-packages\ipykernel\ipkernel.py", line 411, in do_execute
      res = shell.run_cell(
    File "C:\Users\gji19\AppData\Local\anaconda3\Lib\site-packages\ipykernel\zmqshell.py", line 531, in run_cell
      return super().run_cell(*args, **kwargs)
    File "C:\Users\gji19\AppData\Local\anaconda3\Lib\site-packages\IPython\core\interactiveshell.py", line 3006, in run_cell
      result = self._run_cell(
    File "C:\Users\gji19\AppData\Local\anaconda3\Lib\site-packages\IPython\core\interactiveshell.py", line 3061, in _run_cell
      result = runner(coro)
    File "C:\Users\gji19\AppData\Local\anaconda3\Lib\site-packages\IPython\core\async_helpers.py", line 129, in _pseudo_sync_runner
      coro.send(None)
    File "C:\Users\gji19\AppData\Local\anaconda3\Lib\site-packages\IPython\core\interactiveshell.py", line 3266, in run_cell_async
      has_raised = await self.run_ast_nodes(code_ast.body, cell_name,
    File "C:\Users\gji19\AppData\Local\anaconda3\Lib\site-packages\IPython\core\interactiveshell.py", line 3445, in run_ast_nodes
      if await self.run_code(code, result, async_=asy):
    File "C:\Users\gji19\AppData\Local\anaconda3\Lib\site-packages\IPython\core\interactiveshell.py", line 3505, in run_code
      exec(code_obj, self.user_global_ns, self.user_ns)
    File "C:\Users\gji19\AppData\Local\Temp\ipykernel_22004\3374343633.py", line 1, in <module>
      get_ipython().run_line_magic('run', 'demoDSen2.py')
    File "C:\Users\gji19\AppData\Local\anaconda3\Lib\site-packages\IPython\core\interactiveshell.py", line 2414, in run_line_magic
      result = fn(*args, **kwargs)
    File "C:\Users\gji19\AppData\Local\anaconda3\Lib\site-packages\IPython\core\magics\execution.py", line 834, in run
      run()
    File "C:\Users\gji19\AppData\Local\anaconda3\Lib\site-packages\IPython\core\magics\execution.py", line 819, in run
      runner(filename, prog_ns, prog_ns,
    File "C:\Users\gji19\AppData\Local\anaconda3\Lib\site-packages\IPython\core\pylabtools.py", line 213, in mpl_execfile
      safe_execfile(fname,*where,**kw)
    File "C:\Users\gji19\AppData\Local\anaconda3\Lib\site-packages\IPython\core\interactiveshell.py", line 2863, in safe_execfile
      py3compat.execfile(
    File "C:\Users\gji19\AppData\Local\anaconda3\Lib\site-packages\IPython\utils\py3compat.py", line 55, in execfile
      exec(compiler(f.read(), fname, "exec"), glob, loc)
    File "C:\Users\gji19\OneDrive - University of Canterbury\DSen2\testing\demoDSen2.py", line 47, in <module>
      SR20 = DSen2_20(im10, im20)
    File "C:\Users\gji19\OneDrive - University of Canterbury\DSen2\testing\supres.py", line 27, in DSen2_20
      prediction = _predict(test, input_shape, deep=deep)
    File "C:\Users\gji19\OneDrive - University of Canterbury\DSen2\testing\supres.py", line 65, in _predict
      prediction = model.predict(test, verbose=1)
    File "C:\Users\gji19\AppData\Local\anaconda3\Lib\site-packages\keras\src\utils\traceback_utils.py", line 65, in error_handler
      return fn(*args, **kwargs)
    File "C:\Users\gji19\AppData\Local\anaconda3\Lib\site-packages\keras\src\engine\training.py", line 2554, in predict
      tmp_batch_outputs = self.predict_function(iterator)
    File "C:\Users\gji19\AppData\Local\anaconda3\Lib\site-packages\keras\src\engine\training.py", line 2341, in predict_function
      return step_function(self, iterator)
    File "C:\Users\gji19\AppData\Local\anaconda3\Lib\site-packages\keras\src\engine\training.py", line 2327, in step_function
      outputs = model.distribute_strategy.run(run_step, args=(data,))
    File "C:\Users\gji19\AppData\Local\anaconda3\Lib\site-packages\keras\src\engine\training.py", line 2315, in run_step
      outputs = model.predict_step(data)
    File "C:\Users\gji19\AppData\Local\anaconda3\Lib\site-packages\keras\src\engine\training.py", line 2283, in predict_step
      return self(x, training=False)
    File "C:\Users\gji19\AppData\Local\anaconda3\Lib\site-packages\keras\src\utils\traceback_utils.py", line 65, in error_handler
      return fn(*args, **kwargs)
    File "C:\Users\gji19\AppData\Local\anaconda3\Lib\site-packages\keras\src\engine\training.py", line 569, in __call__
      return super().__call__(*args, **kwargs)
    File "C:\Users\gji19\AppData\Local\anaconda3\Lib\site-packages\keras\src\utils\traceback_utils.py", line 65, in error_handler
      return fn(*args, **kwargs)
    File "C:\Users\gji19\AppData\Local\anaconda3\Lib\site-packages\keras\src\engine\base_layer.py", line 1150, in __call__
      outputs = call_fn(inputs, *args, **kwargs)
    File "C:\Users\gji19\AppData\Local\anaconda3\Lib\site-packages\keras\src\utils\traceback_utils.py", line 96, in error_handler
      return fn(*args, **kwargs)
    File "C:\Users\gji19\AppData\Local\anaconda3\Lib\site-packages\keras\src\engine\functional.py", line 512, in call
      return self._run_internal_graph(inputs, training=training, mask=mask)
    File "C:\Users\gji19\AppData\Local\anaconda3\Lib\site-packages\keras\src\engine\functional.py", line 669, in _run_internal_graph
      outputs = node.layer(*args, **kwargs)
    File "C:\Users\gji19\AppData\Local\anaconda3\Lib\site-packages\keras\src\utils\traceback_utils.py", line 65, in error_handler
      return fn(*args, **kwargs)
    File "C:\Users\gji19\AppData\Local\anaconda3\Lib\site-packages\keras\src\engine\base_layer.py", line 1150, in __call__
      outputs = call_fn(inputs, *args, **kwargs)
    File "C:\Users\gji19\AppData\Local\anaconda3\Lib\site-packages\keras\src\utils\traceback_utils.py", line 96, in error_handler
      return fn(*args, **kwargs)
    File "C:\Users\gji19\AppData\Local\anaconda3\Lib\site-packages\keras\src\layers\convolutional\base_conv.py", line 290, in call
      outputs = self.convolution_op(inputs, self.kernel)
    File "C:\Users\gji19\AppData\Local\anaconda3\Lib\site-packages\keras\src\layers\convolutional\base_conv.py", line 262, in convolution_op
      return tf.nn.convolution(
Node: 'model_1/conv2d_14/Conv2D'
The Conv2D op currently only supports the NHWC tensor format on the CPU. The op was given the format: NCHW
     [[{{node model_1/conv2d_14/Conv2D}}]] [Op:__inference_predict_function_1337]
RMNT commented 1 month ago

Currenly having the same issue. Maybe you solved it?

RMNT commented 2 weeks ago

I am tried to edit the code for it to work with CPU. What I have done so far:

  1. In supres.py change input_shape = ((4, 0, 0), (6, 0, 0)) to ((0, 0, 4), (0, 0, 6)) and before this line add transposition to (0, 2, 3, 1) to p10 and p20
  2. Change axis=1 to -1 in DSen2Net.py at Concatenate(axis=1)
  3. In DSen2Net.py change K.set_image_data_format to 'channels_last'
  4. In DSen2Net.py change image_shape[-1][0] to [-1][-1]

After that I am stuck getting error (128, 112, 0) cannot be fitted into (128, 112, 112) It is a shame that no one is looking after this project

lanha commented 2 weeks ago

I just had a look, what you did is correct. You just need to transpose back to channels first before calling recompose_images(...), i.e. prediction = prediction.transpose(0, 3, 1, 2) in DSen2Net.py. I may be able to commit this, need to check if it affects runs on GPU.