search

zsyzzsoft / co-mod-gan

[ICLR 2021, Spotlight] Large Scale Image Completion via Co-Modulated Generative Adversarial Networks
Other
445 stars 67 forks source link

Creating custom datasets throw ValueError: axes don't match array #10

Closed karynaur closed 3 years ago

karynaur commented 3 years ago

I was trying to create custom dataset from dataset_tools/create_from_images.py but even after ensuring all images are of same size (512,512,3) it throws the error:

Processes created.
WARNING:tensorflow:From /content/co-mod-gan/dataset_tools/tfrecord_utils.py:36: The name tf.python_io.TFRecordOptions is deprecated. Please use tf.io.TFRecordOptions instead.

WARNING:tensorflow:From /content/co-mod-gan/dataset_tools/tfrecord_utils.py:36: The name tf.python_io.TFRecordCompressionType is deprecated. Please use tf.compat.v1.python_io.TFRecordCompressionType instead.

WARNING:tensorflow:From /content/co-mod-gan/dataset_tools/tfrecord_utils.py:38: The name tf.python_io.TFRecordWriter is deprecated. Please use tf.io.TFRecordWriter instead.

Processing training images...
./data/training
100% 5541/5541 [00:00<00:00, 76086.06it/s]
Process Process-3:
Traceback (most recent call last):
  File "/usr/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/usr/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "dataset_tools/create_from_images.py", line 30, in worker
    img = np.asarray(img).transpose([2, 0, 1])
ValueError: axes don't match array
Process Process-4:
Traceback (most recent call last):
  File "/usr/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/usr/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "dataset_tools/create_from_images.py", line 30, in worker
    img = np.asarray(img).transpose([2, 0, 1])
ValueError: axes don't match array
  0% 0/5541 [00:00<?, ?it/s]Process Process-1:
Traceback (most recent call last):
  File "/usr/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/usr/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "dataset_tools/create_from_images.py", line 30, in worker
    img = np.asarray(img).transpose([2, 0, 1])
ValueError: axes don't match array
Process Process-2:
Traceback (most recent call last):
  File "/usr/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/usr/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "dataset_tools/create_from_images.py", line 30, in worker
    img = np.asarray(img).transpose([2, 0, 1])
ValueError: axes don't match array
Process Process-5:
Process Process-7:
Process Process-8:
Traceback (most recent call last):
Traceback (most recent call last):
  File "/usr/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/usr/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "dataset_tools/create_from_images.py", line 30, in worker
    img = np.asarray(img).transpose([2, 0, 1])
ValueError: axes don't match array
Traceback (most recent call last):
  File "/usr/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/usr/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "dataset_tools/create_from_images.py", line 30, in worker
    img = np.asarray(img).transpose([2, 0, 1])
ValueError: axes don't match array
Process Process-6:
  File "/usr/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/usr/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "dataset_tools/create_from_images.py", line 30, in worker
    img = np.asarray(img).transpose([2, 0, 1])
ValueError: axes don't match array
Traceback (most recent call last):
  File "/usr/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/usr/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "dataset_tools/create_from_images.py", line 30, in worker
    img = np.asarray(img).transpose([2, 0, 1])
ValueError: axes don't match array
zsyzzsoft commented 3 years ago

I think maybe there are grayscale images causing the error. Please try the updated code which correctly handles grayscale images.

karynaur commented 3 years ago

It works! Thank you so much! Closing the issue

karynaur commented 3 years ago

I was wondering if it is possible to train the network in Google Colab. I tried changing the args to --total-kimg 1 --num-gpus=1 but colab with a P100 GPU and 16 gb RAM stops abruptly with a ^C. Im sorry for the silly question :)

zsyzzsoft commented 3 years ago

I was wondering if it is possible to train the network in Google Colab. I tried changing the args to --total-kimg 1 --num-gpus=1 but colab with a P100 GPU and 16 gb RAM stops abruptly with a ^C. Im sorry for the silly question :)

I haven't tried Colab... Maybe it's interrupted from the device side?