Does the input always require 8 images?

[andrewjong]

There are 8 images in the test_data.pkl sample. Does MGN always require 8 images as input? Or can we use less/more for worse/better results?

[bharat-b7]

You can test MGN with variable number of images.

[andrewjong]

Thanks! For others who come across this issue, I found this part of the code in test_network.py that supports this. It only extracts NUM number of images. I can just set NUM to 1.

    images = [inp[f'image_{i}'].astype('float32') for i in range(NUM)]
    J_2d = [inp[f'J_2d_{i}'].astype('float32') for i in range(NUM)]
    vertex_label = inp['vertexlabel'].astype('int64')

[Qingcsai]

@andrewjong Hi, did you succeed just by setting NUM to 1? When I set the NUM to 1 it met the following questions:

Traceback (most recent call last):
  File "test_network.py", line 191, in <module>
    pred = get_results(m, dat)
  File "test_network.py", line 55, in get_results
    out = m([images, vertex_label, J_2d])
  File "/home/***/anaconda3/envs/dirtest/lib/python3.5/site-packages/tensorflow/python/keras/engine/base_layer.py", line 592, in __call__
    outputs = self.call(inputs, *args, **kwargs)
  File "/home/***/Workspace/MultiGarmentNetwork/network/base_network.py", line 308, in call
    latent_code_offset = self.avg([q[0] for q in lat_codes])
  File "/home/***/anaconda3/envs/dirtest/lib/python3.5/site-packages/tensorflow/python/keras/engine/base_layer.py", line 538, in __call__
    self._maybe_build(inputs)
  File "/home/***/anaconda3/envs/dirtest/lib/python3.5/site-packages/tensorflow/python/keras/engine/base_layer.py", line 1603, in _maybe_build
    self.build(input_shapes)
  File "/home/***/anaconda3/envs/dirtest/lib/python3.5/site-packages/tensorflow/python/keras/utils/tf_utils.py", line 151, in wrapper
    output_shape = fn(instance, input_shape)
  File "/home/***/anaconda3/envs/dirtest/lib/python3.5/site-packages/tensorflow/python/keras/layers/merge.py", line 94, in build
    'Got ' + str(len(input_shape)) + ' inputs.')
ValueError: A merge layer should be called on a list of at least 2 inputs. Got 1 inputs.

And when I set NUM to 4 it met some other questions:

Traceback (most recent call last):
  File "test_network.py", line 209, in <module>
    m = fine_tune(m, dat, dat, display=False)
  File "test_network.py", line 124, in fine_tune
    lo = m.train(inp, out, loss_dict=losses_2d, vars2opt=vars2opt)
  File "/home/***/Workspace/MultiGarmentNetwork/network/base_network.py", line 441, in train
    loss = self.loss_model(gt_dict, out_dict, wt_dict=loss_dict)
  File "/home/***/Workspace/MultiGarmentNetwork/network/base_network.py", line 429, in loss_model
    loss[k] = tf.losses.absolute_difference(gt_dict[k], pred_dict[k], weights=wt_dict[k])
  File "/home/***/anaconda3/envs/dirtest/lib/python3.5/site-packages/tensorflow/python/ops/losses/losses_impl.py", line 279, in absolute_difference
    predictions.get_shape().assert_is_compatible_with(labels.get_shape())
  File "/home/***/anaconda3/envs/dirtest/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 1023, in assert_is_compatible_with
    raise ValueError("Shapes %s and %s are incompatible" % (self, other))
ValueError: Shapes (2, 720, 720, 3, 4) and (2, 720, 720, 3, 8) are incompatible

I don't know whether I should change some other codes or just set NUM = 4, do you have any suggestions please?

[Qingcsai]

Oh I finally solved this question, by adding this line below the dat = pkl.load'(***):

dat['rendered'] = dat['rendered'][ : , : , : , : , : NUM]

I think the reason why it works is from the bug reporting:

ValueError: Shapes (2, 720, 720, 3, 4) and (2, 720, 720, 3, 8) are incompatible

It seems like the dimention incompatible is caused from dat['rendered'], and so it works.

[Frank-Dz]

Oh I finally solved this question, by adding this line below the dat = pkl.load'(***):

dat['rendered'] = dat['rendered'][ : , : , : , : , : NUM]

I think the reason why it works is from the bug reporting:

ValueError: Shapes (2, 720, 720, 3, 4) and (2, 720, 720, 3, 8) are incompatible

It seems like the dimention incompatible is caused from dat['rendered'], and so it works.

Hi~ Thanks for your sharing. But when I set NUM to be 1 or 4 and added dat['rendered'] = dat['rendered'][ : , : , : , : , : NUM]

I still get this error:

2020-04-16 20:57:00.741211: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-04-16 20:57:00.865870: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:998] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-04-16 20:57:00.866577: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x65c2c20 executing computations on platform CUDA. Devices:
2020-04-16 20:57:00.866590: I tensorflow/compiler/xla/service/service.cc:158]   StreamExecutor device (0): GeForce RTX 2080, Compute Capability 7.5
2020-04-16 20:57:00.887257: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2592000000 Hz
2020-04-16 20:57:00.888185: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x662dda0 executing computations on platform Host. Devices:
2020-04-16 20:57:00.888204: I tensorflow/compiler/xla/service/service.cc:158]   StreamExecutor device (0): <undefined>, <undefined>
2020-04-16 20:57:00.888398: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1433] Found device 0 with properties: 
name: GeForce RTX 2080 major: 7 minor: 5 memoryClockRate(GHz): 1.095
pciBusID: 0000:01:00.0
totalMemory: 7.77GiB freeMemory: 6.98GiB
2020-04-16 20:57:00.888408: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1512] Adding visible gpu devices: 0
2020-04-16 20:57:00.889152: I tensorflow/core/common_runtime/gpu/gpu_device.cc:984] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-04-16 20:57:00.889159: I tensorflow/core/common_runtime/gpu/gpu_device.cc:990]      0 
2020-04-16 20:57:00.889164: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1003] 0:   N 
2020-04-16 20:57:00.889303: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6792 MB memory) -> physical GPU (device: 0, name: GeForce RTX 2080, pci bus id: 0000:01:00.0, compute capability: 7.5)
WARNING:tensorflow:From /home/frank/PycharmProjects/MultiGarmentNetwork/smpl/batch_lbs.py:53: div (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Deprecated in favor of operator or tf.math.divide.
2020-04-16 20:57:01.979366: I tensorflow/stream_executor/dso_loader.cc:152] successfully opened CUDA library libcublas.so.10.0 locally
Using latest checkpoint at saved_model/ckpt-26
WARNING:tensorflow:From /home/frank/Desktop/mesh-py3/my_venv/lib/python3.5/site-packages/tensorflow/python/ops/resource_variable_ops.py:642: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
Traceback (most recent call last):
  File "/home/frank/PycharmProjects/MultiGarmentNetwork/test_network.py", line 191, in <module>
    pred = get_results(m, dat)
  File "/home/frank/PycharmProjects/MultiGarmentNetwork/test_network.py", line 54, in get_results
    out = m([images, vertex_label, J_2d])
  File "/home/frank/Desktop/mesh-py3/my_venv/lib/python3.5/site-packages/tensorflow/python/keras/engine/base_layer.py", line 592, in __call__
    outputs = self.call(inputs, *args, **kwargs)
  File "/home/frank/PycharmProjects/MultiGarmentNetwork/network/base_network.py", line 309, in call
    latent_code_offset = self.avg([q[0] for q in lat_codes])
  File "/home/frank/Desktop/mesh-py3/my_venv/lib/python3.5/site-packages/tensorflow/python/keras/engine/base_layer.py", line 538, in __call__
    self._maybe_build(inputs)
  File "/home/frank/Desktop/mesh-py3/my_venv/lib/python3.5/site-packages/tensorflow/python/keras/engine/base_layer.py", line 1603, in _maybe_build
    self.build(input_shapes)
  File "/home/frank/Desktop/mesh-py3/my_venv/lib/python3.5/site-packages/tensorflow/python/keras/utils/tf_utils.py", line 151, in wrapper
    output_shape = fn(instance, input_shape)
  File "/home/frank/Desktop/mesh-py3/my_venv/lib/python3.5/site-packages/tensorflow/python/keras/layers/merge.py", line 94, in build
    'Got ' + str(len(input_shape)) + ' inputs.')
ValueError: A merge layer should be called on a list of at least 2 inputs. Got 1 inputs.

Process finished with exit code 1

I would like to ask how you handled it? Thanks in advance!

Best, Frank

[andrewjong]

For future viewers, please see my comment here to get MGN to work on single images.

[neonb88]

Hope someone else can confirm this. If Andrew's solution works for you too, please comment or at least "thumbs up" him

[Frank-Dz]

For future viewers, please see my comment here to get MGN to work on single images.

Great! Thanks!