ValueError: setting an array element with a sequence.

[danielbaak]

I'm getting the following error:

deep-anpr git:(master) ✗ python -m trace --count -C . train.py                                                                                                        [25/9700]
WARNING:tensorflow:From train.py:133: softmax_cross_entropy_with_logits (from tensorflow.python.ops.nn_ops) is deprecated and will be removed in a future version.
Instructions for updating:

Future major versions of TensorFlow will allow gradients to flow
into the labels input on backprop by default.

See tf.nn.softmax_cross_entropy_with_logits_v2.

2018-01-14 20:01:22.889672: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FM
A
2018-01-14 20:01:23.053906: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:895] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node
, so returning NUMA node zero
2018-01-14 20:01:23.054162: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1105] Found device 0 with properties: 
name: GeForce GTX 1070 major: 6 minor: 1 memoryClockRate(GHz): 1.721
pciBusID: 0000:01:00.0
totalMemory: 7.92GiB freeMemory: 6.09GiB
2018-01-14 20:01:23.054173: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1195] Creating TensorFlow device (/device:GPU:0) -> (device: 0, name: GeForce GTX 1070, pci bus id: 0000:01:00.
0, compute capability: 6.1)
2018-01-14 20:01:23.192321: E tensorflow/stream_executor/cuda/cuda_driver.cc:936] failed to allocate 7.52G (8079625216 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
2018-01-14 20:01:23.192831: E tensorflow/stream_executor/cuda/cuda_driver.cc:936] failed to allocate 6.77G (7271662592 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
2018-01-14 20:01:23.193332: E tensorflow/stream_executor/cuda/cuda_driver.cc:936] failed to allocate 6.09G (6544496128 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/lib/python2.7/trace.py", line 819, in <module>
    main()
  File "/usr/lib/python2.7/trace.py", line 807, in main
    t.runctx(code, globs, globs)
  File "/usr/lib/python2.7/trace.py", line 513, in runctx
    exec cmd in globals, locals
  File "train.py", line 261, in <module>
    initial_weights=init_weights)
  File "train.py", line 234, in train
    do_batch()
  File "train.py", line 213, in do_batch
    sess.run(train_step, feed_dict={x: batch_xs, y_: batch_ys})
  File "/home/u/Projects/deep-anpr/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 895, in run
    run_metadata_ptr)
  File "/home/u/Projects/deep-anpr/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1097, in _run
    np_val = np.asarray(subfeed_val, dtype=subfeed_dtype)
  File "/home/u/.local/lib/python2.7/site-packages/numpy/core/numeric.py", line 531, in asarray
    return array(a, dtype, copy=False, order=order)
ValueError: setting an array element with a sequence.

it seems that the sess.run() inside of do_batch() is complaining about being passed malformed data. I can't so far figure out the source of the error and am using the same data as explained in the blog post.

Also, i did fix the error regarding sigmoid_cross_entropy_with_logits that was discussed in other issues before getting to this problem.

[danielbaak]

Nevermind, I snuck in an annoying bug while updating the code to run with tensorflow 1.5.

[SarikGhazarian]

Daniel,

It seems I have the same problem. When I was running my code with Tensorflow 1.9.0, I didn't have that problem, but this issue comes with Tensorflow 1.5.0

[sintetico82]

@danielbaak how did you resolve that? I must run tensorflow 1.5 because of CPU compatibility but i have that bug.

[madevelascom]

I tried with 1.9, 1.10, 1.14 and doesn't work