ValueError: If printing histograms, validation_data must be provided, and cannot be a generator.

[developeralgo8888]

I am running

TF = 1.6 , Tensorboard = 1.5.0 and Keras 2.1.5 but i am getting this error below:

Developer@DEEPGPU:~/Downloads/nntimeseries-master/nnts/models# python ./CNN.py --dataset=artificial .... ... ... .. ... Total model parameters: 3451 WARNING:tensorflow:From /home/anaconda3/lib/python3.6/site-packages/tensorflow/contrib/learn/python/learn/datasets/base.py:198: retry (from tensorflow.contrib.learn.python.learn.datasets.base) is deprecated and will be removed in a future version. Instructions for updating: Use the retry module or similar alternatives. Epoch 1/1000 ---current learning rate: 0.00100000 124/124 [==============================] - 13s 103ms/step - loss: 1.2770 - val_loss: 2.7545

---current best val_loss: 2.75454 960/960 [==============================] - 0s 113us/step

--- test_loss: 2.290810 Traceback (most recent call last): File "./CNN.py", line 109, in runner.run(CNNmodel, log=log, limit=1) File "/root/Downloads/nntimeseries-master/nnts/utils.py", line 130, in run model_results, nn = model.run() File "/root/Downloads/nntimeseries-master/nnts/utils.py", line 287, in run verbose=self.verbose File "/home/anaconda3/lib/python3.6/site-packages/keras/legacy/interfaces.py", line 91, in wrapper return func(*args, **kwargs) File "/home/anaconda3/lib/python3.6/site-packages/keras/engine/training.py", line 2262, in fit_generator callbacks.on_epoch_end(epoch, epoch_logs) File "/home/anaconda3/lib/python3.6/site-packages/keras/callbacks.py", line 77, in on_epoch_end callback.on_epoch_end(epoch, logs) File "/home/anaconda3/lib/python3.6/site-packages/keras/callbacks.py", line 820, in on_epoch_end raise ValueError('If printing histograms, validation_data must be ' ValueError: If printing histograms, validation_data must be provided, and cannot be a generator.

[mbinkowski]

The code seems to try to produce histograms, which is not intended. I believe that this is caused by some default values for histogram arguments in keras that differ between versions.

Unless you're working with very large dataset, a simple fix you can try would be to change the validation_data argument at line 285 of nnts/utils.py from validation_data=self.G.gen('valid', func=self.io_func, shuffle=self.shuffle) to validation_data=next(self.G.gen('valid', func=self.io_func, shuffle=self.shuffle))

[developeralgo8888]

Made the above changes but now i am getting this error:

developer@DEEPGPU:~/Downloads/nntimeseries-master/nnts/models#python ./CNN.py --dataset=artificial Working directory: '/developer/Downloads/nntimeseries-master/' /home/anaconda3/lib/python3.6/site-packages/h5py/init.py:36: FutureWarning: Conversion of the second argument of issubdtype from float to np.floating is deprecated. In future, it will be treated as np.float64 == np.dtype(float).type. from ._conv import register_converters as _register_converters Using TensorFlow backend. no save_file specified datasets found: ['data/artificialET1SS1n10000S16.csv', 'data/artificialET1SS0n10000S16.csv', 'data/artificialET1SS1n10000S64.csv', 'data/artificialET1SS0n10000S64.csv'] results will be saved in 'results/artificial_CNN.pkl' Found 0 (< limit = 1) computed results for the setting. As yet, for this configuration: success: 0, errors: 0 verbose: 1 train_share: (0.8, 0.9, 1.0) input_length: 60 output_length: 1 batch_size: 64 objective: regr diffs: False target_cols: default patience: 5 reduce_nb: 2 lr: 0.001 clipnorm: 1.0 dropout: 0.2 norm: 10 filters: 16 act: leakyrelu kernelsize: [1, 3] poolsize: 2 layers_no: 10 maxpooling: 3 resnet: False using <class 'main.CNNmodel'> to build the model using <class 'nnts.artificial.ArtificialGenerator'> to draw samples WARNING:tensorflow:From /home/anaconda3/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py:497: calling conv1d (from tensorflow.python.ops.nn_ops) with data_format=NHWC is deprecated and will be removed in a future version. Instructions for updating: NHWC for data_format is deprecated, use NWC instead 2018-03-16 12:16:57.676120: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1344] Found device 0 with properties: name: GeForce GTX TITAN X major: 5 minor: 2 memoryClockRate(GHz): 1.076 pciBusID: 0000:05:00.0 totalMemory: 11.93GiB freeMemory: 11.81GiB 2018-03-16 12:16:57.676178: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1423] Adding visible gpu devices: 0 2018-03-16 12:16:58.055774: I tensorflow/core/common_runtime/gpu/gpu_device.cc:911] Device interconnect StreamExecutor with strength 1 edge matrix: 2018-03-16 12:16:58.055816: I tensorflow/core/common_runtime/gpu/gpu_device.cc:917] 0 2018-03-16 12:16:58.055823: I tensorflow/core/common_runtime/gpu/gpu_device.cc:930] 0: N 2018-03-16 12:16:58.056158: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1041] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 11436 MB memory) -> physical GPU (device: 0, name: GeForce GTX TITAN X, pci bus id: 0000:05:00.0, compute capability: 5.2) Total model parameters: 3451 WARNING:tensorflow:From /home/anaconda3/lib/python3.6/site-packages/tensorflow/contrib/learn/python/learn/datasets/base.py:198: retry (from tensorflow.contrib.learn.python.learn.datasets.base) is deprecated and will be removed in a future version. Instructions for updating: Use the retry module or similar alternatives. Epoch 1/1000 ---current learning rate: 0.00100000 124/124 [==============================] - 11s 89ms/step - loss: 1.0252 - val_loss: 0.2063

---current best val_loss: 0.20626 960/960 [==============================] - 0s 92us/step

--- test_loss: 0.063176 Traceback (most recent call last): File "/home/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1327, in _do_call return fn(*args) File "/home/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1312, in _run_fn options, feed_dict, fetch_list, target_list, run_metadata) File "/home/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1420, in _call_tf_sessionrun status, run_metadata) File "/home/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py", line 516, in exit c_api.TF_GetCode(self.status.status)) tensorflow.python.framework.errors_impl.InvalidArgumentError: Tensor must be 4-D with last dim 1, 3, or 4, not [1,1] [[Node: 10_conv/bias_0_1 = ImageSummaryT=DT_FLOAT, bad_color=Tensor<type: uint8 shape: [4] values: 255 0 0...>, max_images=3, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "./CNN.py", line 109, in runner.run(CNNmodel, log=log, limit=1) File "/developer/Downloads/nntimeseries-master/nnts/utils.py", line 130, in run model_results, nn = model.run() File "/developer/Downloads/nntimeseries-master/nnts/utils.py", line 288, in run verbose=self.verbose File "/home/anaconda3/lib/python3.6/site-packages/keras/legacy/interfaces.py", line 91, in wrapper return func(*args, **kwargs) File "/home/anaconda3/lib/python3.6/site-packages/keras/engine/training.py", line 2262, in fit_generator callbacks.on_epoch_end(epoch, epoch_logs) File "/home/anaconda3/lib/python3.6/site-packages/keras/callbacks.py", line 77, in on_epoch_end callback.on_epoch_end(epoch, logs) File "/home/anaconda3/lib/python3.6/site-packages/keras/callbacks.py", line 846, in on_epoch_end result = self.sess.run([self.merged], feed_dict=feed_dict) File "/home/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 905, in run run_metadata_ptr) File "/home/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1140, in _run feed_dict_tensor, options, run_metadata) File "/home/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1321, in _do_run run_metadata) File "/home/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1340, in _do_call raise type(e)(node_def, op, message) tensorflow.python.framework.errors_impl.InvalidArgumentError: Tensor must be 4-D with last dim 1, 3, or 4, not [1,1] [[Node: 10_conv/bias_0_1 = ImageSummaryT=DT_FLOAT, bad_color=Tensor<type: uint8 shape: [4] values: 255 0 0...>, max_images=3, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]

Caused by op '10_conv/bias_0_1', defined at: File "./CNN.py", line 109, in runner.run(CNNmodel, log=log, limit=1) File "/developer/Downloads/nntimeseries-master/nnts/utils.py", line 130, in run model_results, nn = model.run() File "/developer/Downloads/nntimeseries-master/nnts/utils.py", line 288, in run verbose=self.verbose File "/home/anaconda3/lib/python3.6/site-packages/keras/legacy/interfaces.py", line 91, in wrapper return func(*args, **kwargs) File "/home/anaconda3/lib/python3.6/site-packages/keras/engine/training.py", line 2133, in fit_generator callbacks.set_model(callback_model) File "/home/anaconda3/lib/python3.6/site-packages/keras/callbacks.py", line 52, in set_model callback.set_model(model) File "/developer/Downloads/nntimeseries-master/nnts/keras_utils.py", line 370, in set_model tf.summary.image(weight.name, w_img) File "/home/anaconda3/lib/python3.6/site-packages/tensorflow/python/summary/summary.py", line 156, in image tag=tag, tensor=tensor, max_images=max_outputs, name=scope) File "/home/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/gen_logging_ops.py", line 387, in image_summary bad_color=bad_color, name=name) File "/home/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper op_def=op_def) File "/home/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3290, in create_op op_def=op_def) File "/home/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1654, in init self._traceback = self._graph._extract_stack() # pylint: disable=protected-access

InvalidArgumentError (see above for traceback): Tensor must be 4-D with last dim 1, 3, or 4, not [1,1] [[Node: 10_conv/bias_0_1 = ImageSummaryT=DT_FLOAT, bad_color=Tensor<type: uint8 shape: [4] values: 255 0 0...>, max_images=3, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]

On Fri, Mar 16, 2018 at 4:53 AM, Mikołaj Bińkowski <notifications@github.com

wrote:

The code seems to try to produce histograms, which is not intended. I believe that this is caused by some default values for histogram arguments in keras that differ between versions.

Unless you're working with very large dataset, a simple fix you can try would be to change the validation_data argument at line 285 of nnts/utils.py from validation_data=self.G.gen('valid', func=self.io_func, shuffle=self.shuffle) to validation_data=next(self.G.gen('valid', func=self.io_func, shuffle=self.shuffle))

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/mbinkowski/nntimeseries/issues/4#issuecomment-373645271, or mute the thread https://github.com/notifications/unsubscribe-auth/AJ0O0oIoy4vKsHZChKnd9pjydXw7VHcyks5te32ggaJpZM4StL3P .

[sorin-pascu]

I had the same issue (Tensor must be 4-D with last dim 1, 3, or 4, not [1,1]) on Windows and tried it on Linux (Mikołaj might use Mac or Linux), it worked. Not sure if it's important - I tried to use exact the same Python lib-versions, so it worked with tensorflow 1.2.0rc2 and
keras==2.0.4 # not 2.0.2 numpy==1.12.1 # not 1.12.12 pandas==0.19.2 h5py==2.7.0 # statt 2.6.0 seaborn==0.7.1 # 0.7.1 (June 2016)

Good luck, Sorin

[masaru-tsuruta]

I had faced the same issue. To run the program, I changed the below line. Therefore, I can run the process. From my understanding, tensorboard is not used by this change.

tensorboard argument at line 273 of nnts/utils.py from log_dir=tb_dir, histogram_freq=1, write_images=True to log_dir=tb_dir, histogram_freq=0, write_images=True