InvalidArgumentError: Nan in summary histogram for: pred

Hi, Dear lilian thanks for your code and tutorial. I'm running your code on python 3 environment with the following setup:

tensorflow==1.8.0 pandas==0.24.1 scikit-learn==0.17.1 scipy==1.1.0 numpy==1.15.2 requests==2.9.1

This command works. /usr/bin/python3.5 main.py --stock_symbol=SP500 --train --input_size=1 --lstm_size=128 --max_epoch=50

/usr/bin/python3.5 main.py --stock_count=100 --train --input_size=1 --lstm_size=128 --max_epoch=50 --embed_size=8

When I run above command it gives error with the following output:

Start training for stocks: ['AAPL', 'GOOGL', 'GOOG', 'MSFT', 'AMZN', 'FB', 'JPM', 'JNJ', 'XOM', 'BAC', 'WMT', 'WFC', 'V', 'T', 'HD', 'CVX', 'UNH', 'INTC', 'PFE', 'VZ', 'PG', 'BA', 'ORCL', 'CSCO', 'C', 'KO', 'MA', 'CMCSA', 'ABBV', 'DWDP', 'PEP', 'DIS', 'PM', 'MRK', 'IBM', 'MMM', 'NVDA', 'GE', 'MCD', 'AMGN', 'MO', 'NFLX', 'HON', 'MDT', 'GILD', 'NKE', 'UTX', 'BMY', 'ABT', 'UNP', 'TXN', 'ACN', 'LMT', 'MS', 'GS', 'SLB', 'UPS', 'QCOM', 'ADBE', 'AVGO', 'CAT', 'USB', 'PYPL', 'KHC', 'CHTR', 'BLK', 'LLY', 'TMO', 'LOW', 'COST', 'AXP', 'CRM', 'SBUX', 'CVS', 'CELG', 'PNC', 'WBA', 'SCHW', 'NEE', 'BIIB', 'CB', 'FDX', 'DHR', 'FOX', 'MDLZ', 'COP', 'GD', 'CL', 'GM', 'ANTM', 'EOG', 'AMT', 'RTN', 'NOC', 'SYK', 'AGN', 'BK', 'ITW', 'CME', 'AIG']
Step:1 [Epoch:0] [Learning rate: 0.001000] train_loss:0.523090 test_loss:0.378728

Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1322, in _do_call
    return fn(*args)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1307, in _run_fn
    options, feed_dict, fetch_list, target_list, run_metadata)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1409, in _call_tf_sessionrun
    run_metadata)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Nan in summary histogram for: pred
     [[Node: pred = HistogramSummary[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"](pred/tag, add/_65)]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "main.py", line 114, in <module>
    tf.app.run()
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/platform/app.py", line 126, in run
    _sys.exit(main(argv))
  File "main.py", line 107, in main
    rnn_model.train(stock_data_list, FLAGS)
  File "/home/myuser/PycharmProjects/project1/stock-rnn-master/model_rnn.py", line 229, in train
    [self.loss, self.optim, self.merged_sum], train_data_feed)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 900, in run
    run_metadata_ptr)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1135, in _run
    feed_dict_tensor, options, run_metadata)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1316, in _do_run
    run_metadata)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1335, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Nan in summary histogram for: pred
     [[Node: pred = HistogramSummary[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"](pred/tag, add/_65)]]

Caused by op 'pred', defined at:
  File "main.py", line 114, in <module>
    tf.app.run()
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/platform/app.py", line 126, in run
    _sys.exit(main(argv))
  File "main.py", line 94, in main
    embed_size=FLAGS.embed_size,
  File "/home/myuser/PycharmProjects/project1/stock-rnn-master/model_rnn.py", line 53, in __init__
    self.build_graph()
  File "/home/myuser/PycharmProjects/project1/stock-rnn-master/model_rnn.py", line 119, in build_graph
    self.pred_summ = tf.summary.histogram("pred", self.pred)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/summary/summary.py", line 203, in histogram
    tag=tag, values=values, name=scope)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/gen_logging_ops.py", line 283, in histogram_summary
    "HistogramSummary", tag=tag, values=values, name=name)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 3392, in create_op
    op_def=op_def)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 1718, in __init__
    self._traceback = self._graph._extract_stack()  # pylint: disable=protected-access

InvalidArgumentError (see above for traceback): Nan in summary histogram for: pred
     [[Node: pred = HistogramSummary[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"](pred/tag, add/_65)]]

Could you please help me on this issue? Is it related to my dataset?

lilianweng / stock-rnn

InvalidArgumentError: Nan in summary histogram for: pred #31