search

Hyeokreal / Actor-Critic-Continuous-Keras

Keras Implementation of the continuous control with actor-critic, a3c
13 stars 3 forks source link

tensorflow.python.framework.errors_impl.NotFoundError #1

Open Tomas0413 opened 6 years ago

Tomas0413 commented 6 years ago

@Hyeokreal which Tensorflow and Keras version do yo use?

I tried running a3c_continuous.py, but I get these errors:

2018-02-16 11:55:39.292348: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.2 AVX
[2018-02-16 11:55:39,904] Making new env: Pendulum-v0
[2018-02-16 11:55:39,911] Making new env: Pendulum-v0
[2018-02-16 11:55:39,923] Making new env: Pendulum-v0
[2018-02-16 11:55:39,963] Making new env: Pendulum-v0
[2018-02-16 11:55:39,979] Making new env: Pendulum-v0
[2018-02-16 11:55:39,996] Making new env: Pendulum-v0
[2018-02-16 11:55:40,024] Making new env: Pendulum-v0
[2018-02-16 11:55:40,088] Making new env: Pendulum-v0
Exception in thread Thread-3:
Traceback (most recent call last):
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1350, in _do_call
    return fn(*args)
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1329, in _run_fn
    status, run_metadata)
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py", line 473, in __exit__
    c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.NotFoundError: FetchOutputs node IsVariableInitialized_1/IsVariableInitialized_2/IsVariableInitialized_46:0: not found

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/anaconda/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "a3c_continuous.py", line 180, in run
    action = self.get_action(state)
  File "a3c_continuous.py", line 244, in get_action
    mu, sigma_sq = self.actor.predict(np.reshape(state, [1, self.state_size]))
  File "/anaconda/lib/python3.6/site-packages/keras/engine/training.py", line 1842, in predict
    verbose=verbose, steps=steps)
  File "/anaconda/lib/python3.6/site-packages/keras/engine/training.py", line 1337, in _predict_loop
    batch_outs = f(ins_batch)
  File "/anaconda/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py", line 2473, in __call__
    session = get_session()
  File "/anaconda/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py", line 189, in get_session
    [tf.is_variable_initialized(v) for v in candidate_vars])
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 895, in run
    run_metadata_ptr)
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1128, in _run
    feed_dict_tensor, options, run_metadata)
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1344, in _do_run
    options, run_metadata)
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1363, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.NotFoundError: FetchOutputs node IsVariableInitialized_1/IsVariableInitialized_2/IsVariableInitialized_46:0: not found

Exception in thread Thread-7:
Traceback (most recent call last):
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1350, in _do_call
    return fn(*args)
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1329, in _run_fn
    status, run_metadata)
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py", line 473, in __exit__
    c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.NotFoundError: FetchOutputs node IsVariableInitialized_8/IsVariableInitialized_3/IsVariableInitialized_2/IsVariableInitialized_43:0: not found

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/anaconda/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "a3c_continuous.py", line 180, in run
    action = self.get_action(state)
  File "a3c_continuous.py", line 244, in get_action
    mu, sigma_sq = self.actor.predict(np.reshape(state, [1, self.state_size]))
  File "/anaconda/lib/python3.6/site-packages/keras/engine/training.py", line 1842, in predict
    verbose=verbose, steps=steps)
  File "/anaconda/lib/python3.6/site-packages/keras/engine/training.py", line 1337, in _predict_loop
    batch_outs = f(ins_batch)
  File "/anaconda/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py", line 2473, in __call__
    session = get_session()
  File "/anaconda/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py", line 189, in get_session
    [tf.is_variable_initialized(v) for v in candidate_vars])
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 895, in run
    run_metadata_ptr)
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1128, in _run
    feed_dict_tensor, options, run_metadata)
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1344, in _do_run
    options, run_metadata)
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1363, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.NotFoundError: FetchOutputs node IsVariableInitialized_8/IsVariableInitialized_3/IsVariableInitialized_2/IsVariableInitialized_43:0: not found

Exception in thread Thread-5:
Traceback (most recent call last):
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1350, in _do_call
    return fn(*args)
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1329, in _run_fn
    status, run_metadata)
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py", line 473, in __exit__
    c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.NotFoundError: FetchOutputs node IsVariableInitialized_8/IsVariableInitialized_6/IsVariableInitialized_51:0: not found

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/anaconda/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "a3c_continuous.py", line 180, in run
    action = self.get_action(state)
  File "a3c_continuous.py", line 244, in get_action
    mu, sigma_sq = self.actor.predict(np.reshape(state, [1, self.state_size]))
  File "/anaconda/lib/python3.6/site-packages/keras/engine/training.py", line 1842, in predict
    verbose=verbose, steps=steps)
  File "/anaconda/lib/python3.6/site-packages/keras/engine/training.py", line 1337, in _predict_loop
    batch_outs = f(ins_batch)
  File "/anaconda/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py", line 2473, in __call__
    session = get_session()
  File "/anaconda/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py", line 189, in get_session
    [tf.is_variable_initialized(v) for v in candidate_vars])
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 895, in run
    run_metadata_ptr)
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1128, in _run
    feed_dict_tensor, options, run_metadata)
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1344, in _do_run
    options, run_metadata)
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1363, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.NotFoundError: FetchOutputs node IsVariableInitialized_8/IsVariableInitialized_6/IsVariableInitialized_51:0: not found

Exception in thread Thread-4:
Traceback (most recent call last):
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1350, in _do_call
    return fn(*args)
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1329, in _run_fn
    status, run_metadata)
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py", line 473, in __exit__
    c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.NotFoundError: FetchOutputs node IsVariableInitialized_36/IsVariableInitialized_11/IsVariableInitialized_6:0: not found

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/anaconda/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "a3c_continuous.py", line 180, in run
    action = self.get_action(state)
  File "a3c_continuous.py", line 244, in get_action
    mu, sigma_sq = self.actor.predict(np.reshape(state, [1, self.state_size]))
  File "/anaconda/lib/python3.6/site-packages/keras/engine/training.py", line 1842, in predict
    verbose=verbose, steps=steps)
  File "/anaconda/lib/python3.6/site-packages/keras/engine/training.py", line 1337, in _predict_loop
    batch_outs = f(ins_batch)
  File "/anaconda/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py", line 2473, in __call__
    session = get_session()
  File "/anaconda/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py", line 189, in get_session
    [tf.is_variable_initialized(v) for v in candidate_vars])
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 895, in run
    run_metadata_ptr)
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1128, in _run
    feed_dict_tensor, options, run_metadata)
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1344, in _do_run
    options, run_metadata)
  File "/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1363, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.NotFoundError: FetchOutputs node IsVariableInitialized_36/IsVariableInitialized_11/IsVariableInitialized_6:0: not found

episode:  1 / score :  [-168.84377024] / step :  200
episode:  2 / score :  [-161.87116017] / step :  200
episode:  3 / score :  [-159.24898023] / step :  200
episode:  4 / score :  [-187.5850562] / step :  200
episode:  5 / score :  [-167.4228075] / step :  200
episode:  6 / score :  [-185.07506142] / step :  200
episode:  7 / score :  [-169.44206692] / step :  200
episode:  8 / score :  [-183.16870669] / step :  200
episode:  9 / score :  [-137.96890288] / step :  200
episode:  10 / score :  [-171.25066072] / step :  200
episode:  11 / score :  [-151.99627662] / step :  200
episode:  12 / score :  [-121.34443564] / step :  200
^CTraceback (most recent call last):
  File "a3c_continuous.py", line 267, in <module>
    global_agent.train()
  File "a3c_continuous.py", line 137, in train
    time.sleep(20)
KeyboardInterrupt
episode:  13 / score :  [-177.44202396] / step :  200
episode:  14 / score :  [-177.48592821] / step :  200
episode:  15 / score :  [-167.54128117] / step :  200
episode:  16 / score :  [-187.03103497] / step :  200
episode:  17 / score :  [-175.32132064] / step :  200
episode:  18 / score :  [-124.67301435] / step :  200
episode:  19 / score :  [-177.01564837] / step :  200
episode:  20 / score :  [-142.00295605] / step :  200
^CException ignored in: <module 'threading' from '/anaconda/lib/python3.6/threading.py'>
Traceback (most recent call last):
  File "/anaconda/lib/python3.6/threading.py", line 1294, in _shutdown
    t.join()
  File "/anaconda/lib/python3.6/threading.py", line 1056, in join
    self._wait_for_tstate_lock()
  File "/anaconda/lib/python3.6/threading.py", line 1072, in _wait_for_tstate_lock
    elif lock.acquire(block, timeout):
KeyboardInterrupt
^C

This is my environment:

$ pip show keras
Name: Keras
Version: 2.1.4
Summary: Deep Learning for humans
Home-page: https://github.com/keras-team/keras
Author: Francois Chollet
Author-email: francois.chollet@gmail.com
License: MIT
Location: /anaconda/lib/python3.6/site-packages
Requires: scipy, pyyaml, six, numpy
$ pip show tensorflow
Name: tensorflow
Version: 1.5.0
Summary: TensorFlow helps the tensors flow
Home-page: https://www.tensorflow.org/
Author: Google Inc.
Author-email: opensource@google.com
License: Apache 2.0
Location: /anaconda/lib/python3.6/site-packages
Requires: tensorflow-tensorboard, protobuf, numpy, wheel, six, absl-py
Hyeokreal commented 6 years ago

Hi , @Tomas0413 . I'm sorry I did not have Readme with enough information. I used python 3.5, tensorflow 1.0.0, Keras 2.0.3 . Also, only a2c_continuous.py would work. a3c_continous.py is on progress.

Thanks

Tomas0413 commented 6 years ago

@Hyeokreal thanks for a quick answer! I'll keep an eye on a3c_continous.py then!

I liked your code which I think you maintain together with few more people on: https://github.com/rlcode/reinforcement-learning

I tried to find a good and working example of A3C for continuous action space that also uses Keras. I didn't find one yet, yours was close!

However today I found this: https://github.com/tensorlayer/tensorlayer/blob/master/example/tutorial_bipedalwalker_a3c_continuous_action.py from TensorLayer (https://github.com/tensorlayer/tensorlayer).

What's your opinion of TensorLayer?