search

rail-berkeley / softlearning

Softlearning is a reinforcement learning framework for training maximum entropy policies in continuous domains. Includes the official implementation of the Soft Actor-Critic algorithm.
https://sites.google.com/view/sac-and-applications
Other
1.2k stars 239 forks source link

'FeedforwardGaussianPolicy' object has no attribute '_Serializable__initialize' #107

Open ZhanPython opened 5 years ago

ZhanPython commented 5 years ago

The output from the console is:

/home/huixin/anaconda3/envs/softlearning/bin/python /home/huixin/PycharmProjects/softlearning/examples/development/main.py /home/huixin/anaconda3/envs/softlearning/lib/python3.7/site-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.25.1) or chardet (3.0.4) doesn't match a supported version! RequestsDependencyWarning) Warning: dm_control package not found. Runpip install git+https://github.com/deepmind/dm_control.git` to use dm_control environments. 2019-09-16 20:49:16,811 INFO node.py:498 -- Process STDOUT and STDERR is being redirected to /tmp/ray/session_2019-09-16_20-49-16_810674_7958/logs. 2019-09-16 20:49:16,925 INFO services.py:409 -- Waiting for redis server at 127.0.0.1:63967 to respond... 2019-09-16 20:49:17,047 INFO services.py:409 -- Waiting for redis server at 127.0.0.1:56793 to respond... 2019-09-16 20:49:17,049 INFO services.py:806 -- Starting Redis shard with 3.34 GB max memory. 2019-09-16 20:49:17,089 INFO node.py:512 -- Process STDOUT and STDERR is being redirected to /tmp/ray/session_2019-09-16_20-49-16_810674_7958/logs. 2019-09-16 20:49:17,090 INFO services.py:1442 -- Starting the Plasma object store with 5.01 GB memory using /dev/shm. 2019-09-16 20:49:17,217 INFO tune.py:65 -- Did not find checkpoint file in /home/huixin/ray_results/gym/Pendulum/v0/2019-09-16T20-49-16-2019-09-16T20-49-16. 2019-09-16 20:49:17,217 INFO tune.py:232 -- Starting a new experiment. 2019-09-16 20:49:17,218 INFO web_server.py:241 -- Starting Tune Server... == Status == Using FIFO scheduling algorithm. Resources requested: 0/16 CPUs, 0/1 GPUs Memory usage on this node: 11.3/16.7 GB

WARNING: Logging before flag parsing goes to stderr. W0916 20:49:17.326717 140070514243008 deprecation_wrapper.py:119] From /home/huixin/anaconda3/envs/softlearning/lib/python3.7/site-packages/ray/tune/logger.py:131: The name tf.VERSION is deprecated. Please use tf.version.VERSION instead.

W0916 20:49:17.327078 140070514243008 deprecation_wrapper.py:119] From /home/huixin/anaconda3/envs/softlearning/lib/python3.7/site-packages/ray/tune/logger.py:136: The name tf.summary.FileWriter is deprecated. Please use tf.compat.v1.summary.FileWriter instead.

== Status == Using FIFO scheduling algorithm. Resources requested: 16/16 CPUs, 0/1 GPUs Memory usage on this node: 11.4/16.7 GB Result logdir: /home/huixin/ray_results/gym/Pendulum/v0/2019-09-16T20-49-16-2019-09-16T20-49-16 Number of trials: 1 ({'RUNNING': 1}) RUNNING trials:

(pid=8062) /home/huixin/anaconda3/envs/softlearning/lib/python3.7/site-packages/requests/init.py:91: RequestsDependencyWarning: urllib3 (1.25.1) or chardet (3.0.4) doesn't match a supported version! (pid=8062) RequestsDependencyWarning) (pid=8062) Warning: dm_control package not found. Run pip install git+https://github.com/deepmind/dm_control.git to use dm_control environments. (pid=8062) 2019-09-16 20:49:20.153718: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 AVX512F FMA (pid=8062) 2019-09-16 20:49:20.179777: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 3800000000 Hz (pid=8062) 2019-09-16 20:49:20.181510: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x563a01abd7f0 executing computations on platform Host. Devices: (pid=8062) 2019-09-16 20:49:20.181557: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): , (pid=8062) WARNING: Logging before flag parsing goes to stderr. (pid=8062) W0916 20:49:20.183301 140071170913728 deprecation_wrapper.py:119] From /home/huixin/PycharmProjects/softlearning/examples/development/main.py:33: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. (pid=8062) (pid=8062) Using seed 898 (pid=8062) W0916 20:49:20.199773 140071170913728 deprecation.py:506] From /home/huixin/anaconda3/envs/softlearning/lib/python3.7/site-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.init (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version. (pid=8062) Instructions for updating: (pid=8062) Call initializer instance with the dtype argument instead of passing it to the constructor 2019-09-16 20:49:20,403 ERROR trial_runner.py:487 -- Error processing event. Traceback (most recent call last): File "/home/huixin/anaconda3/envs/softlearning/lib/python3.7/site-packages/ray/tune/trial_runner.py", line 436, in _process_trial result = self.trial_executor.fetch_result(trial) File "/home/huixin/anaconda3/envs/softlearning/lib/python3.7/site-packages/ray/tune/ray_trial_executor.py", line 323, in fetch_result result = ray.get(trial_future[0]) File "/home/huixin/anaconda3/envs/softlearning/lib/python3.7/site-packages/ray/worker.py", line 2189, in get raise value ray.exceptions.RayTaskError: ray_ExperimentRunner:train() (pid=8062, host=huixin-Alienware-Area-51-R5) File "/home/huixin/anaconda3/envs/softlearning/lib/python3.7/site-packages/ray/tune/trainable.py", line 151, in train result = self._train() File "/home/huixin/PycharmProjects/softlearning/examples/development/main.py", line 82, in _train self._build() File "/home/huixin/PycharmProjects/softlearning/examples/development/main.py", line 59, in _build variant, training_environment) File "/home/huixin/PycharmProjects/softlearning/softlearning/policies/utils.py", line 75, in get_policy_from_variant return get_policy_from_params(policy_params, args, kwargs) File "/home/huixin/PycharmProjects/softlearning/softlearning/policies/utils.py", line 68, in get_policy_from_params kwargs) File "/home/huixin/PycharmProjects/softlearning/softlearning/policies/utils.py", line 10, in get_gaussian_policy policy = FeedforwardGaussianPolicy(args, **kwargs) File "/home/huixin/PycharmProjects/softlearning/softlearning/policies/gaussian_policy.py", line 226, in init self._Serializable__initialize(locals()) AttributeError: 'FeedforwardGaussianPolicy' object has no attribute '_Serializable__initialize'

2019-09-16 20:49:20,405 INFO ray_trial_executor.py:187 -- Destroying actor for trial id=5bf385e2-seed=898. If your trainable is slow to initialize, consider setting reuse_actors=True to reduce actor creation overheads. 2019-09-16 20:49:20,408 INFO trial_runner.py:524 -- Attempting to recover trial state from last checkpoint. (pid=8483) /home/huixin/anaconda3/envs/softlearning/lib/python3.7/site-packages/requests/init.py:91: RequestsDependencyWarning: urllib3 (1.25.1) or chardet (3.0.4) doesn't match a supported version! (pid=8483) RequestsDependencyWarning) (pid=8483) Warning: dm_control package not found. Run pip install git+https://github.com/deepmind/dm_control.git to use dm_control environments. (pid=8483) Using seed 898 (pid=8483) 2019-09-16 20:49:22.897939: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 AVX512F FMA (pid=8483) 2019-09-16 20:49:22.919721: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 3800000000 Hz (pid=8483) 2019-09-16 20:49:22.920702: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55c5307cc240 executing computations on platform Host. Devices: (pid=8483) 2019-09-16 20:49:22.920751: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): , (pid=8483) WARNING: Logging before flag parsing goes to stderr. (pid=8483) W0916 20:49:22.922283 139638051562944 deprecation_wrapper.py:119] From /home/huixin/PycharmProjects/softlearning/examples/development/main.py:33: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. (pid=8483) (pid=8483) W0916 20:49:22.935199 139638051562944 deprecation.py:506] From /home/huixin/anaconda3/envs/softlearning/lib/python3.7/site-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.init (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version. (pid=8483) Instructions for updating: (pid=8483) Call initializer instance with the dtype argument instead of passing it to the constructor 2019-09-16 20:49:23,132 ERROR trial_runner.py:487 -- Error processing event. Traceback (most recent call last): File "/home/huixin/anaconda3/envs/softlearning/lib/python3.7/site-packages/ray/tune/trial_runner.py", line 436, in _process_trial result = self.trial_executor.fetch_result(trial) File "/home/huixin/anaconda3/envs/softlearning/lib/python3.7/site-packages/ray/tune/ray_trial_executor.py", line 323, in fetch_result result = ray.get(trial_future[0]) File "/home/huixin/anaconda3/envs/softlearning/lib/python3.7/site-packages/ray/worker.py", line 2189, in get raise value ray.exceptions.RayTaskError: ray_ExperimentRunner:train() (pid=8483, host=huixin-Alienware-Area-51-R5) File "/home/huixin/anaconda3/envs/softlearning/lib/python3.7/site-packages/ray/tune/trainable.py", line 151, in train result = self._train() File "/home/huixin/PycharmProjects/softlearning/examples/development/main.py", line 82, in _train self._build() File "/home/huixin/PycharmProjects/softlearning/examples/development/main.py", line 59, in _build variant, training_environment) File "/home/huixin/PycharmProjects/softlearning/softlearning/policies/utils.py", line 75, in get_policy_from_variant return get_policy_from_params(policy_params, args, kwargs) File "/home/huixin/PycharmProjects/softlearning/softlearning/policies/utils.py", line 68, in get_policy_from_params kwargs) File "/home/huixin/PycharmProjects/softlearning/softlearning/policies/utils.py", line 10, in get_gaussian_policy policy = FeedforwardGaussianPolicy(args, **kwargs) File "/home/huixin/PycharmProjects/softlearning/softlearning/policies/gaussian_policy.py", line 226, in init self._Serializable__initialize(locals()) AttributeError: 'FeedforwardGaussianPolicy' object has no attribute '_Serializable__initialize'

2019-09-16 20:49:23,133 INFO ray_trial_executor.py:187 -- Destroying actor for trial id=5bf385e2-seed=898. If your trainable is slow to initialize, consider setting reuse_actors=True to reduce actor creation overheads. 2019-09-16 20:49:23,137 INFO trial_runner.py:524 -- Attempting to recover trial state from last checkpoint. == Status == Using FIFO scheduling algorithm. Resources requested: 16/16 CPUs, 0/1 GPUs Memory usage on this node: 11.9/16.7 GB Result logdir: /home/huixin/ray_results/gym/Pendulum/v0/2019-09-16T20-49-16-2019-09-16T20-49-16 Number of trials: 1 ({'RUNNING': 1}) RUNNING trials:

(pid=8054) /home/huixin/anaconda3/envs/softlearning/lib/python3.7/site-packages/requests/init.py:91: RequestsDependencyWarning: urllib3 (1.25.1) or chardet (3.0.4) doesn't match a supported version! (pid=8054) RequestsDependencyWarning) (pid=8054) Warning: dm_control package not found. Run pip install git+https://github.com/deepmind/dm_control.git to use dm_control environments. (pid=8054) 2019-09-16 20:49:25.546920: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 AVX512F FMA (pid=8054) Using seed 898 (pid=8054) 2019-09-16 20:49:25.571804: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 3800000000 Hz (pid=8054) 2019-09-16 20:49:25.573208: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x5622e9eab0f0 executing computations on platform Host. Devices: (pid=8054) 2019-09-16 20:49:25.573249: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): , (pid=8054) WARNING: Logging before flag parsing goes to stderr. (pid=8054) W0916 20:49:25.574851 140703493096896 deprecation_wrapper.py:119] From /home/huixin/PycharmProjects/softlearning/examples/development/main.py:33: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. (pid=8054) (pid=8054) W0916 20:49:25.591375 140703493096896 deprecation.py:506] From /home/huixin/anaconda3/envs/softlearning/lib/python3.7/site-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.init (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version. (pid=8054) Instructions for updating: (pid=8054) Call initializer instance with the dtype argument instead of passing it to the constructor 2019-09-16 20:49:25,810 ERROR trial_runner.py:487 -- Error processing event. Traceback (most recent call last): File "/home/huixin/anaconda3/envs/softlearning/lib/python3.7/site-packages/ray/tune/trial_runner.py", line 436, in _process_trial result = self.trial_executor.fetch_result(trial) File "/home/huixin/anaconda3/envs/softlearning/lib/python3.7/site-packages/ray/tune/ray_trial_executor.py", line 323, in fetch_result result = ray.get(trial_future[0]) File "/home/huixin/anaconda3/envs/softlearning/lib/python3.7/site-packages/ray/worker.py", line 2189, in get raise value ray.exceptions.RayTaskError: ray_ExperimentRunner:train() (pid=8054, host=huixin-Alienware-Area-51-R5) File "/home/huixin/anaconda3/envs/softlearning/lib/python3.7/site-packages/ray/tune/trainable.py", line 151, in train result = self._train() File "/home/huixin/PycharmProjects/softlearning/examples/development/main.py", line 82, in _train self._build() File "/home/huixin/PycharmProjects/softlearning/examples/development/main.py", line 59, in _build variant, training_environment) File "/home/huixin/PycharmProjects/softlearning/softlearning/policies/utils.py", line 75, in get_policy_from_variant return get_policy_from_params(policy_params, args, kwargs) File "/home/huixin/PycharmProjects/softlearning/softlearning/policies/utils.py", line 68, in get_policy_from_params kwargs) File "/home/huixin/PycharmProjects/softlearning/softlearning/policies/utils.py", line 10, in get_gaussian_policy policy = FeedforwardGaussianPolicy(args, **kwargs) File "/home/huixin/PycharmProjects/softlearning/softlearning/policies/gaussian_policy.py", line 226, in init self._Serializable__initialize(locals()) AttributeError: 'FeedforwardGaussianPolicy' object has no attribute '_Serializable__initialize'

2019-09-16 20:49:25,812 INFO ray_trial_executor.py:187 -- Destroying actor for trial id=5bf385e2-seed=898. If your trainable is slow to initialize, consider setting reuse_actors=True to reduce actor creation overheads. 2019-09-16 20:49:25,816 INFO trial_runner.py:524 -- Attempting to recover trial state from last checkpoint. (pid=8057) /home/huixin/anaconda3/envs/softlearning/lib/python3.7/site-packages/requests/init.py:91: RequestsDependencyWarning: urllib3 (1.25.1) or chardet (3.0.4) doesn't match a supported version! (pid=8057) RequestsDependencyWarning) (pid=8057) Warning: dm_control package not found. Run pip install git+https://github.com/deepmind/dm_control.git to use dm_control environments. (pid=8057) Using seed 898 (pid=8057) 2019-09-16 20:49:28.414622: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 AVX512F FMA (pid=8057) 2019-09-16 20:49:28.439801: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 3800000000 Hz (pid=8057) 2019-09-16 20:49:28.440759: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x5623f06bd340 executing computations on platform Host. Devices: (pid=8057) 2019-09-16 20:49:28.440784: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): , (pid=8057) WARNING: Logging before flag parsing goes to stderr. (pid=8057) W0916 20:49:28.442358 140376052176320 deprecation_wrapper.py:119] From /home/huixin/PycharmProjects/softlearning/examples/development/main.py:33: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. (pid=8057) (pid=8057) W0916 20:49:28.458849 140376052176320 deprecation.py:506] From /home/huixin/anaconda3/envs/softlearning/lib/python3.7/site-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.init (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version. (pid=8057) Instructions for updating: (pid=8057) Call initializer instance with the dtype argument instead of passing it to the constructor 2019-09-16 20:49:28,655 ERROR trial_runner.py:487 -- Error processing event. Traceback (most recent call last): File "/home/huixin/anaconda3/envs/softlearning/lib/python3.7/site-packages/ray/tune/trial_runner.py", line 436, in _process_trial result = self.trial_executor.fetch_result(trial) File "/home/huixin/anaconda3/envs/softlearning/lib/python3.7/site-packages/ray/tune/ray_trial_executor.py", line 323, in fetch_result result = ray.get(trial_future[0]) File "/home/huixin/anaconda3/envs/softlearning/lib/python3.7/site-packages/ray/worker.py", line 2189, in get raise value ray.exceptions.RayTaskError: ray_ExperimentRunner:train() (pid=8057, host=huixin-Alienware-Area-51-R5) File "/home/huixin/anaconda3/envs/softlearning/lib/python3.7/site-packages/ray/tune/trainable.py", line 151, in train result = self._train() File "/home/huixin/PycharmProjects/softlearning/examples/development/main.py", line 82, in _train self._build() File "/home/huixin/PycharmProjects/softlearning/examples/development/main.py", line 59, in _build variant, training_environment) File "/home/huixin/PycharmProjects/softlearning/softlearning/policies/utils.py", line 75, in get_policy_from_variant return get_policy_from_params(policy_params, args, kwargs) File "/home/huixin/PycharmProjects/softlearning/softlearning/policies/utils.py", line 68, in get_policy_from_params kwargs) File "/home/huixin/PycharmProjects/softlearning/softlearning/policies/utils.py", line 10, in get_gaussian_policy policy = FeedforwardGaussianPolicy(args, **kwargs) File "/home/huixin/PycharmProjects/softlearning/softlearning/policies/gaussian_policy.py", line 226, in init self._Serializable__initialize(locals()) AttributeError: 'FeedforwardGaussianPolicy' object has no attribute '_Serializable__initialize'

2019-09-16 20:49:28,657 INFO ray_trial_executor.py:187 -- Destroying actor for trial id=5bf385e2-seed=898. If your trainable is slow to initialize, consider setting reuse_actors=True to reduce actor creation overheads. == Status == Using FIFO scheduling algorithm. Resources requested: 0/16 CPUs, 0/1 GPUs Memory usage on this node: 11.9/16.7 GB Result logdir: /home/huixin/ray_results/gym/Pendulum/v0/2019-09-16T20-49-16-2019-09-16T20-49-16 Number of trials: 1 ({'ERROR': 1}) ERROR trials:

== Status == Using FIFO scheduling algorithm. Resources requested: 0/16 CPUs, 0/1 GPUs Memory usage on this node: 11.9/16.7 GB Result logdir: /home/huixin/ray_results/gym/Pendulum/v0/2019-09-16T20-49-16-2019-09-16T20-49-16 Number of trials: 1 ({'ERROR': 1}) ERROR trials:

Traceback (most recent call last): File "/home/huixin/PycharmProjects/softlearning/examples/development/main.py", line 263, in main(argv=sys.argv[1:]) File "/home/huixin/PycharmProjects/softlearning/examples/development/main.py", line 259, in main run_example_local('examples.development', argv) File "/home/huixin/PycharmProjects/softlearning/examples/instrument.py", line 224, in run_example_local reuse_actors=True) File "/home/huixin/anaconda3/envs/softlearning/lib/python3.7/site-packages/ray/tune/tune.py", line 272, in run raise TuneError("Trials did not complete", errored_trials) ray.tune.error.TuneError: ('Trials did not complete', [id=5bf385e2-seed=898])

Process finished with exit code 1`

wookayin commented 4 years ago

Use git+https://github.com/hartikainen/serializable.git@76516385a3a716ed4a2a9ad877e2d5cbcf18d4e6 (Defined in requirements.txt).

However, the authors should needs to improve this so that it does not use a very specific, non-pypi, pinned version of libraries. There is no dependencies/libraries maintained in setup.py.

/cc @hartikainen could you maintain serializable?