search

p-christ / Deep-Reinforcement-Learning-Algorithms-with-PyTorch

PyTorch implementations of deep reinforcement learning algorithms and environments
MIT License
5.64k stars 1.2k forks source link

Exception in CartPole with A2C: AttributeError: 'Process' object has no attribute 'kill' #51

Open acriptis opened 4 years ago

acriptis commented 4 years ago

I've launched python3 results/Cart_Pole.py

And during training A2C got the error:


AGENT NAME: A2C
7.1: A2C
TITLE  CartPole
{'learning_rate': 0.005, 'linear_hidden_units': [20, 10], 'final_layer_activation': ['SOFTMAX', None], 'gradient_clipping_norm': 5.0, 'discount_rate': 0.99, 'epsilon_decay_rate_denominator': 1.0, 'normalise_rewards': True, 'exploration_worker_difference': 2.0, 'clip_rewards': False, 'Actor': {'learning_rate': 0.0003, 'linear_hidden_units': [64, 64], 'final_layer_activation': 'Softmax', 'batch_norm': False, 'tau': 0.005, 'gradient_clipping_norm': 5, 'initialiser': 'Xavier'}, 'Critic': {'learning_rate': 0.0003, 'linear_hidden_units': [64, 64], 'final_layer_activation': None, 'batch_norm': False, 'buffer_size': 1000000, 'tau': 0.005, 'gradient_clipping_norm': 5, 'initialiser': 'Xavier'}, 'min_steps_before_learning': 400, 'batch_size': 256, 'mu': 0.0, 'theta': 0.15, 'sigma': 0.25, 'action_noise_std': 0.2, 'action_noise_clipping_range': 0.5, 'update_every_n_steps': 1, 'learning_updates_per_learning_session': 1, 'automatically_tune_entropy_hyperparameter': True, 'entropy_term_weight': None, 'add_extra_noise': False, 'do_evaluation_iterations': True, 'output_activation': None, 'hidden_activations': 'relu', 'dropout': 0.0, 'initialiser': 'default', 'batch_norm': False, 'columns_of_data_to_be_embedded': [], 'embedding_dimensions': [], 'y_range': ()}
RANDOM SEED  4098626839
 Episode 449, Score:  32.00, Max score seen:  148.00, Rolling score:  43.49, Max rolling score seen:  45.49Process Process-1:
Traceback (most recent call last):
  File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "/home/alx/Cloud/projects/reinforcement_learning/Deep-Reinforcement-Learning-Algorithms-with-PyTorch/agents/actor_critic_agents/A2C.py", line 19, in update_shared_model
    new_grads = gradient_updates_queue.get()
  File "/usr/lib/python3.6/multiprocessing/queues.py", line 113, in get
    return _ForkingPickler.loads(res)
  File "/home/alx/.local/lib/python3.6/site-packages/torch/multiprocessing/reductions.py", line 151, in rebuild_storage_fd
    fd = df.detach()
  File "/usr/lib/python3.6/multiprocessing/resource_sharer.py", line 57, in detach
    with _resource_sharer.get_connection(self._id) as conn:
  File "/usr/lib/python3.6/multiprocessing/resource_sharer.py", line 87, in get_connection
    c = Client(address, authkey=process.current_process().authkey)
  File "/usr/lib/python3.6/multiprocessing/connection.py", line 487, in Client
    c = SocketClient(address)
  File "/usr/lib/python3.6/multiprocessing/connection.py", line 614, in SocketClient
    s.connect(address)
FileNotFoundError: [Errno 2] No such file or directory
Traceback (most recent call last):
  File "results/Cart_Pole.py", line 141, in <module>
    trainer.run_games_for_agents()
  File "/home/alx/Cloud/projects/reinforcement_learning/Deep-Reinforcement-Learning-Algorithms-with-PyTorch/agents/Trainer.py", line 79, in run_games_for_agents
    self.run_games_for_agent(agent_number + 1, agent_class)
  File "/home/alx/Cloud/projects/reinforcement_learning/Deep-Reinforcement-Learning-Algorithms-with-PyTorch/agents/Trainer.py", line 117, in run_games_for_agent
    game_scores, rolling_scores, time_taken = agent.run_n_episodes()
  File "/home/alx/Cloud/projects/reinforcement_learning/Deep-Reinforcement-Learning-Algorithms-with-PyTorch/agents/actor_critic_agents/A3C.py", line 48, in run_n_episodes
    optimizer_worker.kill()
AttributeError: 'Process' object has no attribute 'kill'

Does anybody knows what is the best fix for this?