Hi, the codes of "gather_experience " is missing in your actor.py

[tbwxmu]

I am a new guy to the apex DQN. Thanks for your contribution. Your codes help me to understand this algorithm. please also add the codes of the gather_experience from the Actor class.

    actor = Actor(1, env_conf, shared_state, shared_replay_mem, params)
    actor.gather_experience(101)  #  missing "gather_experience" code
    print("Main: replay_mem.size:", shared_replay_mem.qsize())

[praveen-palanisamy]

Hi @tbwxmu ,

Glad that you found the code in this repo useful to understand Ape-X DQN.

Thank you for catching that and reporting it here! I had renamed the gather_experience(...) method to run() to stick with the naming convention used in multiprocessing. I have merged a PR #2 to update the code. Please try running it now (python actor.py).

[tbwxmu]

Thanks for your reply. But I also find another problem in "main.py".

        actor_proc = Actor(i, env_conf, shared_state, shared_mem, actor_params)
        actor_proc.start()
        actor_procs.append(actor_proc)

those codes never use the Actor.run() function right? So the ape-x DQN learning never work? Please correct me if I missing something.

[praveen-palanisamy]

The following line: https://github.com/praveen-palanisamy/Ape-X-DQN/blob/b66921bfe460ffcd067e7cd8e2be6e37599d8165/main.py#L53 Will start the actor process which essentially will call Actor's run(...) method. This is the behavior of PyTorch's Multiprocessing (which is a wrapper around Python's Multiprocessing). The Actor class is a subclass of torch.multiprocessing.Process.

Hope that provides enough clarification?

[tbwxmu]

Thanks for your detailed explanation. I discovered the Ray lib may be better for parallel computing. I want to know if you have a plan to modify your codes with Ray?