lost reference to actor

[hlsafin]

[koulanurag]

Please, refer to #3

[grifball]

I couldn't install a usable ray version with pip. Had to modify the code a bit, based on this issue with ray: https://github.com/ray-project/ray/issues/6265 here's my patch, I really don't understand this problem, but I tried to emulate how they fixed it in this issue and it seemed to work.

(venv) grifball@grifball-Nobilis:~/checkout/muzero-pytorch$ git diff core/train.py
diff --git a/core/train.py b/core/train.py
index 099aa53..94d2195 100644
--- a/core/train.py
+++ b/core/train.py
@@ -339,8 +339,10 @@ def train(config, summary_writer=None):
     storage = SharedStorage.remote(config.get_uniform_network())
     replay_buffer = ReplayBuffer.remote(batch_size=config.batch_size, capacity=config.window_size,
                                         prob_alpha=config.priority_prob_alpha)
-    workers = [DataWorker.remote(rank, config, storage, replay_buffer).run.remote()
+    workers = [DataWorker.remote(rank, config, storage, replay_buffer)
                for rank in range(0, config.num_actors)]
+    for worker in workers:
+        worker.run.remote()
     workers += [_test.remote(config, storage)]
     _train(config, storage, replay_buffer, summary_writer)
     ray.wait(workers, len(workers))

(venv) grifball@grifball-Nobilis:~/checkout/muzero-pytorch$ pip install ray==0.7.6
ERROR: Could not find a version that satisfies the requirement ray==0.7.6
ERROR: No matching distribution found for ray==0.7.6

(venv) grifball@grifball-Nobilis:~/checkout/muzero-pytorch$ pip freeze | grep ray
ray==1.1.0

[koulanurag]

HI @grifball,

Thanks for pointing out the issue with the ray version. Apparently, as per this, it's due to some issue in python3.7 and python3.6.

I tried to resolve the ray version at my end and had the same issue, as found by you.

With respect to changes suggested by you regarding the "ray version" and "ray remote worker initialization", I would be happy to receive a pull request.