Distributed training

[louis030195]

First off, I doubt deep RL can work (RL without deep learning isn't really interesting imho) without distributed training, e.g. enough data.

First need to make it work as single instance ofc, I see something like:

• gym.make(prismarine-gym) - give some config (i want 10 parallelism, 300 agents per server, nothing related to algorithm of course)
• it should spawn X servers with X agents then for the distributed model, need to dig what is done, but I guess something brute force first: do a step, sync with others (websocket or whatever) (prob slow because sync)

[louis030195]

https://excalidraw.com/#json=5157858798731264,hkwKyNJz1IXqulSZ7K6Y4A

or in a sort of "federated learning" manner with X models with maybe some weights sharing in-between (need to dig distributed training options)

https://excalidraw.com/#json=4786774228336640,tXjHK7wVutBTXxtaml-M9g

worker = flying-squid (or any minecraft server)

[louis030195]

Unsure the distributed part should be in this repo or a wrapper around it to make it easier to do distributed training (and if its splitable ofc)

[louis030195]

Going for option 1, i.e. non-multi-agent first, it's much simpler https://excalidraw.com/#json=6238340198170624,bp6qjjPRP9rzQWdIo4Mtng