search

DeNA / HandyRL

HandyRL is a handy and simple framework based on Python and PyTorch for distributed reinforcement learning that is applicable to your own environments.
MIT License
282 stars 39 forks source link

Large scale training #343

Closed Jogima-cyber closed 1 year ago

Jogima-cyber commented 1 year ago

I use HandyRL very intensively and would like to take advantage of the server/client functionality to optimize my costs on vastai. My machines are not on the same local network and I'm asking if it is still possible for them to connect to each other. From the vastai point of view, it is possible to open ports on the vms (9999 and 9998) and you have access to the public IP of the machines. I tried but my server does not receive client requests, and I don't know if it is a problem with HandyRL or with my machines.

Jogima-cyber commented 1 year ago

Actually this is working now - issue resolved.

YuriCat commented 1 year ago

Hi @Jogima-cyber, thanks for using HandyRL in so many ways! We tried creating connections through WebSocket in this branch. https://github.com/DeNA/HandyRL/compare/develop...YuriCat:HandyRL:experiment/websocket_remote_connection

Worker machines do not need to be in the same network with this structure.

Jogima-cyber commented 1 year ago

It’s already working with socket when not on the same network for me. For 2 days I’ve been using 150CPUs across the earth and 2 RTX 3090. The only thing one must be careful about is the firewall. So I’m asking myself what would be the benefits of WebSocket ? I’m asking because I don’t anything in applied network haha.