search

SforAiDl / genrl

A PyTorch reinforcement learning library for generalizable and reproducible algorithm implementations with an aim to improve accessibility in RL
https://genrl.readthedocs.io
MIT License
403 stars 59 forks source link

Add PyTorch Lightning support #177

Open sauradefy99 opened 4 years ago

sauradefy99 commented 4 years ago

PyTorch Lightning automates away most of the engineering required on PyTorch for features like multi-GPU distributed training, early stopping, etc. More details can be found on the blog for the latest version here: https://medium.com/pytorch/pytorch-multi-gpu-metrics-and-more-in-pytorch-lightning-0-8-1-b7cadd04893e.

Sharad24 commented 4 years ago

Any thoughts on how you're going to go about it? Not sure but I think we can use model parallelism but not data parallelism unless we have offline RL datasets (which we should add support for too)

Sharad24 commented 4 years ago

@sauradefy99 Did you think on this? I think a good first step could be in trying out Offline RL?

Multi-GPU distributed training isn't the same at least in the online/off-policy case, it might be for Offline/Batch RL.

Sharad24 commented 4 years ago

Reopen if you're working on it

ugurkanates commented 3 years ago

I think this issue needs to be reopened. PLightning support should be critical

sampreet-arthi commented 3 years ago

Hey @ugurkanates , thanks for your interest in GenRL! We'd closed this initially because a lot of our team is working on some or the other thing right now. Would you be interested in helping us with this? Let us know if you can!

ugurkanates commented 3 years ago

Yes I'm actually looking forward because we are trying to integrate RL frameworks to our codebase and we still want benefits from PL Lightning to apply.

sampreet-arthi commented 3 years ago

Sounds great! I'll assign you to this then. Let us know how you're gonna go about this.

cc: @Sharad24

ugurkanates commented 3 years ago

Sorry but we instead decided to go with RLLib for now. Maybe later someone(or me) will keep updated on this task