[Feature Request] TorchRL for Python 3.13t

pytorch / rl

A modular, primitive-first, python-first PyTorch library for Reinforcement Learning.

https://pytorch.org/rl

MIT License

2.35k stars 310 forks source link

[Feature Request] TorchRL for Python 3.13t #2535

Open patchmeifyoucan opened 2 weeks ago

patchmeifyoucan commented 2 weeks ago

Hello, PyTorch recently added support for Python 3.13t, aka "no-gil" Python (https://github.com/pytorch/pytorch/issues/130249). Are there any plans to make TorchRL usable from this version? It would be nice if this could be supported soon, as the ability to run CPU code for interactions with the environment from multiple no-gil threads could lead to major performance improvements and higher GPU utilization.

vmoens commented 1 week ago

That's definitely on our radar! Beyond multithreaded envs another big benefit would be reducing the cpu overheads across the lib (sampling in rbs, iterating over TensorDict etc).

Is there anything we should prioritise in your opinion?

patchmeifyoucan commented 1 week ago

Thank you, good to know.

For me, simply general support would be sufficient to get a better GPU utilization, that's what I'm after. I mostly use torchrl's replay buffers but their performance is far from being a bottleneck in my application.