search

pytorch / rl

A modular, primitive-first, python-first PyTorch library for Reinforcement Learning.
https://pytorch.org/rl
MIT License
2.2k stars 290 forks source link

[Question] Item called in loss modules forces synchronization? #1783

Open skandermoalla opened 8 months ago

skandermoalla commented 8 months ago

Aren't operations like this blocking? causing performance issues due to GPU synchronization? https://github.com/pytorch/rl/blob/5b3dd988ad7f0f91a2f61f77a8b3380eae3131fd/torchrl/objectives/ppo.py#L443

vmoens commented 8 months ago

Good point we should check that!