Obtain gradient information explicitly

[glue25]

May I ask how to explicitly obtain gradient information in the environment, such as obtaining the gradient of s_{t+1} to s_t? Or is the information obtained using torch.autograd accurate? Thank you!

[eanswer]

Although it's not tested, since dflex is built upon PyTorch, you should be able to use PyTorch functions to get jacobian info. To be noted, the backward gradients are computed via reverse mode auto differentiation like PyTorch, so the intermediate jacobians are not explicitly computed, which means you may only get the jacobian from output to input (i.e. call s_{s+1} = step(st) then use PyTorch function to get d(s{t+1})/d(s_{t}).