Element 0 of tensors does not require grad and does not have a grad_fn

[weiheng-liu]

If it’s not a bug, please use discussions: https://github.com/NVlabs/curobo/discussions

Please provide the below information in addition to your issue:

1. cuRobo installation mode (choose from [python, isaac sim, docker python, docker isaac sim]): python
2. python version: Python 3.8.19
3. Isaac Sim version (if using): not using When I use curobo_motion_plan : self.motion_gen_config = MotionGenConfig.load_from_robot_config( self.robot_cfg, self.world_cfg, self.tensor_args, interpolation_steps=300, collision_checker_type=self.collision_checker_type, trajopt_tsteps=32, num_trajopt_seeds=12, num_graph_seeds=3, interpolation_dt=0.02, use_cuda_graph=False, ) self.motion_generator = MotionGen(self.motion_gen_config) and then I use batch_plan to get the collision-free trajectory: result = self.motion_generator.batch_plan(curr_js, ik_goal, enable_graph=True, enable_opt=False, num_graph_seeds=1) But the problem is that: when I take the trajectory as the ground Truth to supervised a policy, and I make the L2 loss to update the model, but it has the following problem during the process of newton optimizer: Element 0 of tensors does not require grad and does not have a grad_fn How should I fix it? I have tried for a few weeks to solve this problem :(

---

Issue Details

[balakumar-s]

If you are stuck for more than 1 day with something in cuRobo, please ask in our discussions (https://github.com/NVlabs/curobo/discussions) as we might have faced a similar issue and someone will provide a fix.

As for your issue, yes! we know why this happens. cuRobo uses torch's autograd internally to collect gradients during numerical optimization. So if you call cuRobo's IK or planning calls within a torch.no_grad or torch.inference_mode context, you might get this error.

For e.g.,

with torch.no_grad():
    motion_gen.plan_single()

or

with torch.inference_mode():
   motion_gen.plan_single()

If you have a no_grad context globally, you can disable it temporarily with torch.enable_grad context:

with torch.no_grad():
    with torch.enable_grad():
          motion_gen.plan_single()

Let us know if this does not fix your issue.

[weiheng-liu]

Thank u very much. I also have another problem: In motion planning, I want to attach a new object to the robot, But I find there is only a API for single environment: self.motion_generator = MotionGen(self.motion_gen_config) self.motion_generator.attach_new_object_to_robot(curr_js, cuboid, surface_sphere_radius=0.002, link_name=link_name) I want to ask is there no API for batch attach_new_object_to_robot ?

[balakumar-s]

It's not supported for batched environments as we have only one representation robot across the batch. You could implement it as an additional collision checking cost across batch. More details are here: https://github.com/NVlabs/curobo/discussions/91#discussioncomment-7930459