search

Ericcsr / ARCap

Data collection part for ARCap
38 stars 5 forks source link

Robot on VR does not update automatically #2

Open fredshentu opened 2 weeks ago

fredshentu commented 2 weeks ago

I would like to thank you for open-sourcing this awesome project! It's a super cool concept.

I have successfully set up the Quest 3 app and Windows environment. When I move my left-hand controller, I can see the joint information updating and sending to the Quest 3. https://github.com/Ericcsr/ARCap/blob/d50affb315a1ab641ad3791be3d5b2ffd6e8345e/data_processing/quest_robot_module.py#L416 I am confident that the VR IP/host IP I entered is correct. Additionally, when I press button A on the right controller, the robot in VR updates slightly.

Despite this setup, I'm encountering an issue where the joint information does not seem to be updating fully or as expected.

Do you know what could be the possible issue?

Thank you!

Ericcsr commented 2 weeks ago

Thanks for your interest! I am not super clear about your issue discription so here are some questions:

  1. Which robot are you using: Gripper, Leap hand or bimanual?
  2. When you mentioned joint information didn't update correctly, did you saw a green frame which indicate the system is indeed in data collection mode.
  3. "joint information does not seem to be updating fully", does this indicate the virtual robot freeze and not tracking controller or it keep jittering but response to different controller positions.
fredshentu commented 20 hours ago

sorry for the late reply I was busy with other project the past weeks.

1) I am using gripper, since I don't have the Rokoko glove, I just hardcode the left_positions to np.zeros([4,3]) - 0.03 2) if I press X it will show recording with a red box. 3) actually i print the arm_q arm_q, hand_q, wrist_pos, wrist_orn = quest.solve_system_world(wrist_pos, wrist_orn, hand_tip_pose) here, the arm_q is updating, just the vertual robot is freezing and will jitter if I press A

do you know what could cause this issue?

fredshentu commented 18 hours ago

since the rokoko glove are pretty expensive it will be great if you can provide a test script which allows people to try this new type of visual feedback (like simply moving a robot) without further hardware integration. I feel I've removed the rokoko dependency but for some reason the robot does not render (can confirm that ik works and new robot arm_q is constantly sending to the quest3)

Ericcsr commented 17 hours ago

I think the cause may be data_collection_server is waiting for rokoko update in rokoko.receive(), if you have a hard coded hand position you can comment out everything related to rokoko. I will also add support for running code without rokoko using quest’s hand tracker recently, hopefully it will be more accessible

fredshentu commented 4 hours ago

I’ve already commented out everything related to rokoko. It would be fantastic if you could provide a script that allows users without a Rokoko glove to try the system. Thanks in advance!