search

NVIDIA / Dataset_Synthesizer

NVIDIA Deep learning Dataset Synthesizer (NDDS)
Other
563 stars 124 forks source link

Extracting joint keypoint information to .json files (like seen generated in the DREAM project datasets) #90

Open lakshmi-sathi opened 3 years ago

lakshmi-sathi commented 3 years ago

Hi,

Thanks for such a wonderful tool offering to the research community. I wanted to extract the joint keypoint information of the Jaco-2 robot arm (3D and projected coordinates) into the .json files created during data capture.

I have imported the model of the Jaco2 arm (skeletal mesh) into the NDDS project in UE4 and have done some domain randomization. I have used NVCapturableActorTag with which I could extract the location and rotation of the arm model as a whole and also of the camera, into .json files with the images captured, however, I want to extract the keypoint information also for training keypoint detection.

I can see the joint keypoints getting represented with blue markers during the domain randomized data capture (below image).

image

but I would like to extract this keypoint information into the .json file as seen in the synthetic datasets in the DREAM project (below image)

image

What may be a way to proceed with this, is there any direct option in NDDS to extract this keypoint information to the .json file?

Thanks.

wongsinglam commented 3 years ago

Hi, I am quite interested in your keypoint in your first image. How do you mark your keypoint to the robot? And how do you configure your robot in the ue4 project, thanks

hetolin commented 3 years ago

@lakshmi-sathi hi, I am also quite interested in your keypoint in your first image. How do you import the Jaco robot model? Do you import the urdf file or other types of files? And after importing the model, how could you control the robot joints?