Open Dong09 opened 12 months ago
Hi @Dong09,
The dataset without an annotation file can not be used for training our method, as our method requires a 3D hand pose at each time step during training.
But I guess visualization using a pre-trained model is still possible, and you can do that by modifying the code to not calculate the number by commenting out the evaluation process.
Thanks for your reply. Actually, I want to be able to see the rendered effect on the local image or video, but the load_data function of the test script needs 4 json files to get the datalist, is there any way to do it
I guess you need to modify some code to do that.
I recommend you make your own dataset python file, e.g., src/data/own_dataset.py
, which not loads json
file, and just loads the image.
I guess you need to modify some code to do that.
I recommend you make your own dataset python file, e.g., , which not loads file, and just loads the image.
src/data/own_dataset.py``json
okk, i will try it. thks for your reply! :)
I guess you need to modify some code to do that.
I recommend you make your own dataset python file, e.g.,
src/data/own_dataset.py
, which not loadsjson
file, and just loads the image.
sorry, i am back again. i have made my own dataset.py and now i have no idea about result visualization like example in README, could you mind do a demo? i would be grateful if could.
hello, when i try to render mesh in the img, i get some problem. img and code are following.
def render_mesh(img, mesh, face, cam_param):
# mesh
mesh = trimesh.Trimesh(mesh, face)
rot = trimesh.transformations.rotation_matrix(
np.radians(180), [1, 0, 0])
mesh.apply_transform(rot)
material = pyrender.MetallicRoughnessMaterial(metallicFactor=0.0, alphaMode='OPAQUE', baseColorFactor=(1.0, 1.0, 0.9, 1.0))
mesh = pyrender.Mesh.from_trimesh(mesh, material=material, smooth=False)
scene = pyrender.Scene(ambient_light=(0.3, 0.3, 0.3))
scene.add(mesh, 'mesh')
focal, princpt = cam_param['focal'], cam_param['princpt']
camera = pyrender.IntrinsicsCamera(fx=focal[0], fy=focal[1], cx=princpt[0], cy=princpt[1])
scene.add(camera)
# renderer
renderer = pyrender.OffscreenRenderer(viewport_width=img.shape[1], viewport_height=img.shape[0], point_size=1.0)
# light
light = pyrender.DirectionalLight(color=[1.0, 1.0, 1.0], intensity=0.8)
light_pose = np.eye(4)
light_pose[:3, 3] = np.array([0, -1, 1])
scene.add(light, pose=light_pose)
light_pose[:3, 3] = np.array([0, 1, 1])
scene.add(light, pose=light_pose)
light_pose[:3, 3] = np.array([1, 1, 2])
scene.add(light, pose=light_pose)
# render
rgb, depth = renderer.render(scene, flags=pyrender.RenderFlags.RGBA)
rgb = rgb[:,:,:3].astype(np.float32)
valid_mask = (depth > 0)[:,:,None]
# save to image
img = rgb * valid_mask + img * (1-valid_mask)
return img
self.opt_params['focal'] = [5000,5000]
self.opt_params['input_img_shape'] = [256,256]
focal = [self.opt_params['focal'][0] / self.opt_params['input_img_shape'][1] * out['bbox'][2], self.opt_params['focal'][1] / self.opt_params['input_img_shape'][0] * out['bbox'][3]]
princpt = [self.opt_params['princpt'][0] / self.opt_params['input_img_shape'][1] * out['bbox'][2] + out['bbox'][0], self.opt_params['princpt'][1] / self.opt_params['input_img_shape'][0] * out['bbox'][3] + out['bbox'][1]]
rendered_img = render_mesh(blur_img, mesh_out*np.array([1,-1,-1]), mano.face['right'], {'focal': focal, 'princpt': princpt})
Sorry for the late reply.
Did you get a solution?
As we used cropped hand images in both the training and inference phases, I think inputting images with the whole body may cause a problem.
hello,tks for this amazing work. And how to run this on my own dataset without annot_file?