search

google-research / kubric

A data generation pipeline for creating semi-realistic synthetic multi-object videos with rich annotations such as instance segmentation masks, depth maps, and optical flow.
Apache License 2.0
2.22k stars 218 forks source link

How to register new object 3d model? #250

Open WenyuanQ opened 2 years ago

WenyuanQ commented 2 years ago

Is it possible to register new object into Kubric? For example I have a 3d model of .kmz or .blender, how can I add it into the generated video?

MrXandbadas commented 2 years ago

Hello! Please refer to the following

Is there any example of loading objects in the scene? https://github.com/google-research/kubric/issues/185

WenyuanQ commented 2 years ago

Thanks! I will try it now

WenyuanQ commented 2 years ago

Hello! Please refer to the following

Is there any example of loading objects in the scene?

185

Thank you very much for helping me with it! I still get a little bit confused in the dataset ShapeNet. For the models that not in ShapeNetCore but in ShapeNet, seems I cannot find them in our provided json file. Could I ask how can I use these kind of models?

MrXandbadas commented 1 year ago

@WenyuanQ here

This dataset contains each ShapeNet object rendered from 25 random perspectives on transparent background. Images are rendered at 512x512 and then cropped to fit the object, so they vary in size:

docker run --rm --interactive \
  --user $(id -u):$(id -g)    \
  --volume "$(pwd):/kubric"   \
  kubricdockerhub/kubruntu    \
  /usr/bin/python3 challenges/shapenet_pretraining/shapenet_pretraining_worker.py

Do this first: https://github.com/google-research/kubric/blob/b9e7358728c4323f7dfb871b5974eca8e4a96d61/challenges/pretraining_visual/worker.py#L20

# TODO: go to https://shapenet.org/ create an account and agree to the terms # then find the URL for the kubric preprocessed ShapeNet and put it here:

WenyuanQ commented 1 year ago

@WenyuanQ here

This dataset contains each ShapeNet object rendered from 25 random perspectives on transparent background. Images are rendered at 512x512 and then cropped to fit the object, so they vary in size:

docker run --rm --interactive \
  --user $(id -u):$(id -g)    \
  --volume "$(pwd):/kubric"   \
  kubricdockerhub/kubruntu    \
  /usr/bin/python3 challenges/shapenet_pretraining/shapenet_pretraining_worker.py

Do this first:

https://github.com/google-research/kubric/blob/b9e7358728c4323f7dfb871b5974eca8e4a96d61/challenges/pretraining_visual/worker.py#L20

# TODO: go to https://shapenet.org/ create an account and agree to the terms # then find the URL for the kubric preprocessed ShapeNet and put it here:

Hi Thank you very much for the helping!

I have successfully added '.obj' file into scene using function: kb.FileBasedObject(). I'd like to ask if I can add '.mtl' or '.jpg' texture information into this object? Thanks. And another question, when I add object into scene, seems it cannot generate segmentation annotation. Could you please tell me how can I make it correct?