allenai / objathor

Python package for importing and loading external assets into AI2THOR
Apache License 2.0
15 stars 1 forks source link

Accessing glb files #27

Open Singh-sid930 opened 1 month ago

Singh-sid930 commented 1 month ago

Hi, I am trying to use this dataset in the Unity editor in which I can load glb files. I also need the sbert and clip embeddings to perform matching of information to choose which asset to load. I have downloaded the assets and the features at the 2023_09_23 version. however in the assets directory inside the uid based folders I only see pkl files of the asset and not glb files. Is it possible to get the glb files of the same uids so that I can directly load the glb files?

Lucaweihs commented 1 month ago

Hi @Singh-sid930,

The original glb files are available via the objaverse library, here's how we download them before we process them with objathor (see here):

path_to_glb_file = objaverse.load_objects(uids=[uid])[uid]

where you replace uid with the object you want.

A few caveats, the original objaverse assets are:

  1. Unoptimized: you can load them into Unity but they may be very large.
  2. Unscaled: we automatically rescale the objects to respect the sizes in the annotations, the original glbs will not be scaled.
  3. Unrotated: we rotate/center the objects so that they are in a more uniform position, this will also not be the case for the original glbs.

If the above are deal breakers for you, it should be pretty easy to write some code that translates the pkl files into obj files (which I believe are loadable in Unity?), if you read the pkl file into Python you'll see that it's just a dictionary containing the usual information required to define a 3D asset (e.g. triangles, vertices, paths to textures, etc).

Singh-sid930 commented 1 month ago

I see , that sounds about right. I am trying to convert the ply files into glb files like you mentioned from the dictionary values. https://gitlab.com/dodgyville/pygltflib I am trying to use this package but adding material textures seems to be breaking things. Please let me know if there is anything you guys frequently use to convert the ply files into glb would really appreciate it