Colin97 / OpenShape_code

official code of “OpenShape: Scaling Up 3D Shape Representation Towards Open-World Understanding”
https://colin97.github.io/OpenShape/
Apache License 2.0
224 stars 15 forks source link

how to easily retrieve the desired Shape files (Mesh) of objects using your code? #5

Closed StellarCheng closed 1 year ago

StellarCheng commented 1 year ago

Hello author, thank you for your excellent work. I would like to know how to easily retrieve the desired Mesh files of objects using your code. For example, I only want to retrieve the top N Shape files that match the description of 'a vintage American sports car' based on text. Could you please guide me on how to obtain the corresponding Shape files? Thank you

eliphatfs commented 1 year ago

If you are not running this in a large volume, you may try the online demo. By clicking on links below results you will jump to a dataset page where you can download them.

If you want to do it for a large set of descriptions, you will need to download the embeddings. You can check the relevant code here on how to use them to retrieve results. You can download from the glb key in the returned entries (an example of returned entry is pasted in the code comments) at this uri: https://huggingface.co/datasets/allenai/objaverse/resolve/main/{glb}.

Edit: in your case, you should pass a CLIP text embedding to the retrieve function.

StellarCheng commented 1 year ago

Thank you for your response. So it means that I can use the code in example.py to get the text or image embedding?

Colin97 commented 1 year ago

Yes, you can use the OpenCLIP (ViT-bigG-14) for the text and image embedding.

StellarCheng commented 1 year ago

Yes, you can use the OpenCLIP (ViT-bigG-14) for the text and image embedding.

Thanks, that's clear~