Use custom

[vrk7]

Hi @ashawkey @alvinliu0

How to change the prior used to a custom one, like a pot or vase?

[alvinliu0]

Hi, many thanks for your interest in our work:)

You can tweak the code here to not set the initialized 3D Gaussian Splatting position on SMPL-X mesh, but on the customized mesh like pot or vase you mentioned, or use text-to-point-cloud prior like Point-E and Shap-E for initialization. The function inherited from GaussianDreamer gives an example of Shap-E initialization. Also, note that since general object like pot or vase may not have a clearly defined skeleton, the guidance should also be changed to an uncondition SD accordingly.

Best,

[vrk7]

Hi @alvinliu0

Does tweaking code in that part does the work? Cause in the configure definition i see a initialization of skeleton SMPLX as given below.

[vrk7]

@alvinliu0 @ashawkey ?

[alvinliu0]

Because the actual part for 3DGS point cloud initialization is here. So if you want to change to shap-e initialization, you can comment off those part of code and turn on shap-e part.

[vrk7]

Hi @alvinliu0

I have uncommented this shap-e initialization and got decent results. But if I need to use prior like SMAL, 3DGS point cloud initialization does work or should I be creating a separate skeleton class for the prior shape under consideration?

[alvinliu0]

Yes, because there's no strict alignment between prior like Shap-E or Point-E with body skeleton (you can not even guarantee that the 3d human generated by them are correct or complete). Hence it might be difficult to use them as prior/initialization while still use skeleton as condition for ControlNet or our texture-structure joint model as SDS.