search

xingy038 / Dreamer-XL

50 stars 2 forks source link

sd xl question #2

Open Zhang-Jiahui opened 1 month ago

Zhang-Jiahui commented 1 month ago

hello, you paper is great. I want to ask some questions for help about using the sd xl. These questions may have nothing to do with your paper.

When I use the sd xl and SDS loss to generate a 3D object, it always failed and produce bad color just like this: image I use the shap-e as the geometry prior. But, when I change the model to sd 2.1, the generation is ok. It really puzzled me. I don't know what the trouble is. Could you provide some advice about generating 3D using sd xl and SDS? I would appreciate if you could help. It will really help to me. Thanks.

xingy038 commented 1 month ago

I'm not sure what your baseline is. I think this problem arises from the SD gradient as mentioned in our paper.

Zhang-Jiahui commented 1 month ago

Thank you for your advice. I also have some questions about the paper. Why the eauqtion(12) has two min() functions. And why the depth participated in the grad. I only used the rgb values from rendered image to calculate the SDS but not used depth. Of course, maybe we use different methods. image

And one more thing, could you provide some codes about the core grad cliping method? Several lines of codes are also ok. I only need the grad clipping part codes. I can't wait to use sd xl to generate 3d.

Thank you for your help.