Closed lkccu closed 1 week ago
Hi @lkccu ,
Does this imply that the ddpm module is not involved in updating the network parameters?
No. The denoising process was involved in the training process and evaluation through the call of WrappedModel
in LGD/diffusion/respace.py
. Please refer to L227 of LGQ/train_network_diffusion.py
. The one you mentioned was use for evaluation metrics, not training loss.
I am confused which is the right way to interpret updated net.pos_output.
It is our design choice to implement the proposed architecture in our paper using guiding points as attention maps. We add the guiding pos_output predicted from conditions to the actual pos_output predicted from the denoising process, or q_0 as you mentioned. This adding mechanism aligns with our network design.
The lgd architecture appears to be simliar to grconvnet.
They are similar to some extend. However, our network integrates combining features from ALBEF modules and adding guiding features, as I have explained in your previous question. The t_embedding
is the embedding from input timestep, together with text and image features, helps to predict guiding_points
.
I hope my explanations help. Please do not hesitate to contact me if you have any further questions.
Best regards, An Vuong
Thank you for your kind and so detailed explanation! It is so helpful. Regards, lkc
Hi, @andvg3 I want to train lgrconvnet3/clipfusion/proposed LGD mode in Grasp-anything++ dataset.What is the recommended settings? Best regards, lkc.
Hi @lkccu ,
You should first download both Grasp-Anything and Grasp-Anything++ datasets. Then please follow the step-by-step instructions provided in the README file, it should work properly. Please let me know if you have any further questions.
Best,
An Vuong
Closed due to inactivity.
Thank you for your great work. However, I am little confused on the released code. I will be highly appreciated if you could help me out. 1.In LGD/train_network_diffusion.py#L247, it appears that the loss computed by GaussianDiffusion:training_loss is superseded by that of net:compute_loss. Does this imply that the ddpm module is not involved in updating the network parameters? (https://github.com/andvg3/LGD/blob/2471385507a883ba9147624ce2b3983e691f5fd9/inference/models/lgdm/network.py#L140)
https://github.com/andvg3/LGD/blob/2471385507a883ba9147624ce2b3983e691f5fd9/diffusion/gaussian_diffusion.py#L808
https://github.com/andvg3/LGD/blob/2471385507a883ba9147624ce2b3983e691f5fd9/train_network_diffusion.py#L237-L247
2.In LGD/diffusion/gaussian_diffusion.py:#L867, the guiding_point is computed using net.pos_output, with pos_output being updated by x+net.pos_output. My interpretation is that x represents the noised q_score_map at time t, while net.pos_output might be the predicted q0 or noise added to the q_0 at time t. I am confused which is the right way to interpret updated net.pos_output: x + net.pos_output
https://github.com/andvg3/LGD/blob/2471385507a883ba9147624ce2b3983e691f5fd9/diffusion/gaussian_diffusion.py#L866-L868
https://github.com/andvg3/LGD/blob/2471385507a883ba9147624ce2b3983e691f5fd9/inference/models/lgdm/network.py#L114-L119
3.Furthermore, I've observed that t_embedding is not utilized in the computations. In LGD/inference/models/lgdm/network.py#L107, the lgd architecture appears to be simliar to grconvnet.
https://github.com/andvg3/LGD/blob/2471385507a883ba9147624ce2b3983e691f5fd9/inference/models/lgdm/network.py#L98-L117
I would be most grateful for any clarification you could provide on these matters. Thank you in advance for your time and consideration. Best regards, lkc.