Open JINzezhong7 opened 8 months ago
hunto
commented
8 months ago Hi @JINzezhong7 ,
You can see the code below. For logits with shape [B, C], we reshape it to [B, C, 1, 1], and use 1x1 convolutions in diffusion loss (diffusion model, autoencoder).
BeiDaoya
commented
8 months ago Hi, for this diffusion distillation, is it diffkd with both features and logits? Or just diffkd with features. looking forward to your reply.
JINzezhong7
commented
7 months ago 您好，我最近在看您的diffusion knowledge distillation 的paper，然后我尝试将它应用在我的任务上，但我发现ddim_loss 不收敛，请问是什么原因呢。非常期待您的回答。
发件人: Tao Huang @.> 日期: 星期一, 2024年3月11日 上午10:43 收件人: hunto/image_classification_sota @.> 抄送: JIN, Zezhong [Student] @.>, Mention @.> 主题: Re: [hunto/image_classification_sota] about the dimension (Issue #14)
CAUTION: This email is not originated from PolyU. Do not click links or open attachments unless you recognize the sender and know the content is safe.
Hi @JINzezhong7https://github.com/JINzezhong7 ,
You can see the code below. For logits with shape [B, C], we reshape it to [B, C, 1, 1], and use 1x1 convolutions in diffusion loss (diffusion model, autoencoder).
― Reply to this email directly, view it on GitHubhttps://github.com/hunto/image_classification_sota/issues/14#issuecomment-1987527444, or unsubscribehttps://github.com/notifications/unsubscribe-auth/A7NOWSTPPS5TW7P4BUDUFZTYXUK3RAVCNFSM6AAAAABEKKWJA6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOBXGUZDONBUGQ. You are receiving this because you were mentioned.Message ID: @.***>
[https://www.polyu.edu.hk/emaildisclaimer/PolyU_Email_Signature.jpg]
Disclaimer:
This message (including any attachments) contains confidential information intended for a specific individual and purpose. If you are not the intended recipient, you should delete this message and notify the sender and The Hong Kong Polytechnic University (the University) immediately. Any disclosure, copying, or distribution of this message, or the taking of any action based on it, is strictly prohibited and may be unlawful.
The University specifically denies any responsibility for the accuracy or quality of information obtained through University E-mail Facilities. Any views and opinions expressed are only those of the author(s) and do not necessarily represent those of the University and the University accepts no liability whatsoever for any losses or damages incurred or caused to any party as a result of the use of such information.
Hi, thanks for opening the source code. I read the paper, I find you use logits and features before pooling to perform diffusion. but for the logits, I guess the dimension is [B, C] B is the batch size, and C is the class number. This will cause a dimension mismatch in autoencoder, how to solve it. Thanks for your reply.