No Cross Attention between Seq Embedding and E_map

cure-lab / MagicDrive

[ICLR24] Official implementation of the paper “MagicDrive: Street View Generation with Diverse 3D Geometry Control”

https://gaoruiyuan.com/magicdrive/

GNU Affero General Public License v3.0

637 stars 38 forks source link

No Cross Attention between Seq Embedding and E_map #83

Open haibao-yu opened 1 month ago

haibao-yu commented 1 month ago

Thanks for your great work! I have a question about your implementation:

In Figure 3, the embeded road map is interacted with Seq Embedding and then added to input feature. However, I find the embeded road map is directly added to input feature in your implementation. Can you check it is right implementation?

https://github.com/cure-lab/MagicDrive/blob/6038a24de9fc4df641a9775aab3f735a3c590589/magicdrive/networks/unet_addon_rawbox.py#L743-L850

flymin commented 1 month ago

We follow the implementation of ControlNet, where the control signal (map) is first added with x (noisy latent) and then goes through the copied encoder block. The whole file you referred to is the encoder from Figure 3 (not just the conv block).

github-actions[bot] commented 1 month ago

This issue is stale because it has been open for 7 days with no activity. If you do not have any follow-ups, the issue will be closed soon.