search

avishkarsaha / translating-images-into-maps

Official PyTorch code for 'Translating Images Into Maps' ICRA 2022 (Outstanding Paper Award)
Other
408 stars 49 forks source link

Only the first vector out of the nn.Embedding is ever used #5

Open vnmusat opened 2 years ago

vnmusat commented 2 years ago

Hi,

Is there a specific reason why only the first vector out of the nn.Embedding is ever used? tgt8 ... tgt64 are always zeros at this stage so you end up picking the 0-th vector for each spatial position, or in other words qe8 .. qe64 will always be filled with identical repeating values.... the values in the Embedding will obviously change over time with training but they will always be repeated throughout the qe-s

https://github.com/avishkarsaha/translating-images-into-maps/blob/92b9627bef43e9a50f136c13a438a02be9ceebb2/src/model/network.py#L1150

https://github.com/avishkarsaha/translating-images-into-maps/blob/92b9627bef43e9a50f136c13a438a02be9ceebb2/src/model/network.py#L1161

After permuting qe8 to [batch, spatial, spatial, len_embedding]:

tensor([[[ 0.8230, -1.4024, -0.8630,  ..., -1.0915, -0.6329,  0.0921],
         [ 0.8230, -1.4024, -0.8630,  ..., -1.0915, -0.6329,  0.0921],
         [ 0.8230, -1.4024, -0.8630,  ..., -1.0915, -0.6329,  0.0921],
         ...,
         [ 0.8230, -1.4024, -0.8630,  ..., -1.0915, -0.6329,  0.0921],
         [ 0.8230, -1.4024, -0.8630,  ..., -1.0915, -0.6329,  0.0921],
         [ 0.8230, -1.4024, -0.8630,  ..., -1.0915, -0.6329,  0.0921]],

        [[ 0.8230, -1.4024, -0.8630,  ..., -1.0915, -0.6329,  0.0921],
         [ 0.8230, -1.4024, -0.8630,  ..., -1.0915, -0.6329,  0.0921],
         [ 0.8230, -1.4024, -0.8630,  ..., -1.0915, -0.6329,  0.0921],
         ...,
         [ 0.8230, -1.4024, -0.8630,  ..., -1.0915, -0.6329,  0.0921],
         [ 0.8230, -1.4024, -0.8630,  ..., -1.0915, -0.6329,  0.0921],
         [ 0.8230, -1.4024, -0.8630,  ..., -1.0915, -0.6329,  0.0921]],

        [[ 0.8230, -1.4024, -0.8630,  ..., -1.0915, -0.6329,  0.0921],
         [ 0.8230, -1.4024, -0.8630,  ..., -1.0915, -0.6329,  0.0921],
         [ 0.8230, -1.4024, -0.8630,  ..., -1.0915, -0.6329,  0.0921],
         ...,
         [ 0.8230, -1.4024, -0.8630,  ..., -1.0915, -0.6329,  0.0921],
         [ 0.8230, -1.4024, -0.8630,  ..., -1.0915, -0.6329,  0.0921],
         [ 0.8230, -1.4024, -0.8630,  ..., -1.0915, -0.6329,  0.0921]]
....
Chensiyu00 commented 2 years ago

Can you get the results shown on paper based on the given code?

zjr-bit commented 1 year ago

Hi，have you get the answer?