When using prumerge_plus for training, it will concate top-k ids and arthimetic_sequence (which is spatially sampled tokens)
However, top-k ids and arthimetic_sequence could have overlap ids, and this will cause complement_idx() to get wrong complement ids.
Is this desired behavior? Please correct me if I was wrong.
Thank you.
fmy7834
commented
5 days ago
Greetings.
When using prumerge_plus for training, it will concate top-k ids and arthimetic_sequence (which is spatially sampled tokens) However, top-k ids and arthimetic_sequence could have overlap ids, and this will cause complement_idx() to get wrong complement ids.
Is this desired behavior? Please correct me if I was wrong. Thank you.
YC