search

bytedance / 1d-tokenizer

This repo contains the code for 1D tokenizer and generator
Apache License 2.0
560 stars 24 forks source link

torch.multinomial 而不是 torch.argmax() #51

Open Peterande opened 1 week ago

Peterande commented 1 week ago

作者您好，在RAR sample的过程中，我发现将sampled = torch.multinomial(probs, num_samples=1)改成torch.argmax(probs, dim=-1)，生成的结果完全崩坏、没有意义。这是什么原因导致的呢？ARLoss中算token是否正确不就是用的torch.argmax(shift_logits, dim=1)吗？

cornettoyu commented 1 week ago

Hi,

Thanks for sharing the findings. Unfortunately I do not have experience on using argmax (greedy search) for image generation, so I may not be able to share any insights on this