Open ptrblck opened 1 month ago
I updated the discussion thread with details about why the use of VSL_BRNGMCG31 in vslNewStream dooms exponential to returning just a subset of a predetermined ordered list of (2^31-1) possible values. This means any sequence returned from exponential_ is just a block of N sequential elements from the larger series of 2 billion. Youāll always see the same exact series of numbers generated in the same exact order.
bernoulli or any other method using VSL_BRNG_MCG31 likely suffers from a similar problem.
Here's the discussion thread again for reference: https://discuss.pytorch.org/t/bug-in-torch-multinomial-generated-distribution-is-modestly-incorrect-edit-this-is-a-regression-and-appears-to-be-due-to-an-analogous-bug-in-tensor-exponential/207149/5
Please note, that this only affects x86_64 platform, on aarch64 default RNG is used which seems to be free from that issue.
š Describe the bug
KFrank
andjeffc
narrowed down an issue intorch.multinomial
generating duplicated permutations using a low number of iterations. Details can be found here: https://discuss.pytorch.org/t/bug-in-torch-multinomial-generated-distribution-is-modestly-incorrect-edit-this-is-a-regression-and-appears-to-be-due-to-an-analogous-bug-in-tensor-exponential/207149/5Cross-posting:
I can add more details once I'm back at my workstation.
Versions
torch==2.4.0
cc @ezyang @gchanan @zou3519 @kadeng @msaroufim @fritzo @neerajprad @alicanb @nikitaved @jgong5 @mingfeima @XiaobingSuper @sanchitintel @ashokei @jingxu10 @frank-wei