stan-dev / cmdstan

CmdStan, the command line interface to Stan
https://mc-stan.org/users/interfaces/cmdstan
BSD 3-Clause "New" or "Revised" License
210 stars 93 forks source link

Sample with seed=0 and seed=1 gives identical draws #1241

Closed nsiccha closed 7 months ago

nsiccha commented 7 months ago

Reproducible Steps:

In e.g. ~/.cmdstan/cmdstan-2.34.0/examples/bernoulli:

./bernoulli sample data file=bernoulli.data.json random seed=0 output file=0.csv
./bernoulli sample data file=bernoulli.data.json random seed=1 output file=1.csv
diff *.csv

Current Output:

< # start_datetime = 2024-01-18 07:05:35 UTC
---
> # start_datetime = 2024-01-18 07:05:39 UTC
37c37
< #   seed = 0
---
> #   seed = 1
39c39
< #   file = 0.csv
---
> #   file = 1.csv
1054,1056c1054,1056
< #  Elapsed Time: 0.013 seconds (Warm-up)
< #                0.022 seconds (Sampling)
< #                0.035 seconds (Total)
---
> #  Elapsed Time: 0.003 seconds (Warm-up)
> #                0.008 seconds (Sampling)
> #                0.011 seconds (Total)

Expected Output:

I'd expect the draws to be different. The cmdstan help (e.g. bernoulli help-all) says 0 is a valid input:

  random
    Random number configuration
    Valid subarguments: seed

    seed=<x>
      Random number generator seed
      Valid values: non-negative integer < 4294967296  or -1 to generate seed from system time
      Defaults to -1

Additional Information:

Same behavior happens for v2.33.1

Current Version:

v2.34.0

WardBrian commented 7 months ago

This is not a Stan bug. The boost::ecuyer1988 RNG has this behavior all on its own: https://godbolt.org/z/acKhd41Pj

After https://github.com/stan-dev/stan/issues/3167 last year it was recommended we stop using this generator: https://github.com/boostorg/random/issues/92

nsiccha commented 7 months ago

Makes sense. I also had a very quick look at cmdstan's cpp, but nothing there jumped out to me as the source of this behavior :)

WardBrian commented 7 months ago

I opened https://github.com/stan-dev/stan/issues/3256, so I think we can close this one now