possible bug in stan_summary

gbarz commented 3 years ago

In Rel 6.1.6 - Fix for update stansummary option switch, the code on lines 37 and 82 was changed from:
cmd = $(pstring) --csv_file=$(csvfile) $(file) to: cmd = $(pstring) -c $(csvfile) $(file)

I believe this change may be causing the generation of the "model"_summary.csv files to fail (at least for me). I have tried both versions of the command-line options directly in CmdStan (2.23.0) stansummary and only the original successfully generates the target summary.csv file output.

goedman commented 3 years ago

Thank you for reporting this. Which cmdstan version are you using? cmdstan v2.25.0 and 2.26.0/1? or older ones?

goedman commented 3 years ago

Actually, you've given me the answer on my question already (cmdstan v2.23.0). Is it a problem updating to a more recent version?

gbarz commented 3 years ago

I am using 2.23.0 presently. Please let me know this is not an issue in a later CmdStan version. I had some issues with Stanc3 and external functions in CmdStan so did not upgrade to 2.25. However, I have not yet tried 2.26.

On Feb 11, 2021, at 12:35 PM, Rob J Goedman notifications@github.com wrote:

Thank you for reporting this. Which cmdstan version are you using? cmdstan v2.25.0 and 2.26.0/1? or older ones?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/StanJulia/CmdStan.jl/issues/108#issuecomment-777665069, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA2MOIHEUAH7TEACE7JJHSLS6QINZANCNFSM4XPFPW2Q.

goedman commented 3 years ago

Hi @gbarz

Assuming you run in a base environment, something like this should fix it:

(@v1.5) pkg> st
Status `~/.julia/environments/v1.5/Project.toml`
  [634d3b9d] DrWatson v1.18.1
  [c3e4b0f8] Pluto v0.12.20
  [7f904dfe] PlutoUI v0.7.1
  [44cfe95a] Pkg

(@v1.5) pkg> add CmdStan@6.1.5
   Updating registry at `~/.julia/registries/General`
  Resolving package versions...
Updating `~/.julia/environments/v1.5/Project.toml`
  [593b3428] + CmdStan v6.1.5
Updating `~/.julia/environments/v1.5/Manifest.toml`
  [336ed68f] + CSV v0.8.3
  [324d7699] + CategoricalArrays v0.9.2
  ... (snipped)
  [189a3867] + Reexport v1.0.0
  [91c51154] + SentinelArrays v1.2.16
  [a2af1166] + SortingAlgorithms v0.3.1
  [856f2bd8] + StructTypes v1.2.3
  [8bb1440f] + DelimitedFiles
  [9fa8497b] + Future
  [a63ad114] + Mmap
  [1a1011a3] + SharedArrays
  [2f01184e] + SparseArrays
  [10745b16] + Statistics

julia> using CmdStan
[ Info: Precompiling CmdStan [593b3428-ca2f-500c-ae53-031589ec8ddd]

julia> CmdStan.set_cmdstan_home!("/Users/rob/Projects/StanSupport/cmdstan-2.23.0")
"/Users/rob/Projects/StanSupport/cmdstan-2.23.0"

julia> include("/Users/rob/.julia/dev/CmdStan/examples/Bernoulli/bernoulli.jl");

File /Users/rob/.julia/dev/CmdStan/examples/Bernoulli/tmp/bernoulli.stan will be updated.

8×10 DataFrame
 Row │ parameters     mean       mcse          std       5%         50%        95%        ess         n_eff/ ⋯
     │ Symbol         Float64    Float64       Float64   Float64    Float64    Float64    Float64     Float6 ⋯
─────┼────────────────────────────────────────────────────────────────────────────────────────────────────────
   1 │ lp__           -8.15045     0.0196351   0.775619  -9.68573   -7.85174   -7.64011   1560.38     15372. ⋯
   2 │ accept_stat__   0.916214    0.00755291  0.12826    0.635285   0.971456   1.0        288.374     2840.
   3 │ stepsize__      1.01934     0.100239    0.141972   0.795438   1.12531    1.15669      2.00602     19.
   4 │ treedepth__     1.415       0.00924631  0.523778   1.0        1.0        2.0       3208.91     31613.
   5 │ n_leapfrog__    2.534       0.222701    1.29895    1.0        3.0        3.0         34.0205     335. ⋯
   6 │ divergent__     0.0       NaN           0.0        0.0        0.0        0.0        NaN          NaN
   7 │ energy__        8.64003     0.0262001   1.04102    7.68432    8.30297   10.6538    1578.74     15553.
   8 │ theta           0.340567    0.00358637  0.129292   0.139214   0.334716   0.565987  1299.67     12804.
                                                                                             2 columns omitted

1×3 DataFrame
 Row │ mean      ess      r_hat   
     │ Float64   Float64  Float64 
─────┼────────────────────────────
   1 │ 0.340567  1299.67  1.00252

If you have a MWE of the issue you encountered with stanc3 I could have a look at that.

Let me know if this works for you.

Rob

gbarz commented 3 years ago

Thank you for the suggestion. Pinning CmdStan.jl to 6.1.5 has fixed my issue.

Additionally, I will try the latest CmdStan release 2.26.0 with the updated CmdStan.jl v6.1.7 version and let you know how it goes..

Very much appreciate the prompt reply as well as the Julia package.

On Feb 11, 2021, at 3:17 PM, Rob J Goedman notifications@github.com wrote:

Hi @gbarz https://github.com/gbarz Assuming you run in a base environment, something like this should fix it:

(@v1.5) pkg> st Status ~/.julia/environments/v1.5/Project.toml [634d3b9d] DrWatson v1.18.1 [c3e4b0f8] Pluto v0.12.20 [7f904dfe] PlutoUI v0.7.1 [44cfe95a] Pkg

(@v1.5) pkg> add CmdStan@6.1.5 Updating registry at ~/.julia/registries/General Resolving package versions... Updating ~/.julia/environments/v1.5/Project.toml [593b3428] + CmdStan v6.1.5 Updating ~/.julia/environments/v1.5/Manifest.toml [336ed68f] + CSV v0.8.3 [324d7699] + CategoricalArrays v0.9.2 ... (snipped) [189a3867] + Reexport v1.0.0 [91c51154] + SentinelArrays v1.2.16 [a2af1166] + SortingAlgorithms v0.3.1 [856f2bd8] + StructTypes v1.2.3 [8bb1440f] + DelimitedFiles [9fa8497b] + Future [a63ad114] + Mmap [1a1011a3] + SharedArrays [2f01184e] + SparseArrays [10745b16] + Statistics

julia> using CmdStan [ Info: Precompiling CmdStan [593b3428-ca2f-500c-ae53-031589ec8ddd]

julia> CmdStan.set_cmdstan_home!("/Users/rob/Projects/StanSupport/cmdstan-2.23.0") "/Users/rob/Projects/StanSupport/cmdstan-2.23.0"

julia> include("/Users/rob/.julia/dev/CmdStan/examples/Bernoulli/bernoulli.jl");

File /Users/rob/.julia/dev/CmdStan/examples/Bernoulli/tmp/bernoulli.stan will be updated.

8×10 DataFrame Row │ parameters mean mcse std 5% 50% 95% ess n_eff/ ⋯ │ Symbol Float64 Float64 Float64 Float64 Float64 Float64 Float64 Float6 ⋯ ─────┼──────────────────────────────────────────────────────────────────────────────────────────────────────── 1 │ lp -8.15045 0.0196351 0.775619 -9.68573 -7.85174 -7.64011 1560.38 15372. ⋯ 2 │ accept_stat 0.916214 0.00755291 0.12826 0.635285 0.971456 1.0 288.374 2840. 3 │ stepsize 1.01934 0.100239 0.141972 0.795438 1.12531 1.15669 2.00602 19. 4 │ treedepth__ 1.415 0.00924631 0.523778 1.0 1.0 2.0 3208.91 31613. 5 │ n_leapfrog 2.534 0.222701 1.29895 1.0 3.0 3.0 34.0205 335. ⋯ 6 │ divergent 0.0 NaN 0.0 0.0 0.0 0.0 NaN NaN 7 │ energy 8.64003 0.0262001 1.04102 7.68432 8.30297 10.6538 1578.74 15553. 8 │ theta 0.340567 0.00358637 0.129292 0.139214 0.334716 0.565987 1299.67 12804. 2 columns omitted

1×3 DataFrame Row │ mean ess r_hat
│ Float64 Float64 Float64 ─────┼──────────────────────────── 1 │ 0.340567 1299.67 1.00252 If you have a MWE of the issue you encountered with stanc3 I could have a look at that.

Let me know if this works for you.

Rob

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/StanJulia/CmdStan.jl/issues/108#issuecomment-777762004, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA2MOIEISTK44NPSGTVRIADS6Q3N5ANCNFSM4XPFPW2Q.

goedman commented 3 years ago

Next Monday v2.26.1 should be available (contains a bug fix), so you might as well wait going through the build steps.

If it doesn't work and you can create a MWE I would love to add that to the CmdStan and Stan test cases.

gbarz commented 3 years ago

Per your suggestion, I tried the unpinned code with CmdStan 2.26.0. It ran without issue (and the earlier Stanc3 issue I was having in 2.25.0 has also been addressed)

Thanks for the prompt attention and helpful suggestions.

On Feb 11, 2021, at 3:17 PM, Rob J Goedman notifications@github.com wrote:

Hi @gbarz https://github.com/gbarz Assuming you run in a base environment, something like this should fix it:

(@v1.5) pkg> st Status ~/.julia/environments/v1.5/Project.toml [634d3b9d] DrWatson v1.18.1 [c3e4b0f8] Pluto v0.12.20 [7f904dfe] PlutoUI v0.7.1 [44cfe95a] Pkg

(@v1.5) pkg> add CmdStan@6.1.5 Updating registry at ~/.julia/registries/General Resolving package versions... Updating ~/.julia/environments/v1.5/Project.toml [593b3428] + CmdStan v6.1.5 Updating ~/.julia/environments/v1.5/Manifest.toml [336ed68f] + CSV v0.8.3 [324d7699] + CategoricalArrays v0.9.2 ... (snipped) [189a3867] + Reexport v1.0.0 [91c51154] + SentinelArrays v1.2.16 [a2af1166] + SortingAlgorithms v0.3.1 [856f2bd8] + StructTypes v1.2.3 [8bb1440f] + DelimitedFiles [9fa8497b] + Future [a63ad114] + Mmap [1a1011a3] + SharedArrays [2f01184e] + SparseArrays [10745b16] + Statistics

julia> using CmdStan [ Info: Precompiling CmdStan [593b3428-ca2f-500c-ae53-031589ec8ddd]

julia> CmdStan.set_cmdstan_home!("/Users/rob/Projects/StanSupport/cmdstan-2.23.0") "/Users/rob/Projects/StanSupport/cmdstan-2.23.0"

julia> include("/Users/rob/.julia/dev/CmdStan/examples/Bernoulli/bernoulli.jl");

File /Users/rob/.julia/dev/CmdStan/examples/Bernoulli/tmp/bernoulli.stan will be updated.

8×10 DataFrame Row │ parameters mean mcse std 5% 50% 95% ess n_eff/ ⋯ │ Symbol Float64 Float64 Float64 Float64 Float64 Float64 Float64 Float6 ⋯ ─────┼──────────────────────────────────────────────────────────────────────────────────────────────────────── 1 │ lp -8.15045 0.0196351 0.775619 -9.68573 -7.85174 -7.64011 1560.38 15372. ⋯ 2 │ accept_stat 0.916214 0.00755291 0.12826 0.635285 0.971456 1.0 288.374 2840. 3 │ stepsize 1.01934 0.100239 0.141972 0.795438 1.12531 1.15669 2.00602 19. 4 │ treedepth__ 1.415 0.00924631 0.523778 1.0 1.0 2.0 3208.91 31613. 5 │ n_leapfrog 2.534 0.222701 1.29895 1.0 3.0 3.0 34.0205 335. ⋯ 6 │ divergent 0.0 NaN 0.0 0.0 0.0 0.0 NaN NaN 7 │ energy 8.64003 0.0262001 1.04102 7.68432 8.30297 10.6538 1578.74 15553. 8 │ theta 0.340567 0.00358637 0.129292 0.139214 0.334716 0.565987 1299.67 12804. 2 columns omitted

1×3 DataFrame Row │ mean ess r_hat
│ Float64 Float64 Float64 ─────┼──────────────────────────── 1 │ 0.340567 1299.67 1.00252 If you have a MWE of the issue you encountered with stanc3 I could have a look at that.

Let me know if this works for you.

Rob

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/StanJulia/CmdStan.jl/issues/108#issuecomment-777762004, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA2MOIEISTK44NPSGTVRIADS6Q3N5ANCNFSM4XPFPW2Q.

StanJulia / CmdStan.jl

possible bug in stan_summary #108