Issue with stan?

[maxgarman]

I am running into an issue with the stan file functionality that was implemented recently. Im not sure if its an issue on my end specifically or something happening overall. I've put the traceback below.

File "/Users/maxgarman/AusteniteCalculator/app/app.py", line 670, in update_output stan_fit, unique_phases = compute_uncertainties.run_stan(results_table) File "/Users/maxgarman/AusteniteCalculator/app/compute_uncertainties.py", line 160, in run_stan iter_sampling=2000) File "/Users/maxgarman/opt/anaconda3/envs/gsas-AustCalc/lib/python3.7/site-packages/cmdstanpy/model.py", line 1188, in sample raise RuntimeError(msg) RuntimeError: Error during sampling:

Command and output files: RunSet: chains=4, chain_ids=[1, 2, 3, 4], num_processes=4 cmd (chain 1): ['/Users/maxgarman/AusteniteCalculator/stan_files/one_sample.exe', 'id=1', 'random', 'seed=79443', 'data', 'file=/var/folders/00/j86sgdl14j58ndx3twspz9600000gn/T/tmpqcl92vep/55mivgaw.json', 'output', 'file=/var/folders/00/j86sgdl14j58ndx3twspz9600000gn/T/tmpqcl92vep/one_sample8zfhqnda/one_sample-20220726120059_1.csv', 'method=sample', 'num_samples=2000', 'num_warmup=2000', 'algorithm=hmc', 'adapt', 'engaged=1'] retcodes=[-1, -1, -1, -1] per-chain output files (showing chain 1 only): csv_file: /var/folders/00/j86sgdl14j58ndx3twspz9600000gn/T/tmpqcl92vep/one_sample8zfhqnda/one_sample-20220726120059_1.csv console_msgs (if any): /var/folders/00/j86sgdl14j58ndx3twspz9600000gn/T/tmpqcl92vep/one_sample8zfhqnda/one_sample-20220726120059_0-stdout.txt Consider re-running with show_console=True if the above output is unclear!

[dnewton600]

Nice, one potential thing to check would be if the first stan example in /ExampleScripts/pf_uncertainty_stan.ipynb works, to see if it is at the stan level specifically or something more at the app level. (I suspect it is at the stan level). Perhaps we can have a quick chat to sort this out.

[dnewton600]

These conda environments are doing a great job with cross-workspace consistency lately lol

[creuzige]

I think I might be onto something with this issue. On a mac, we rarely can run .exe files directly, so that was my first hint that something might be different...

I looked at the cmdstan guide (https://mc-stan.org/docs/2_24/cmdstan-guide-2_24.pdf) and tried to run through the bernoulli example.

Following section 4.1,

$ ./bernoulli sample data file=bernoulli.data.json
-bash: ./bernoulli: No such file or directory

So it didn't seem like the stan program got built correctly, since there's nothing to run. Similarly in the stan_files directory, there isn't an executable file I can run.

I then tried the troubleshooting guide in section 1.5, and running the make file failed with a long string of error messages, but I'll only include the last one

unsupported tapi file type '!tapi-tbd' in YAML file '/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/lib/libSystem.tbd' for architecture x86_64
clang-12: error: linker command failed with exit code 1 (use -v to see invocation)

Hunting this down with some searches brought be back to the CmdStan guide, where the end of section 1.1 has a note: Note MacOS installations may include old version of the g++ compiler which is a version 4.2.1. CmdStan requires g++ at 4.9.3 or later. Trying to install later versions of g++ using homebrew or macports is no longer recommended; use the XCode toolchain.

I checked, and the g++ version I have is indeed 4.2.1. Unfortunately when I try to add things to the XCode Toolchain, nothing shows up... I'll check with some of our mac folks here to see if they have any ideas what to do.

[creuzige]

So some progress on this...

I got some help from another mac user, and the issue seems to be that there's a mismatch between g++ installers when nested in conda on the mac. But by explicitly defining the compiler, things mostly worked out.

To recompile cmdstan:

cd to cmdstan directory (/Users/creuzige/gsas2full/envs/stan-test/bin/cmdstan)

make clean-all
CXX=$(xcrun -f clang++) make build

Then we need to compile the .stan files (David has included the windows executables as .exe). These commands need to be run from the cmdstan directory.

CXX=$(xcrun -f clang++) make /Users/creuzige/Documents/NIST_Research/GitHub/AusteniteCalculator/stan_files/
one_sample
CXX=$(xcrun -f clang++) make /Users/creuzige/Documents/NIST_Research/GitHub/AusteniteCalculator/stan_files/multiple_samples

I couldn't run pf_uncertainty_stan.ipynb completely, I got the following error at the end (and further than before)

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Input In [19], in <cell line: 1>()
----> 1 fit.summary()

File ~/gsas2full/envs/stan-test/lib/python3.10/site-packages/cmdstanpy/stanfit/mcmc.py:505, in CmdStanMCMC.summary(self, percentiles, sig_figs)
    497     get_logger().warning(
    498         'Requesting %d significant digits of output, but CSV files'
    499         ' only have %d digits of precision.',
    500         sig_figs,
    501         csv_sig_figs,
    502     )
    503 sig_figs_str = f'--sig_figs={sig_figs}'
    504 cmd_path = os.path.join(
--> 505     cmdstan_path(), 'bin', 'stansummary' + EXTENSION
    506 )
    507 tmp_csv_file = 'stansummary-{}-'.format(self.runset._args.model_name)
    508 tmp_csv_path = create_named_text_file(
    509     dir=_TMPDIR, prefix=tmp_csv_file, suffix='.csv', name_only=True
    510 )

File ~/gsas2full/envs/stan-test/lib/python3.10/site-packages/cmdstanpy/utils.py:188, in cmdstan_path()
    186 cmdstan_dir = os.path.expanduser(os.path.join('~', _DOT_CMDSTAN))
    187 if not os.path.exists(cmdstan_dir):
--> 188     raise ValueError(
    189         'No CmdStan installation found, run command "install_cmdstan"'
    190         'or (re)activate your conda environment!'
    191     )
    192 latest_cmdstan = get_latest_cmdstan(cmdstan_dir)
    193 if latest_cmdstan is None:

ValueError: No CmdStan installation found, run command "install_cmdstan"or (re)activate your conda environment!

I think that may be the number of things I have open.

But I was able to run via the command line

>>> fit = model.sample(data=stan_data)
14:35:40 - cmdstanpy - INFO - CmdStan start processing
chain 1 |██████████████████████████████████████████████████████████████████████████████████████████| 00:00 Sampling completed
chain 2 |██████████████████████████████████████████████████████████████████████████████████████████| 00:00 Sampling completed
chain 3 |██████████████████████████████████████████████████████████████████████████████████████████| 00:00 Sampling completed
chain 4 |██████████████████████████████████████████████████████████████████████████████████████████| 00:00 Sampling completed

14:35:40 - cmdstanpy - INFO - CmdStan done processing.
14:35:40 - cmdstanpy - WARNING - Non-fatal error during sampling:
Exception: normal_lpdf: Scale parameter[7] is 0, but must be positive! (in '/Users/creuzige/Documents/NIST_Research/GitHub/AusteniteCalculator/stan_files/one_sample.stan', line 18, column 4 to column 49)
Exception: normal_lpdf: Scale parameter[1] is 0, but must be positive! (in '/Users/creuzige/Documents/NIST_Research/GitHub/AusteniteCalculator/stan_files/one_sample.stan', line 18, column 4 to column 49)
Exception: normal_lpdf: Scale parameter[7] is 0, but must be positive! (in '/Users/creuzige/Documents/NIST_Research/GitHub/AusteniteCalculator/stan_files/one_sample.stan', line 18, column 4 to column 49)
Consider re-running with show_console=True if the above output is unclear!
>>> end = time.time()
>>> fit.summary()
                   Mean      MCSE    StdDev         5%        50%        95%    N_Eff   N_Eff/s    R_hat
name                                                                                                    
lp__          20.165600  0.048319  1.652150  16.801100  20.552500  22.074200  1169.14   7085.72  1.00165
sigma_exp[1]   0.034419  0.000352  0.014641   0.018553   0.030913   0.062734  1727.99  10472.70  1.00158
sigma_exp[2]   0.053213  0.000654  0.026422   0.027408   0.046487   0.098773  1629.82   9877.71  0.99974
phase_mu[1]    0.057176  0.000343  0.014632   0.034167   0.057041   0.080575  1820.17  11031.30  1.00291
phase_mu[2]    0.279572  0.000628  0.025338   0.239918   0.279614   0.319426  1629.27   9874.35  1.00120

[creuzige]

Open items

• Check if the ipynb works better after reboot
• Add code to detect which OS is being used to choose the right stan executable

[creuzige]

Added note to README about this