search

flatironinstitute / mcmc-monitor

Monitor MCMC runs in the browser
Other
35 stars 0 forks source link

error parsing CSV file - "exceeds string limit" #107

Open mitzimorris opened 1 year ago

mitzimorris commented 1 year ago

I ran a model which has a large number of transformed parameters (75K). monitoring during sampling was fine; but attempts to view the saved sample in the monitor fail with this message:

 getChainsForRun umich_binomial_runs
Problem updating ./umich_binomial_runs/umich_binomial_mrp-202308271704-1-12678f.csv: Cannot create a string longer than 0x1fffffe8 characters
Problem updating ./umich_binomial_runs/umich_binomial_mrp-202308271704-2-12678f.csv: Cannot create a string longer than 0x1fffffe8 characters
Problem updating ./umich_binomial_runs/umich_binomial_mrp-202308271704-3-12678f.csv: Cannot create a string longer than 0x1fffffe8 characters
Problem updating ./umich_binomial_runs/umich_binomial_mrp-202308271704-4-12678f.csv: Cannot create a string longer than 0x1fffffe8 characters

the files are indeed very big:

~/github/toan/mrp-interface/interface/umich_binomial_runs (mitzi-edits)> du -sh *
544M    umich_binomial_mrp-202308271704-1-12678f.csv
544M    umich_binomial_mrp-202308271704-2-12678f.csv
545M    umich_binomial_mrp-202308271704-3-12678f.csv
544M    umich_binomial_mrp-202308271704-4-12678f.csv