Save and load model object issue (cmdstanr)

[VojtechFiala]

Dear Rethinking team,

(The length of the report correspond to the time spend solving the issue on my own, maybe it contains hints that may save someone's wellbeing... temporarily... Plus there are many edist, originally I did not understand what I've just written.)

Here we go: I experience a problem saving ulam object (model object) lately.

I've been using Rethinking for a couple of years, fitting models with ulam. R version: 4.2.3 (2023-03-15 ucrt) -- "Shortstop Beagle" R studio version: 2024.04.1 Build 748 Rethinking package version: 2.31 (2023) cmdstanr version: 0.5.3 win11 home, AMD 7 5600H (plus I installed it like six times on different computers throghout the years. It worked just right.)

When the sampler's finished, the object of the model typically contains all the necessary parts: the data, formula, posterior... and can be saved by the functions:

save(model_object, "file_name.Rdata")

And then called back by load("file_name.Rdata"), the posterior can be extracted...

load("file_name.Rdata")

Recently, I installed the package on two computers (win10 pro Legion notebook and an i9 13900k win11 pro, detailed below). Below is the i9 versions' details (but please mind that the same issue appears on an old[er] Legion with win10 pro; please note I used admin accounts). R version: 4.4.1 (Race for Your Life) R studio version: 2024.09.0 Rethinking package version: 2.42 cmdstanr version: 0.8.1.9000

I followed the https://github.com/rmcelreath/rethinking#installation. (however, mind that the webpage https://mc-stan.org/docs/cmdstan-guide/installation.html is quite unclear; that way or another, when check_cmdstan_toolchain() is called, it says C++ toolchains ... is setup properly!) Path exists in the windows environment variables (but first I tried without setting this part), so does the cmdstan (installed with miniconda) and Rtools44.

The sampler works, posterior can be accessed... but the unsaved model object does not contain the posterior and IDK from where the computer loads it to the R environment. When saving the object of the model, it does saves everything but the posterior. Uploading the model back to the environment does not work. EDIT: When using standard installation, the object can be loaded but it does not contain the posterior (and is, therefore, useless). The difference when using the #425 is described below - simply to say, it sometimes works (it loads the object with posterior), sometimes it does not work at all (object cannot be uploaded).

When using the different version of rethinking: I followed this link: https://github.com/rmcelreath/rethinking/pull/425 Out of the two models using the same dataset, one was saved correctly and could be loaded into R (even on a different computer). The other (having similar size in the folder afterwards) could not. Funny thing is that both models' objects are of similar size, which one would take as a bagde of similar characteristics. I prefer not burning electricity and stress CPU for four hours for nothing. So I wont try for the third time lately. The error read:

Error in load("CZRSAUS_T_3_CORREL_FACES_09_10_24.Rdata") : 
  ReadItem: unknown type 63, perhaps written by later version of R

Before using https://github.com/rmcelreath/rethinking/pull/425 the error was different. Better to say: the object could be loaded into the environment, but it did not contain the posterior (the error read accordingly).

Saving the posterior separately does not help. I lack the proper expertise to send it back to the environment. Otherwise this would maybe help.

Given that it happen both on win10 and 11, a four years old and a brand new computer, I ruled out it's just win11 being stupid.

I have the following working hypotheses: (a) There were some changes in stan installation. Now it's simpler, most steps of the installation can be controlled from R console. But the process does not explicitly mention Rtools installation. That is why I mistakenly first installed R, R studio, cmdstan via miniconda, devtools, cmdstanr, and rethinking. Then, I was allerted that cmdstanr is missing path nad Rtools are missing. Subsequently I installed Rtools and configured the path for cmdstanr. Probably there is something I keep forgetting that prevents R to save the posterior properly (i.e., so that I can see the large "large ulam). Calling the model object or the posterior does not help, R can call it, but the "large ulam" object is not visible in the environmnent, either could be saved. Update: After using https://github.com/rmcelreath/rethinking/pull/425 it is still not visible in the environment, it can be saved, but not always loaded back.

Reinstalling everything and installing Rtools earlier does not help (one simply cannot uninstall R sw, btw.). Adding PATH in the environment manually does not help...

(b) Both the computers are localised in Polish language. The AMD Ryzen 7 is not. It speaks Czech.

I searched for similar previous errors. Based on that, please note that:

The posterior exists, it can be accessed but it cannot be saved and re-accessed. There are no errors up to the point I try to load the model object into a new session... I can play around at my free will with the posterior. I only cannot save it. One of the two PCs was a subject to rethinking installation in the summer (?August), the other now (October). No "maybe updating the cmdstanr/rstudio/" is likely to solve the problem.

(c) I am a regrettable creature without a trace of understanding and I deserve this, just spot my Eeenhglish (I mean, there is still some self-esteem left, but computers frequently behave like if this was the case).

That way or another..., I just... Please help.

d <- read.csv2("Whole_Dataset_long_MORF_08_10.csv",T, dec=",")

levels(as.factor(d$Sample_complete))
cor(d$Tru[d$Sample=="AUS"],d$Atr[d$Sample=="AUS"])
cor(d$Tru[d$Sample=="RSA"],d$Atr[d$Sample=="RSA"])
cor(d$Tru[d$Sample=="VN"],d$Atr[d$Sample=="VN"])

stanATTR <- function(x){(x-mean(d$Atr))/sd(d$Atr)}
unstanATTR <- function(x){x*sd(d$Atr)+mean(d$Atr)}

stanTRU <- function(x){(x-mean(d$Tru))/sd(d$Tru)}
unstanTRU <- function(x){x*sd(d$Tru)+mean(d$Tru)}

stanL <- function(x){(x-mean(d$L))/sd(d$L)}
unstanL <- function(x){x*sd(d$L)+mean(d$L)}

stanSShD <- function(x){(x-mean(d$SShD))/sd(d$SShD)}
unstanSShD <- function(x){x*sd(d$SShD)+mean(d$SShD)}

stanDIST <- function(x){(x-mean(d$DIST))/sd(d$DIST)}
unstanDIST <- function(x){x*sd(d$DIST)+mean(d$DIST)}

stanFA <- function(x){(x-mean(d$Asym))/sd(d$Asym)}
unstanFA <- function(x){x*sd(d$Asym)+mean(d$Asym)}

stanAge <- function(x){(x-mean(d$Age))/sd(d$Age)}
unstanAge <- function(x){x*sd(d$Age)+mean(d$Age)}

D_AT <- list(
  TrustwRating = stanTRU(d$Tru),
  AtrRating = stanATTR(d$Atr),
  FSMUi = as.numeric(as.factor(d$Sample_complete)), # Sample complete is: CZ Orig, CZ New Below, CZ New Above... 
  BSMUi = ifelse(d$Above_Below=="above",0.5,ifelse(d$Above_Below=="below",-0.5,0)), # 0.5 and -0.5 for the groups RSA and CZ new, 0 for the group from before...
  face = as.numeric(as.factor(d$Face_ID)),
  rater= as.numeric(as.factor(d$Particip_ID)),
  Age = stanAge(as.numeric(d$Age)), # Age of the stimuli
  dist = stanDIST(as.numeric(d$DIST)), # facial distinctiveness of the stimuli (morphometric measure)
  sshd = stanSShD(as.numeric(d$SShD)), # facial sextypicality of the stimuli (morphometric measure)
  FA = stanFA(as.numeric(d$Asym)), # facial asymmetry (morphometric measure)
  N_raters = length(unique(d$Particip_ID)),
  L = stanL(as.numeric(d$L))# Add the total number of raters
)

summary.data.frame(D_AT)

table(d$Sample_complete,D_AT$FSMUi)

summary.data.frame(D_AT)

library(rethinking)

CZRSAUS_A_1 <- ulam(alist(
  AtrRating ~ dnorm(mu, sigma_atr),

  mu <- a + a_group[FSMUi] + f_per_group[face,FSMUi] + z[rater]*sigma_rater, 

  a ~ dnorm(0, 0.5),

  a_group[FSMUi] ~ dnorm(0, 0.3),

  z[rater] ~ normal(0,1), 
  sigma_rater ~ dexp(1),

  gq > vector[rater]:a_rater <<- a + z*sigma_rater,

  transpars> matrix[face,9]:f_per_group <- compose_noncentered(sigma_FSMUi,L_Rho_FSMUi,z_FSMUi), 

  cholesky_factor_corr[9]:L_Rho_FSMUi ~ lkj_corr_cholesky(2),
  matrix[9,face]:z_FSMUi ~ normal(0, 1),
  vector[9]:sigma_FSMUi ~ dexp(1),

  gq> matrix[9,9]:Rho_FSMUi <<- Chol_to_Corr(L_Rho_FSMUi),

  sigma_atr ~ dexp(1)

), data=D_AT, iter=500, sample=T, cores=20, chains=20, cmdstan = T)

save(CZRSAUS_A_1, file="CZRSAUS_A_1_CORREL_FACES_09_10_24.Rdata")  # _A_ stands for "Attractiveness" 
CZRSAUS_A_1_COEFS <- precis(CZRSAUS_A_1, depth=3)
write.csv2(CZRSAUS_A_1_COEFS, file="CZRS_A_1_CORREL_FACES_COEFS_09_10_24.Rdata.csv")

load("CZRSAUS_A_1_CORREL_FACES_09_10_24.Rdata")  # _A_ stands for "Attractiveness" 
post <- extract.samples(CZRSAUS_A_1)
# This one works! 

CZRSAUS_TW_3 <- ulam(alist(
  TrustwRating ~ dnorm(mu, sigma_atr),

  mu <- a + a_group[FSMUi] + f_per_group[face,FSMUi] + z[rater]*sigma_rater, 

  # Big intercept for Country_1_orig
  a ~ dnorm(0, 0.5),

  # intercept change per country
  a_group[FSMUi] ~ dnorm(0, 0.5),

  z[rater] ~ normal(0,1), 
  sigma_rater ~ dexp(1),

  gq > vector[rater]:a_rater <<- a + z*sigma_rater,

  transpars> matrix[face,9]:f_per_group <- compose_noncentered(sigma_FSMUi,L_Rho_FSMUi,z_FSMUi), 

  cholesky_factor_corr[9]:L_Rho_FSMUi ~ lkj_corr_cholesky(2),
  matrix[9,face]:z_FSMUi ~ normal(0, 1),
  vector[9]:sigma_FSMUi ~ dexp(1),

  gq> matrix[9,9]:Rho_FSMUi <<- Chol_to_Corr(L_Rho_FSMUi),

  sigma_atr ~ dexp(1)

), data=D_AT, iter=500, sample=T, cores=20, chains=20)

save(CZRSAUS_TW_3, file="CZRSAUS_T_3_CORREL_FACES_09_10_24.Rdata")  
CZRSAUS_TW_3_COEFS <- precis(CZRSAUS_TW_3, depth=3)
write.csv2(CZRSAUS_TW_3_COEFS, file="CZRS_T_3_CORREL_FACES_COEFS_09_10_24.Rdata.csv")

load("CZRS_T_3_CORREL_FACES_COEFS_09_10_24.Rdata")  #  This one does not. 

[VojtechFiala]

The next day...

I think no action is needed to be taken. I see no obvious reason the 2nd model does not work. BUT: I tried 14 different models to-day, to get some variance, just in case. No variance: Using the up-to-date version of the package, I can save and load all the models in the environment. (1) In the same session. (2) Even after restarting R, (3) After restarting the computer (hoping win11 still clears MemoryCache) (4) On the above-mentioned Legion laptop. But not on the Ryzen 7, it uses stan, my dear - how this could only work. So it's a matter of compatibility between different versions of the package, plus something else that causes the 2nd model not to save correctly. I'll keep trying but for now. Dear rethinking user: If you experience troubles using saving the model, go for #425. If you want to upload the Rdata object with the model, mind the stan vs cmdstan versions of the packages. Like PlayStation used to be (IDK if it's still the case), the new can read the old, the old cannot read the new... ...my deepest apologies for this mess. VtF

[VojtechFiala]

Last update: save(CZRSAUS_TW_3, file="CZRSAUS_T_3_CORREL_FACES_09_10_24.Rdata")
load("CZRS_T_3_CORREL_FACES_COEFS_09_10_24.Rdata") # There is a typo... OMG. Closing all. The only problem is too much endeavour, too little attention and the lack of mutual compatibility between models from different versions of rethinking. Please, excuse...