michaelhallquist / MplusAutomation

The MplusAutomation package leverages the flexibility of the R language to automate latent variable model estimation and interpretation using Mplus, a powerful latent variable modeling program developed by Muthen and Muthen (www.statmodel.com). Specifically, MplusAutomation provides routines for creating related groups of models, running batches of models, and extracting and tabulating model parameters and fit statistics.
81 stars 46 forks source link

problem with ReadModels #195

Closed sophiahn closed 7 months ago

sophiahn commented 8 months ago

I am trying to use readModels to read multiple Mplus outputs in a specific directory. I run into an error messgae like below: This started showing only recently after I had updated my R and Rstudio.

<simpleError in length(covcoverageSubsections) == 0 || is.na(covcoverageSubsections): 'length = 2' in coercion to 'logical(1)'><simpleError in !is.null(arglist$Parameters) && !is.na(arglist$Parameters): 'length = 3' in coercion to 'logical(1)'><simpleError in if (is.null(summaries) || missing(summaries) || summaries$NCategoricalLatentVars == 1 || is.na(summaries$NCategoricalLatentVars)) { if (is.null(summaries) || missing(summaries) || is.null(summaries$Mplus.version) || as.numeric(summaries$Mplus.version) < 7.3) { modelCounts <- getSection("^FINAL CLASS COUNTS AND PROPORTIONS FOR THE LATENT CLASSES$", outfiletext) ppCounts <- getSection("^FINAL CLASS COUNTS AND PROPORTIONS FOR THE LATENT CLASS PATTERNS$", outfiletext) mostLikelyCounts <- getSection("^CLASSIFICATION OF INDIVIDUALS BASED ON THEIR MOST LIKELY LATENT CLASS MEMBERSHIP$", outfiletext) } else { modelCounts <- getSection("^FINAL CLASS COUNTS AND PROPORTIONS FOR THE LATENT CLASSES$::^BASED ON THE ESTIMATED MODEL$", outfiletext) ppCounts <- getSection("^FINAL CLASS COUNTS AND PROPORTIONS FOR THE LATENT CLASSES$::^BASED ON ESTIMATED POSTERIOR PROBABILITIES$", outfiletext) mostLikelyCounts <- getSection("^FINAL CLASS COUNTS AND PROPORTIONS FOR THE LATENT CLASSES$::^BASED ON THEIR MOST LIKELY LATENT CLASS MEMBERSHIP$", outfiletext) } countlist[["modelEstimated"]] <- getClassCols(modelCounts) countlist[["posteriorProb"]] <- getClassCols(ppCounts) countlist[["mostLikely"]] <- getClassCols(mostLikelyCounts) mostLikelyProbs <- getSection("^Average Latent Class Probabilities for Most Likely Latent Class Membership \((Row|Column)\)$", outfiletext) if (length(mostLikelyProbs) > 1L) { mostLikelyProbs <- mostLikelyProbs[-1L] } countlist[["avgProbs.mostLikely"]] <- unlabeledMatrixExtract(mostLikelyProbs, filename) classificationProbs <- getSection("^Classification Probabilities for the Most Likely Latent Class Membership \((Column|Row)\)$", outfiletext) if (length(classificationProbs) > 1L) { classificationProbs <- classificationProbs[-1L] } countlist[["classificationProbs.mostLikely"]] <- unlabeledMatrixExtract(classificationProbs, filename) classificationLogitProbs <- getSection("^Logits for the Classification Probabilities for the Most Likely Latent Class Membership \((Column|Row)\)$", outfiletext) if (length(classificationLogitProbs) > 1L) { classificationLogitProbs <- classificationLogitProbs[-1L] } countlist[["logitProbs.mostLikely"]] <- unlabeledMatrixExtract(classificationLogitProbs, filename)} else { getClassCols_lta <- function(sectiontext) { numberLines <- grep("^\s([a-zA-Z0-9]+)?(\s+[0-9\.-]{1,}){1,}$", sectiontext, perl = TRUE) if (length(numberLines) > 0) { parsedlines <- strsplit(trimSpace(sectiontext[numberLines]), "\s+", perl = TRUE) num_values <- sapply(parsedlines, length) if (length(unique(num_values)) == 1) { counts <- data.frame(t(sapply(parsedlines, as.numeric)), stringsAsFactors = FALSE) } else { parsedlines[which(num_values != max(num_values))] <- lapply(parsedlines[which(num_values != max(num_values))], function(x) { c(rep(NA, (max(num_values) - length(x))), x) }) counts <- do.call(rbind, parsedlines) counts[, 1] <- inverse.rle(list(lengths = diff(c(which(!is.na(counts[, 1])), (nrow(counts) + 1))), values = counts[, 1][complete.cases(counts[, 1])])) counts <- data.frame(counts, stringsAsFactors = FALSE) counts[, 2:4] <- lapply(counts[, 2:4], as.numeric) } return(counts) } else { return(NULL) } } if (missing(summaries) || is.null(summaries$Mplus.version) || as.numeric(summaries$Mplus.version) < 7.3) { posteriorProb.patterns <- getSection("^FINAL CLASS COUNTS AND PROPORTIONS FOR THE LATENT CLASSES$::^BASED ON ESTIMATED POSTERIOR PROBABILITIES$", outfiletext) mostLikely.patterns <- getSection("^CLASSIFICATION OF INDIVIDUALS BASED ON THEIR MOST LIKELY LATENT CLASS PATTERN$", outfiletext) mostLikelyCounts <- getSection("CLASSIFICATION OF INDIVIDUALS BASED ON THEIR MOST LIKELY LATENT CLASS PATTERN", outfiletext) } else { posteriorProb.patterns <- getSection("^FINAL CLASS COUNTS AND PROPORTIONS FOR THE LATENT CLASS PATTERNS$::^BASED ON ESTIMATED POSTERIOR PROBABILITIES$", outfiletext) mostLikely.patterns <- getSection("^FINAL CLASS COUNTS AND PROPORTIONS FOR THE LATENT CLASS PATTERNS$::^BASED ON THEIR MOST LIKELY LATENT CLASS PATTERN$", outfiletext) mostLikelyCounts <- getSection("^FINAL CLASS COUNTS AND PROPORTIONS FOR EACH LATENT CLASS VARIABLE$::^BASED ON THEIR MOST LIKELY LATENT CLASS PATTERN$", outfiletext) } countlist[["modelEstimated"]] <- getClassCols_lta(getSection("^FINAL CLASS COUNTS AND PROPORTIONS FOR EACH LATENT CLASS VARIABLE$::^BASED ON THE ESTIMATED MODEL$", outfiletext)) countlist[["posteriorProb"]] <- getClassCols_lta(getSection("^FINAL CLASS COUNTS AND PROPORTIONS FOR EACH LATENT CLASS VARIABLE$::^BASED ON ESTIMATED POSTERIOR PROBABILITIES$", outfiletext)) countlist[["mostLikely"]] <- getClassCols_lta(mostLikelyCounts) countlist[which(names(countlist) %in% c("modelEstimated", "posteriorProb", "mostLikely"))] <- lapply(countlist[which(names(countlist) %in% c("modelEstimated", "posteriorProb", "mostLikely"))], setNames, c("variable", "class", "count", "proportion")) countlist[["modelEstimated.patterns"]] <- getClassCols_lta(getSection("^FINAL CLASS COUNTS AND PROPORTIONS FOR THE LATENT CLASS PATTERNS$::^BASED ON THE ESTIMATED MODEL$", outfiletext)) countlist[["posteriorProb.patterns"]] <- getClassCols_lta(posteriorProb.patterns) countlist[["mostLikely.patterns"]] <- getClassCols_lta(mostLikely.patterns) morder <- getSection("MODEL RESULTS USE THE LATENT CLASS VARIABLE ORDER", outfiletext) if (!is.null(morder)) { first_present <- which(morder != "")[1] catvar_names <- strsplit(trimSpace(morder[first_present]), "\s+")[[1]] } else { catvar_names <- unique(countlist[["mostLikely"]]$variable) } rename_sections <- which(names(countlist) %in% c("modelEstimated.patterns", "posteriorProb.patterns", "mostLikely.patterns")) if (length(rename_sections) > 0L) { for (s in rename_sections) { countlist[[s]] <- setNames(countlist[[s]], c(catvar_names, "count", "proportion")) } } avgProbs <- getSection("^Average Latent Class Probabilities for Most Likely Latent Class Pattern \((Row|Column)\)$::^by Latent Class Pattern \((Row|Column)\)$", outfiletext) if (!is.null(avgProbs)) { column_headers <- strsplit(trimws(grep("\sLatent Class\s{2,}", avgProbs, value = TRUE)), "\s+", perl = TRUE)[[1]][-1] variable_pattern_rows <- grep(paste(c("^(\s{2,}\d+){", length(column_headers), "}$"), collapse = ""), avgProbs, perl = TRUE) variable_pattern_rows <- variable_pattern_rows[!c(FALSE, diff(variable_pattern_rows) != 1)] variable_patterns <- avgProbs[variable_pattern_rows] variable_patterns <- data.frame(t(sapply(strsplit(trimws(variable_patterns), "\s+", perl = TRUE), as.numeric))) names(variable_patterns) <- c("Latent Class Pattern No.", column_headers[-1]) probs <- grep(paste(c("^\s+\d{1,}(\s{2,}[0-9\.-]+)+$"), collapse = ""), avgProbs[(variable_pattern_rows[length(variable_pattern_rows)] + 1):length(avgProbs)], perl = TRUE, value = TRUE) if (length(probs)%%nrow(variable_patterns) > 1) { for (i in 2:(length(probs)%%nrow(variable_patterns))) { probs[1:(nrow(variable_patterns) + 1)] <- paste(probs[1:(nrow(variable_patterns) + 1)], substring(probs[((i - 1) (nrow(variable_patterns) + 1) + 1):(i (nrow(variable_patterns) + 1))], first = 8)) } probs <- probs[1:nrow(variable_patterns)] } probs <- t(sapply(strsplit(trimws(probs[-1]), "\s+", perl = TRUE), as.numeric))[, -1] countlist[["avgProbs.mostLikely"]] <- probs countlist[["avgProbs.mostLikely.patterns"]] <- variable_patterns } countlist[["classificationProbs.mostLikely"]] <- NULL countlist[["logitProbs.mostLikely"]] <- NULL transitionProbs <- getSection("^LATENT TRANSITION PROBABILITIES BASED ON THE ESTIMATED MODEL$", outfiletext) if (!is.null(transitionProbs)) { section_starts <- grep("\(Columns\)$", transitionProbs) transitionProbs <- mapply(FUN = function(begin, end) { probs <- grep("^\s+\d{1,}(\s{2,}[0-9\.-]{2,}){1,}$", transitionProbs[begin:end], perl = TRUE, value = TRUE) probs <- do.call(rbind, strsplit(trimws(probs), "\s+", perl = TRUE))[, -1] cbind(paste(gsub("\s+(\w+) Classes.$", "\1", transitionProbs[begin]), ".", rep(c(1:nrow(probs)), ncol(probs)), sep = ""), paste(gsub(".+?by (\w+) Classes.$", "\1", transitionProbs[begin]), ".", as.vector(sapply(1:ncol(probs), rep, nrow(probs))), sep = ""), as.vector(probs)) }, begin = section_starts, end = c(section_starts[-1], length(transitionProbs)), SIMPLIFY = FALSE) if (length(transitionProbs) > 1) { transitionProbs <- do.call(rbind, transitionProbs) } else { transitionProbs <- transitionProbs[[1]] } transitionProbs <- data.frame(transitionProbs, stringsAsFactors = FALSE) names(transitionProbs) <- c("from", "to", "probability") transitionProbs$probability <- as.numeric(transitionProbs$probability) } countlist[["transitionProbs"]] <- transitionProbs}: missing value where TRUE/FALSE needed><simpleError in length(covcoverageSubsections) == 0 || is.na(covcoverageSubsections): 'length = 2' in coercion to 'logical(1)'><simpleError in !is.null(arglist$Parameters) && !is.na(arglist$Parameters): 'length = 3' in coercion to 'logical(1)'><simpleError in if (is.null(summaries) || missing(summaries) || summaries$NCategoricalLatentVars == 1 || is.na(summaries$NCategoricalLatentVars)) { if (is.null(summaries) || missing(summaries) || is.null(summaries$Mplus.version) || as.numeric(summaries$Mplus.version) < 7.3) { modelCounts <- getSection("^FINAL CLASS COUNTS AND PROPORTIONS FOR THE LATENT CLASSES$", outfiletext) ppCounts <- getSection("^FINAL CLASS COUNTS AND PROPORTIONS FOR THE LATENT CLASS PATTERNS$", outfiletext) mostLikelyCounts <- getSection("^CLASSIFICATION OF INDIVIDUALS BASED ON THEIR MOST LIKELY LATENT CLASS MEMBERSHIP$", outfiletext) } else { modelCounts <- getSection("^FINAL CLASS COUNTS AND PROPORTIONS FOR THE LATENT CLASSES$::^BASED ON THE ESTIMATED MODEL$", outfiletext) ppCounts <- getSection("^FINAL CLASS COUNTS AND PROPORTIONS FOR THE LATENT CLASSES$::^BASED ON ESTIMATED POSTERIOR PROBABILITIES$", outfiletext) mostLikelyCounts <- getSection("^FINAL CLASS COUNTS AND PROPORTIONS FOR THE LATENT CLASSES$::^BASED ON THEIR MOST LIKELY LATENT CLASS MEMBERSHIP$", outfiletext) } countlist[["modelEstimated"]] <- getClassCols(modelCounts) countlist[["posteriorProb"]] <- getClassCols(ppCounts) countlist[["mostLikely"]] <- getClassCols(mostLikelyCounts) mostLikelyProbs <- getSection("^Average Latent Class Probabilities for Most Likely Latent Class Membership \((Row|Column)\)$", outfiletext) if (length(mostLikelyProbs) > 1L) { mostLikelyProbs <- mostLikelyProbs[-1L] } countlist[["avgProbs.mostLikely"]] <- unlabeledMatrixExtract(mostLikelyProbs, filename) classificationProbs <- getSection("^Classification Probabilities for the Most Likely Latent Class Membership \((Column|Row)\)$", outfiletext) if (length(classificationProbs) > 1L) { classificationProbs <- classificationProbs[-1L] } countlist[["classificationProbs.mostLikely"]] <- unlabeledMatrixExtract(classificationProbs, filename) classificationLogitProbs <- getSection("^Logits for the Classification Probabilities for the Most Likely Latent Class Membership \((Column|Row)\)$", outfiletext) if (length(classificationLogitProbs) > 1L) { classificationLogitProbs <- classificationLogitProbs[-1L] } countlist[["logitProbs.mostLikely"]] <- unlabeledMatrixExtract(classificationLogitProbs, filename)} else { getClassCols_lta <- function(sectiontext) { numberLines <- grep("^\s([a-zA-Z0-9]+)?(\s+[0-9\.-]{1,}){1,}$", sectiontext, perl = TRUE) if (length(numberLines) > 0) { parsedlines <- strsplit(trimSpace(sectiontext[numberLines]), "\s+", perl = TRUE) num_values <- sapply(parsedlines, length) if (length(unique(num_values)) == 1) { counts <- data.frame(t(sapply(parsedlines, as.numeric)), stringsAsFactors = FALSE) } else { parsedlines[which(num_values != max(num_values))] <- lapply(parsedlines[which(num_values != max(num_values))], function(x) { c(rep(NA, (max(num_values) - length(x))), x) }) counts <- do.call(rbind, parsedlines) counts[, 1] <- inverse.rle(list(lengths = diff(c(which(!is.na(counts[, 1])), (nrow(counts) + 1))), values = counts[, 1][complete.cases(counts[, 1])])) counts <- data.frame(counts, stringsAsFactors = FALSE) counts[, 2:4] <- lapply(counts[, 2:4], as.numeric) } return(counts) } else { return(NULL) } } if (missing(summaries) || is.null(summaries$Mplus.version) || as.numeric(summaries$Mplus.version) < 7.3) { posteriorProb.patterns <- getSection("^FINAL CLASS COUNTS AND PROPORTIONS FOR THE LATENT CLASSES$::^BASED ON ESTIMATED POSTERIOR PROBABILITIES$", outfiletext) mostLikely.patterns <- getSection("^CLASSIFICATION OF INDIVIDUALS BASED ON THEIR MOST LIKELY LATENT CLASS PATTERN$", outfiletext) mostLikelyCounts <- getSection("CLASSIFICATION OF INDIVIDUALS BASED ON THEIR MOST LIKELY LATENT CLASS PATTERN", outfiletext) } else { posteriorProb.patterns <- getSection("^FINAL CLASS COUNTS AND PROPORTIONS FOR THE LATENT CLASS PATTERNS$::^BASED ON ESTIMATED POSTERIOR PROBABILITIES$", outfiletext) mostLikely.patterns <- getSection("^FINAL CLASS COUNTS AND PROPORTIONS FOR THE LATENT CLASS PATTERNS$::^BASED ON THEIR MOST LIKELY LATENT CLASS PATTERN$", outfiletext) mostLikelyCounts <- getSection("^FINAL CLASS COUNTS AND PROPORTIONS FOR EACH LATENT CLASS VARIABLE$::^BASED ON THEIR MOST LIKELY LATENT CLASS PATTERN$", outfiletext) } countlist[["modelEstimated"]] <- getClassCols_lta(getSection("^FINAL CLASS COUNTS AND PROPORTIONS FOR EACH LATENT CLASS VARIABLE$::^BASED ON THE ESTIMATED MODEL$", outfiletext)) countlist[["posteriorProb"]] <- getClassCols_lta(getSection("^FINAL CLASS COUNTS AND PROPORTIONS FOR EACH LATENT CLASS VARIABLE$::^BASED ON ESTIMATED POSTERIOR PROBABILITIES$", outfiletext)) countlist[["mostLikely"]] <- getClassCols_lta(mostLikelyCounts) countlist[which(names(countlist) %in% c("modelEstimated", "posteriorProb", "mostLikely"))] <- lapply(countlist[which(names(countlist) %in% c("modelEstimated", "posteriorProb", "mostLikely"))], setNames, c("variable", "class", "count", "proportion")) countlist[["modelEstimated.patterns"]] <- getClassCols_lta(getSection("^FINAL CLASS COUNTS AND PROPORTIONS FOR THE LATENT CLASS PATTERNS$::^BASED ON THE ESTIMATED MODEL$", outfiletext)) countlist[["posteriorProb.patterns"]] <- getClassCols_lta(posteriorProb.patterns) countlist[["mostLikely.patterns"]] <- getClassCols_lta(mostLikely.patterns) morder <- getSection("MODEL RESULTS USE THE LATENT CLASS VARIABLE ORDER", outfiletext) if (!is.null(morder)) { first_present <- which(morder != "")[1] catvar_names <- strsplit(trimSpace(morder[first_present]), "\s+")[[1]] } else { catvar_names <- unique(countlist[["mostLikely"]]$variable) } rename_sections <- which(names(countlist) %in% c("modelEstimated.patterns", "posteriorProb.patterns", "mostLikely.patterns")) if (length(rename_sections) > 0L) { for (s in rename_sections) { countlist[[s]] <- setNames(countlist[[s]], c(catvar_names, "count", "proportion")) } } avgProbs <- getSection("^Average Latent Class Probabilities for Most Likely Latent Class Pattern \((Row|Column)\)$::^by Latent Class Pattern \((Row|Column)\)$", outfiletext) if (!is.null(avgProbs)) { column_headers <- strsplit(trimws(grep("\sLatent Class\s{2,}", avgProbs, value = TRUE)), "\s+", perl = TRUE)[[1]][-1] variable_pattern_rows <- grep(paste(c("^(\s{2,}\d+){", length(column_headers), "}$"), collapse = ""), avgProbs, perl = TRUE) variable_pattern_rows <- variable_pattern_rows[!c(FALSE, diff(variable_pattern_rows) != 1)] variable_patterns <- avgProbs[variable_pattern_rows] variable_patterns <- data.frame(t(sapply(strsplit(trimws(variable_patterns), "\s+", perl = TRUE), as.numeric))) names(variable_patterns) <- c("Latent Class Pattern No.", column_headers[-1]) probs <- grep(paste(c("^\s+\d{1,}(\s{2,}[0-9\.-]+)+$"), collapse = ""), avgProbs[(variable_pattern_rows[length(variable_pattern_rows)] + 1):length(avgProbs)], perl = TRUE, value = TRUE) if (length(probs)%%nrow(variable_patterns) > 1) { for (i in 2:(length(probs)%%nrow(variable_patterns))) { probs[1:(nrow(variable_patterns) + 1)] <- paste(probs[1:(nrow(variable_patterns) + 1)], substring(probs[((i - 1) (nrow(variable_patterns) + 1) + 1):(i (nrow(variable_patterns) + 1))], first = 8)) } probs <- probs[1:nrow(variable_patterns)] } probs <- t(sapply(strsplit(trimws(probs[-1]), "\s+", perl = TRUE), as.numeric))[, -1] countlist[["avgProbs.mostLikely"]] <- probs countlist[["avgProbs.mostLikely.patterns"]] <- variable_patterns } countlist[["classificationProbs.mostLikely"]] <- NULL countlist[["logitProbs.mostLikely"]] <- NULL transitionProbs <- getSection("^LATENT TRANSITION PROBABILITIES BASED ON THE ESTIMATED MODEL$", outfiletext) if (!is.null(transitionProbs)) { section_starts <- grep("\(Columns\)$", transitionProbs) transitionProbs <- mapply(FUN = function(begin, end) { probs <- grep("^\s+\d{1,}(\s{2,}[0-9\.-]{2,}){1,}$", transitionProbs[begin:end], perl = TRUE, value = TRUE) probs <- do.call(rbind, strsplit(trimws(probs), "\s+", perl = TRUE))[, -1] cbind(paste(gsub("\s+(\w+) Classes.$", "\1", transitionProbs[begin]), ".", rep(c(1:nrow(probs)), ncol(probs)), sep = ""), paste(gsub(".+?by (\w+) Classes.$", "\1", transitionProbs[begin]), ".", as.vector(sapply(1:ncol(probs), rep, nrow(probs))), sep = ""), as.vector(probs)) }, begin = section_starts, end = c(section_starts[-1], length(transitionProbs)), SIMPLIFY = FALSE) if (length(transitionProbs) > 1) { transitionProbs <- do.call(rbind, transitionProbs) } else { transitionProbs <- transitionProbs[[1]] } transitionProbs <- data.frame(transitionProbs, stringsAsFactors = FALSE) names(transitionProbs) <- c("from", "to", "probability") transitionProbs$probability <- as.numeric(transitionProbs$probability) } countlist[["transitionProbs"]] <- transitionProbs}: missing value where TRUE/FALSE needed>

michaelhallquist commented 8 months ago

Thanks for this. I have a hunch as to what is causing the problem, but will be much faster to diagnose and fix if you could upload a single .out file (change the file extension to .txt for Github upload). Could I ask you to upload a file?

Michael

sophiahn commented 8 months ago

Thanks so much for your prompt response.

Here is one of the output file in txt format. Is this what you meant? anxiety-2-multigroup-cfa-gender.txt

sophiahn commented 8 months ago

Follow-up:I reverted back to an older version of R (4.2.1), and the readModels syntax works now! To be more precise, it stil generates some errors, but now I can use SummaryTable function to the list created from the readModels syntax --to create a table of fit indices.

michaelhallquist commented 7 months ago

Hi there, I took a look at this and it read correctly in the readModels function in the current Github version of the package. I need to deploy this to CRAN so that others don't bump into similar problems. In the meantime, here are instructions for getting a working version of the package for your file.

library(devtools)

install_github("michaelhallquist/MplusAutomation")