Investigate using Pathfinder

[marcdotson]

Let's document in Writing, using the mle-optimize branch, what cmdstanr is doing when $optimize() finds the MLE. What does this mean for hierarchical models?

[marcdotson]

[marcdotson]

Instead, use the pathfinder branch to try and implement this new variational approximation method.

[marcdotson]

Currently requires using the release candidate:

cmdstanr::install_cmdstan(version = "2.33.0-rc1", cores = 4)

And possibly a workaround for it not being implemented in cmdstanr yet?

run_pathfinder = function(data,mod){
    data_file = tempfile(fileext='.json')
    output_file = tempfile(fileext='.csv')
    cmdstanr::write_stan_json(data,file=data_file)
    processx::run(
        command = mod$exe_file()
        , args = c(
            'pathfinder'
            , 'data'
            , paste0('file=',data_file)
            , 'output'
            , paste0('file=',output_file)
        )
        , echo_cmd = T
        , stdout = ""
        , stderr = "2>&1"
    )

    (
        paste0("grep '^[#l]' '",output_file,"'")
        %>% system(intern=T)
        %>% strsplit('\n')
        %>% unlist()
    ) -> header
    header_nlines = which(stringr::str_starts(header,'lp'))
    found_samples_col_names = unlist(strsplit(header[header_nlines],','))
    (
        data.table::fread(
            cmd = paste0(
                "tail -n+"
                , header_nlines + 1
                , " '"
                , output_file
                , "' | grep -v '^[#]' --color=never"
            )
            , data.table = FALSE
            , sep = ','
            , header = F
            , col.names = found_samples_col_names
            , colClasses = list(numeric=1:length(found_samples_col_names))
        )
        %>% as_tibble()
    ) ->
        out
    return(out)
}

[marcdotson]

That doesn't work. Keep an eye out for Pathfinder being implement in cmdstanpy first.