Cholesky fails on a non-M1 system

[hrlai]

Hi there, facing chol failures on Windows. Decided to open a new thread in case this can be closed apart from the M1-related issues.

I was trying to fit some spatial correlation structure but couldn't calculate. mcmc also complains about failing to find initial values, but I think maybe isolate this to calculate first, before testing mcmc... Maybe I have simply mispecified the model.

library(greta)
#> Warning: package 'greta' was built under R version 4.2.3
#> 
#> Attaching package: 'greta'
#> The following objects are masked from 'package:stats':
#> 
#>     binomial, cov2cor, poisson
#> The following objects are masked from 'package:base':
#> 
#>     %*%, apply, backsolve, beta, chol2inv, colMeans, colSums, diag,
#>     eigen, forwardsolve, gamma, identity, rowMeans, rowSums, sweep,
#>     tapply

n <- 10

distance <- .7 ^ as.matrix(dist(1:n))
range <- exponential(1)
#> ℹ Initialising python and checking dependencies, this may take a moment.
#> ✔ Initialising python and checking dependencies ... done!               
log_sd <- normal(0, 1)
Sigma <- exp(log_sd - 0.5 * (distance * range)^2)
L <- chol(Sigma)
Z <- normal(0, 1, n)
e <- t(t(Z) %*% L)

calculate(e, nsim = 1)
#> Cholesky decomposition was not successful. The input might not be valid.
#>   [[node Cholesky (defined at \ops\gen_linalg_ops.py:830) ]]
#> 
#> Errors may have originated from an input operation.
#> Input Source operations connected to node Cholesky:
#>  Exp (defined at \ops\gen_math_ops.py:4064)
#> 
#> Original stack trace for 'Cholesky':
#>   File "\ops\gen_linalg_ops.py", line 830, in cholesky
#>     "Cholesky", input=input, name=name)
#>   File "\framework\op_def_library.py", line 788, in _apply_op_helper
#>     op_def=op_def)
#>   File "\util\deprecation.py", line 507, in new_func
#>     return func(*args, **kwargs)
#>   File "\framework\ops.py", line 3616, in create_op
#>     op_def=op_def)
#>   File "\framework\ops.py", line 2005, in __init__
#>     self._traceback = tf_stack.extract_stack()

^{Created on 2023-10-19 with reprex v2.0.2}

> sessionInfo()
R version 4.2.2 (2022-10-31 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19045)

Matrix products: default

locale:
[1] LC_COLLATE=English_New Zealand.utf8  LC_CTYPE=English_New Zealand.utf8    LC_MONETARY=English_New Zealand.utf8
[4] LC_NUMERIC=C                         LC_TIME=English_New Zealand.utf8    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] greta_0.4.3

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.11       pillar_1.9.0      compiler_4.2.2    prettyunits_1.1.1 base64enc_0.1-3   tools_4.2.2      
 [7] progress_1.2.2    digest_0.6.33     tibble_3.2.1      evaluate_0.21     jsonlite_1.8.7    lifecycle_1.0.3  
[13] lattice_0.21-8    pkgconfig_2.0.3   png_0.1-8         rlang_1.1.1       reprex_2.0.2      Matrix_1.6-1     
[19] cli_3.6.1         rstudioapi_0.15.0 yaml_2.3.7        parallel_4.2.2    xfun_0.40         fastmap_1.1.1    
[25] coda_0.19-4       withr_2.5.0       knitr_1.44        vctrs_0.6.3       fs_1.6.3          globals_0.16.2   
[31] hms_1.1.3         rappdirs_0.3.3    grid_4.2.2        reticulate_1.32.0 glue_1.6.2        listenv_0.9.0    
[37] R6_2.5.1          processx_3.8.2    fansi_1.0.4       parallelly_1.36.0 rmarkdown_2.24    callr_3.7.3      
[43] clipr_0.8.0       whisker_0.4.1     magrittr_2.0.3    htmltools_0.5.6   codetools_0.2-19  ps_1.7.5         
[49] tfruns_1.5.1      future_1.33.0     tensorflow_2.13.0 utf8_1.2.3        crayon_1.5.2     

[njtierney]

Thanks for this, @hrlai !

We are currently trying to debug an issue with cholesky, so hopefully this is related.

Here is the behaviour for an M1 mac running the TF2 branch (https://github.com/greta-dev/greta/tree/tf2-poke-tf-fun), and I'll post the output of the non M1 mac running TF1 soon.

It looks like you are using CRAN greta, so using TF2, correct?

So we don't get an error on M1 TF2 greta, but we get a bunch of NaN values, which is also undesirable.

library(greta)
#> 
#> Attaching package: 'greta'
#> The following objects are masked from 'package:stats':
#> 
#>     binomial, cov2cor, poisson
#> The following objects are masked from 'package:base':
#> 
#>     %*%, apply, backsolve, beta, chol2inv, colMeans, colSums, diag,
#>     eigen, forwardsolve, gamma, identity, rowMeans, rowSums, sweep,
#>     tapply
greta_sitrep()
#> ℹ checking if python available
#> ✔ python (v3.8) available
#> 
#> ℹ checking if TensorFlow available
#> ✔ TensorFlow (v2.11.0) available
#> 
#> ℹ checking if TensorFlow Probability available
#> ✔ TensorFlow Probability (v0.19.0) available
#> 
#> ℹ checking if greta conda environment available
#> ✔ greta conda environment available
#> 
#> ℹ Initialising python and checking dependencies, this may take a moment.
#> ✔ Initialising python and checking dependencies ... done!
#> 
#> ℹ greta is ready to use!
n <- 10
distance <- .7 ^ as.matrix(dist(1:n))
range <- exponential(1)
log_sd <- normal(0, 1)
Sigma <- exp(log_sd - 0.5 * (distance * range)^2)
L <- chol(Sigma)
Z <- normal(0, 1, n)
e <- t(t(Z) %*% L)
e
#> greta array (operation)
#> 
#>       [,1]
#>  [1,]  ?  
#>  [2,]  ?  
#>  [3,]  ?  
#>  [4,]  ?  
#>  [5,]  ?  
#>  [6,]  ?  
#>  [7,]  ?  
#>  [8,]  ?  
#>  [9,]  ?  
#> [10,]  ?
calculate(e, nsim = 1)
#> $e
#> , , 1
#> 
#>      [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
#> [1,]  NaN  NaN  NaN  NaN  NaN  NaN  NaN  NaN  NaN   NaN

^{Created on 2023-11-17 with reprex v2.0.2}

Session info

``` r sessioninfo::session_info() #> ─ Session info ─────────────────────────────────────────────────────────────── #> setting value #> version R version 4.3.2 (2023-10-31) #> os macOS Sonoma 14.0 #> system aarch64, darwin20 #> ui X11 #> language (EN) #> collate en_US.UTF-8 #> ctype en_US.UTF-8 #> tz Australia/Hobart #> date 2023-11-17 #> pandoc 3.1.1 @ /Applications/RStudio.app/Contents/Resources/app/quarto/bin/tools/ (via rmarkdown) #> #> ─ Packages ─────────────────────────────────────────────────────────────────── #> package * version date (UTC) lib source #> backports 1.4.1 2021-12-13 [1] CRAN (R 4.3.0) #> base64enc 0.1-3 2015-07-28 [1] CRAN (R 4.3.0) #> callr 3.7.3 2022-11-02 [1] CRAN (R 4.3.0) #> cli 3.6.1 2023-03-23 [1] CRAN (R 4.3.0) #> coda 0.19-4 2020-09-30 [1] CRAN (R 4.3.0) #> codetools 0.2-19 2023-02-01 [1] CRAN (R 4.3.2) #> crayon 1.5.2 2022-09-29 [1] CRAN (R 4.3.0) #> digest 0.6.33 2023-07-07 [1] CRAN (R 4.3.0) #> evaluate 0.23 2023-11-01 [1] CRAN (R 4.3.1) #> fastmap 1.1.1 2023-02-24 [1] CRAN (R 4.3.0) #> fs 1.6.3 2023-07-20 [1] CRAN (R 4.3.0) #> future 1.33.0 2023-07-01 [1] CRAN (R 4.3.0) #> globals 0.16.2 2022-11-21 [1] CRAN (R 4.3.0) #> glue 1.6.2 2022-02-24 [1] CRAN (R 4.3.0) #> greta * 0.4.3.9000 2023-11-17 [1] local #> hms 1.1.3 2023-03-21 [1] CRAN (R 4.3.0) #> htmltools 0.5.7 2023-11-03 [1] CRAN (R 4.3.1) #> jsonlite 1.8.7 2023-06-29 [1] CRAN (R 4.3.0) #> knitr 1.45 2023-10-30 [1] CRAN (R 4.3.1) #> lattice 0.21-9 2023-10-01 [1] CRAN (R 4.3.2) #> lifecycle 1.0.3 2022-10-07 [1] CRAN (R 4.3.0) #> listenv 0.9.0 2022-12-16 [1] CRAN (R 4.3.0) #> magrittr 2.0.3 2022-03-30 [1] CRAN (R 4.3.0) #> Matrix 1.6-1.1 2023-09-18 [1] CRAN (R 4.3.2) #> parallelly 1.36.0 2023-05-26 [1] CRAN (R 4.3.0) #> pkgconfig 2.0.3 2019-09-22 [1] CRAN (R 4.3.0) #> png 0.1-8 2022-11-29 [1] CRAN (R 4.3.0) #> prettyunits 1.2.0 2023-09-24 [1] CRAN (R 4.3.1) #> processx 3.8.2 2023-06-30 [1] CRAN (R 4.3.0) #> progress 1.2.2 2019-05-16 [1] CRAN (R 4.3.0) #> ps 1.7.5 2023-04-18 [1] CRAN (R 4.3.0) #> purrr 1.0.2 2023-08-10 [1] CRAN (R 4.3.0) #> R.cache 0.16.0 2022-07-21 [1] CRAN (R 4.3.0) #> R.methodsS3 1.8.2 2022-06-13 [1] CRAN (R 4.3.0) #> R.oo 1.25.0 2022-06-12 [1] CRAN (R 4.3.0) #> R.utils 2.12.2 2022-11-11 [1] CRAN (R 4.3.0) #> R6 2.5.1 2021-08-19 [1] CRAN (R 4.3.0) #> Rcpp 1.0.11 2023-07-06 [1] CRAN (R 4.3.0) #> reprex 2.0.2 2022-08-17 [1] CRAN (R 4.3.0) #> reticulate 1.34.0 2023-10-12 [1] CRAN (R 4.3.1) #> rlang 1.1.1 2023-04-28 [1] CRAN (R 4.3.0) #> rmarkdown 2.25 2023-09-18 [1] CRAN (R 4.3.1) #> rstudioapi 0.15.0 2023-07-07 [1] CRAN (R 4.3.0) #> sessioninfo 1.2.2 2021-12-06 [1] CRAN (R 4.3.0) #> styler 1.9.1 2023-03-04 [1] CRAN (R 4.3.0) #> tensorflow 2.14.0 2023-09-28 [1] CRAN (R 4.3.1) #> tfautograph 0.3.2 2021-09-17 [1] CRAN (R 4.3.0) #> tfruns 1.5.1 2022-09-05 [1] CRAN (R 4.3.0) #> vctrs 0.6.4 2023-10-12 [1] CRAN (R 4.3.1) #> whisker 0.4.1 2022-12-05 [1] CRAN (R 4.3.0) #> withr 2.5.2 2023-10-30 [1] CRAN (R 4.3.1) #> xfun 0.41 2023-11-01 [1] CRAN (R 4.3.1) #> yaml 2.3.7 2023-01-23 [1] CRAN (R 4.3.0) #> #> [1] /Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library #> #> ─ Python configuration ─────────────────────────────────────────────────────── #> python: /Users/nick/Library/r-miniconda-arm64/envs/greta-env-tf2/bin/python #> libpython: /Users/nick/Library/r-miniconda-arm64/envs/greta-env-tf2/lib/libpython3.8.dylib #> pythonhome: /Users/nick/Library/r-miniconda-arm64/envs/greta-env-tf2:/Users/nick/Library/r-miniconda-arm64/envs/greta-env-tf2 #> version: 3.8.15 | packaged by conda-forge | (default, Nov 22 2022, 08:49:06) [Clang 14.0.6 ] #> numpy: /Users/nick/Library/r-miniconda-arm64/envs/greta-env-tf2/lib/python3.8/site-packages/numpy #> numpy_version: 1.23.2 #> tensorflow: /Users/nick/Library/r-miniconda-arm64/envs/greta-env-tf2/lib/python3.8/site-packages/tensorflow #> #> NOTE: Python version was forced by use_python() function #> #> ────────────────────────────────────────────────────────────────────────────── ```

[njtierney]

Here is the behaviour with some simpler code using chol on a non M1 mac using TF1 greta

library(greta)
#> 
#> Attaching package: 'greta'
#> The following objects are masked from 'package:stats':
#> 
#>     binomial, cov2cor, poisson
#> The following objects are masked from 'package:base':
#> 
#>     %*%, apply, backsolve, beta, chol2inv, colMeans, colSums, diag,
#>     eigen, forwardsolve, gamma, identity, rowMeans, rowSums, sweep,
#>     tapply
greta_sitrep()
#> ℹ checking if python available
#> ✔ python (version 3.7) available
#> 
#> ℹ checking if TensorFlow available
#> ✔ TensorFlow (version 1.14.0) available
#> 
#> ℹ checking if TensorFlow Probability available
#> ✔ TensorFlow Probability (version 0.7.0) available
#> 
#> ℹ checking if greta conda environment available
#> ✔ greta conda environment available
#> 
#> ℹ Initialising python and checking dependencies, this may take a moment.
#> ✔ Initialising python and checking dependencies ... done!
#> 
#> ℹ greta is ready to use!
x <- wishart(df = 4, Sigma = diag(3))
chol_x <- chol(x)
# debugonce(calculate)
calc_chol <- calculate(x, chol_x, nsim = 1)
#> Error in py_call_impl(callable, dots$args, dots$keywords): tensorflow.python.framework.errors_impl.InvalidArgumentError: You must feed a value for placeholder tensor 'Placeholder_1' with dtype double and shape [1,3,3]
#>   [[node Placeholder_1 (defined at /ops/array_ops.py:2143) ]]
#> 
#> Original stack trace for 'Placeholder_1':
#>   File "/ops/array_ops.py", line 2143, in placeholder
#>     return gen_array_ops.placeholder(dtype=dtype, shape=shape, name=name)
#>   File "/ops/gen_array_ops.py", line 6262, in placeholder
#>     "Placeholder", dtype=dtype, shape=shape, name=name)
#>   File "/framework/op_def_library.py", line 788, in _apply_op_helper
#>     op_def=op_def)
#>   File "/util/deprecation.py", line 507, in new_func
#>     return func(*args, **kwargs)
#>   File "/framework/ops.py", line 3616, in create_op
#>     op_def=op_def)
#>   File "/framework/ops.py", line 2005, in __init__
#>     self._traceback = tf_stack.extract_stack()
calc_chol
#> Error in eval(expr, envir, enclos): object 'calc_chol' not found

^{Created on 2023-11-17 by the reprex package (v2.0.1)}

Session info

``` r sessioninfo::session_info() #> ─ Session info ─────────────────────────────────────────────────────────────── #> setting value #> version R version 4.2.1 (2022-06-23) #> os macOS Big Sur ... 10.16 #> system x86_64, darwin17.0 #> ui X11 #> language (EN) #> collate en_AU.UTF-8 #> ctype en_AU.UTF-8 #> tz Australia/Hobart #> date 2023-11-17 #> pandoc 2.19.2 @ /usr/local/bin/ (via rmarkdown) #> #> ─ Packages ─────────────────────────────────────────────────────────────────── #> package * version date (UTC) lib source #> base64enc 0.1-3 2015-07-28 [1] CRAN (R 4.2.0) #> callr 3.7.1 2022-07-13 [1] CRAN (R 4.2.0) #> cli 3.3.0 2022-04-25 [1] CRAN (R 4.2.0) #> coda 0.19-4 2020-09-30 [1] CRAN (R 4.2.0) #> codetools 0.2-18 2020-11-04 [1] CRAN (R 4.2.1) #> crayon 1.5.1 2022-03-26 [1] CRAN (R 4.2.0) #> digest 0.6.29 2021-12-01 [1] CRAN (R 4.2.0) #> ellipsis 0.3.2 2021-04-29 [1] CRAN (R 4.2.0) #> evaluate 0.15 2022-02-18 [1] CRAN (R 4.2.0) #> fansi 1.0.3 2022-03-24 [1] CRAN (R 4.2.0) #> fastmap 1.1.0 2021-01-25 [1] CRAN (R 4.2.0) #> fs 1.5.2 2021-12-08 [1] CRAN (R 4.2.0) #> future 1.26.1 2022-05-27 [1] CRAN (R 4.2.0) #> globals 0.15.1 2022-06-24 [1] CRAN (R 4.2.0) #> glue 1.6.2 2022-02-24 [1] CRAN (R 4.2.0) #> greta * 0.4.3.9000 2022-10-25 [1] local #> here 1.0.1 2020-12-13 [1] CRAN (R 4.2.0) #> highr 0.9 2021-04-16 [1] CRAN (R 4.2.0) #> hms 1.1.1 2021-09-26 [1] CRAN (R 4.2.0) #> htmltools 0.5.3 2022-07-18 [1] CRAN (R 4.2.0) #> jsonlite 1.8.0 2022-02-22 [1] CRAN (R 4.2.0) #> knitr 1.39 2022-04-26 [1] CRAN (R 4.2.0) #> lattice 0.20-45 2021-09-22 [1] CRAN (R 4.2.1) #> lifecycle 1.0.1 2021-09-24 [1] CRAN (R 4.2.0) #> listenv 0.8.0 2019-12-05 [1] CRAN (R 4.2.0) #> magrittr 2.0.3 2022-03-30 [1] CRAN (R 4.2.0) #> Matrix 1.4-1 2022-03-23 [1] CRAN (R 4.2.1) #> parallelly 1.32.0 2022-06-07 [1] CRAN (R 4.2.0) #> pillar 1.8.0 2022-07-18 [1] CRAN (R 4.2.0) #> pkgconfig 2.0.3 2019-09-22 [1] CRAN (R 4.2.0) #> png 0.1-7 2013-12-03 [1] CRAN (R 4.2.0) #> prettyunits 1.1.1 2020-01-24 [1] CRAN (R 4.2.0) #> processx 3.7.0 2022-07-07 [1] CRAN (R 4.2.0) #> progress 1.2.2 2019-05-16 [1] CRAN (R 4.2.0) #> ps 1.7.1 2022-06-18 [1] CRAN (R 4.2.0) #> purrr 0.3.4 2020-04-17 [1] CRAN (R 4.2.0) #> R.cache 0.16.0 2022-07-21 [1] CRAN (R 4.2.0) #> R.methodsS3 1.8.2 2022-06-13 [1] CRAN (R 4.2.0) #> R.oo 1.25.0 2022-06-12 [1] CRAN (R 4.2.0) #> R.utils 2.12.0 2022-06-28 [1] CRAN (R 4.2.0) #> R6 2.5.1 2021-08-19 [1] CRAN (R 4.2.0) #> Rcpp 1.0.9 2022-07-08 [1] CRAN (R 4.2.0) #> reprex 2.0.1 2021-08-05 [1] CRAN (R 4.2.0) #> reticulate 1.25 2022-05-11 [1] CRAN (R 4.2.0) #> rlang 1.0.4 2022-07-12 [1] CRAN (R 4.2.0) #> rmarkdown 2.14 2022-04-25 [1] CRAN (R 4.2.0) #> rprojroot 2.0.3 2022-04-02 [1] CRAN (R 4.2.0) #> rstudioapi 0.13 2020-11-12 [1] CRAN (R 4.2.0) #> sessioninfo 1.2.2 2021-12-06 [1] CRAN (R 4.2.0) #> stringi 1.7.8 2022-07-11 [1] CRAN (R 4.2.0) #> stringr 1.4.0 2019-02-10 [1] CRAN (R 4.2.0) #> styler 1.7.0 2022-03-13 [1] CRAN (R 4.2.0) #> tensorflow 2.9.0 2022-05-21 [1] CRAN (R 4.2.0) #> tfruns 1.5.0 2021-02-26 [1] CRAN (R 4.2.0) #> tibble 3.1.7 2022-05-03 [1] CRAN (R 4.2.0) #> utf8 1.2.2 2021-07-24 [1] CRAN (R 4.2.0) #> vctrs 0.4.1 2022-04-13 [1] CRAN (R 4.2.0) #> whisker 0.4 2019-08-28 [1] CRAN (R 4.2.0) #> withr 2.5.0 2022-03-03 [1] CRAN (R 4.2.0) #> xfun 0.31 2022-05-10 [1] CRAN (R 4.2.0) #> yaml 2.3.5 2022-02-21 [1] CRAN (R 4.2.0) #> #> [1] /Library/Frameworks/R.framework/Versions/4.2/Resources/library #> #> ─ Python configuration ─────────────────────────────────────────────────────── #> python: /Users/njtierney/Library/r-miniconda/envs/greta-env/bin/python #> libpython: /Users/njtierney/Library/r-miniconda/envs/greta-env/lib/libpython3.7m.dylib #> pythonhome: /Users/njtierney/Library/r-miniconda/envs/greta-env:/Users/njtierney/Library/r-miniconda/envs/greta-env #> version: 3.7.12 | packaged by conda-forge | (default, Oct 26 2021, 05:59:23) [Clang 11.1.0 ] #> numpy: /Users/njtierney/Library/r-miniconda/envs/greta-env/lib/python3.7/site-packages/numpy #> numpy_version: 1.16.4 #> tensorflow: /Users/njtierney/Library/r-miniconda/envs/greta-env/lib/python3.7/site-packages/tensorflow #> #> NOTE: Python version was forced by use_python function #> #> ────────────────────────────────────────────────────────────────────────────── ```