Error in spatInSituPlotPoints

[jnmark]

Hi, thanks for adding in capabilities to analyze sub-cellular technologies. I am trying to follow the Vizgen tutorial on the website. Everything runs smoothly until the very last part which is plotting the cells and transcripts on a subset of the data using spatInSituPlotPoints. I get an error: Error in if (spat_loc_name %in% potential_names) { : argument is of length zero

I made sure to check that spatial_locs and spatial_info are not empty and also tried providing spat_unit as cell and spat_loc as raw as per other tutorials. What am I doing wrong? Could you also please provide some more details on wha should be specified in spat_unit and spat_loc_name?

This is the command:

spatInSituPlotPoints(vizgen_subset, feats = list('rna' = c("Oxgr1", "Htr1a", "Gjc3", "Axl", 'Gfap', "Olig1", "Epha7")), polygon_feat_type = 'z0', use_overlap = F, point_size = 0.2, show_polygon = TRUE, polygon_color = 'white', return_plot = FALSE, save_plot = TRUE, show_plot = FALSE, save_param = list(show_saved_plot = TRUE))

I am running Giotto_2.0.0.998 in R 4.1.2.

[jnmark]

Hello, any update on this front?

[RubD]

Hi @jnmark , we're currently looking into this. Thanks for your patience.

[MomenehForoutan]

Hi team, Thanks for looking into this Error! I am getting different errors using the same function when following the CosMx lung 12 manual (although I am working on the lung 13 sample). These errors seem to be related to external pointers to store their underlying data in a C++ class? See a couple of examples below:

spatInSituPlotPoints(fov_join,
         show_polygon = TRUE,
         polygon_feat_type = 'cell',
         polygon_color = 'white',
         polygon_line_size = 0.1,
         polygon_fill = 'cell_types',
         polygon_fill_as_factor = T ,
         polygon_fill_code = cols)

Warning in spatInSituPlotPoints(fov_join, show_polygon = TRUE, polygon_feat_type = "cell",  :
You need to select features (feats) and modify feature types (feat_type) if you want to show individual features (e.g. transcripts)  
 [1] "ok 1"
 [1] "ok 2"
 [1] "ok 3"
 Error in .External(list(name = "CppMethod__invoke_notvoid", address = <pointer: (nil)>,  : 
   NULL value passed as symbol address

spatInSituPlotPoints(
   fov_join,
   show_polygon = TRUE,
   point_size = 0.5,
   polygon_color = 'white',
   polygon_line_size = 0.1,
   polygon_fill = 'total_expr',
   polygon_fill_as_factor = F,
   coord_fix_ratio = T,
  save_param = list(
    save_name =  'QC_spatInSituPlotPoints_TotalExpr',
    base_width = 7,
    base_height = 5.5,
    save_format = 'png'
   )
 )
Error in x@ptr$get_geometry() : external pointer is not valid

Here is my session Info:

sessionInfo()
R version 4.2.0 (2022-04-22)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Red Hat Enterprise Linux 8.4 (Ootpa)

Matrix products: default
BLAS/LAPACK: /usr/lib64/libopenblasp-r0.3.12.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8       
 [4] LC_COLLATE=en_US.UTF-8     LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                  LC_ADDRESS=C              
[10] LC_TELEPHONE=C             LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] Giotto_2.0.0.998        RColorBrewer_1.1-3      qs_0.25.4              
 [4] dplyr_1.0.10            patchwork_1.1.2         viridis_0.6.2          
 [7] viridisLite_0.4.1       ggplot2_3.3.6           sp_1.5-0               
[10] SeuratObject_4.1.2.9003 Seurat_4.2.0.9001      

loaded via a namespace (and not attached):
  [1] Rtsne_0.16            colorspace_2.0-3      deldir_1.0-6          ellipsis_0.3.2       
  [5] ggridges_0.5.4        spatstat.data_3.0-0   farver_2.1.1          leiden_0.4.3         
  [9] listenv_0.8.0         remotes_2.4.2         ggrepel_0.9.1         fansi_1.0.3          
 [13] codetools_0.2-18      splines_4.2.0         knitr_1.40            polyclip_1.10-4      
 [17] jsonlite_1.8.3        ica_1.0-3             cluster_2.1.3         png_0.1-7            
 [21] rgeos_0.5-9           uwot_0.1.14           shiny_1.7.2           sctransform_0.3.5    
 [25] spatstat.sparse_3.0-0 compiler_4.2.0        httr_1.4.4            Matrix_1.5-1         
 [29] fastmap_1.1.0         lazyeval_0.2.2        limma_3.52.4          cli_3.4.1            
 [33] later_1.3.0           htmltools_0.5.3       tools_4.2.0           igraph_1.3.5         
 [37] gtable_0.3.1          glue_1.6.2            RANN_2.6.1            reshape2_1.4.4       
 [41] rappdirs_0.3.3        Rcpp_1.0.9            scattermore_0.8       vctrs_0.5.0          
 [45] nlme_3.1-157          progressr_0.11.0      lmtest_0.9-40         spatstat.random_2.2-0
 [49] xfun_0.33             stringr_1.4.1         globals_0.16.1        mime_0.12            
 [53] miniUI_0.1.1.1        lifecycle_1.0.3       irlba_2.3.5.1         goftest_1.2-3        
 [57] terra_1.6-17          future_1.28.0         edgeR_3.38.4          MASS_7.3-56          
 [61] zoo_1.8-11            scales_1.2.1          spatstat.core_2.4-4   promises_1.2.0.1     
 [65] spatstat.utils_3.0-1  parallel_4.2.0        yaml_2.3.6            reticulate_1.26      
 [69] pbapply_1.5-0         gridExtra_2.3         rpart_4.1.16          stringi_1.7.8        
 [73] rlang_1.0.6           pkgconfig_2.0.3       matrixStats_0.62.0    evaluate_0.16        
 [77] lattice_0.20-45       ROCR_1.0-11           purrr_0.3.5           tensor_1.5           
 [81] labeling_0.4.2        htmlwidgets_1.5.4     cowplot_1.1.1         tidyselect_1.2.0     
 [85] parallelly_1.32.1     RcppAnnoy_0.0.19      plyr_1.8.7            magrittr_2.0.3       
 [89] R6_2.5.1              generics_0.1.3        pillar_1.8.1          withr_2.5.0          
 [93] mgcv_1.8-40           fitdistrplus_1.1-8    survival_3.3-1        abind_1.4-5          
 [97] tibble_3.1.8          future.apply_1.9.1    KernSmooth_2.23-20    utf8_1.2.2           
[101] spatstat.geom_2.4-0   RApiSerialize_0.1.2   plotly_4.10.0         rmarkdown_2.17       
[105] locfit_1.5-9.6        grid_4.2.0            data.table_1.14.4     digest_0.6.30        
[109] xtable_1.8-4          tidyr_1.2.1           httpuv_1.6.6          RcppParallel_5.1.5   
[113] munsell_0.5.0         stringfish_0.15.7 

Many thanks for your help in advance! Cheers.

[MomenehForoutan]

Hi there, Just sharing some more details on the "pointer" Error that I mentioned above, so that you know it is actually reproducible. We have realised that the Error occurs after we save and read the data back in R. This happens for both sample lung 13 as well as the exact same data you are using in the manual here (lung 12, fov2/3/4).

It seems like this may be linked to a known issue in other packages that rely on external pointers as detailed in this post. Apparently objects with external pointers can only be used in the R session where they are created. Exporting and reloading can otherwise lose these external pointers. Do you have any suggestions for how we might be able to periodically save to disk in Giotto? Given the size of the datasets we work with it seems fragile to depend on a persistent R session throughout a workflow.

sampleName <–"lung12"

library(future)
options(future.globals.maxSize = 4000*1024^2)
options(future.globals.onReference = "error")

f <- future(spatInSituPlotPoints(
  gobject_cosmx,
  show_polygon = TRUE,
  point_size = 0.1,
  polygon_color = 'white',
  polygon_line_size = 0.1,
  polygon_fill = 'total_expr',
  polygon_fill_as_factor = F,
  ### feat_type = "rna",
  ### feats = "NKG7",
  coord_fix_ratio = T,
  save_param = list(
    save_name = paste0(sampleName, '_QC_spatInSituPlotPoints_TotalExpr'),
    base_width = 7,
    base_height = 5.5,
    save_format = 'png'
  )
))

Error: Detected a non-exportable reference (‘externalptr’) in one of the globals (‘gobject_cosmx’ of class ‘giotto’) used in the future expression
Timing stopped at: 0.013 0.001 0.014

Thanks for the great package, and looking forward to hearing your thoughts on this issue. Many thanks!

P.S. it seems that the manual for the lung 12 needs to be updated for some parts; happy to email through the details or open a new issue detailing them if you prefer that?

[RubD]

Hi @MomenehForoutan I added two functions yesterday called saveGiotto and loadGiotto that should allow you to save a giotto object to a specific folder and load it back in. In theory it should also be able to share this folder with others and use the same object. I just finished the first version yesterday so it might still be a bit buggy.

Any feedback on the tutorial or usage of these new functions is welcome!

[RubD]

@jnmark I'm not able to replicate this issue. Can you share some code with a public dataset that would re-create your issue?

[MomenehForoutan]

Thank you so much @RubD for the quick reply! The two functions you have defined did indeed solve the issue! Looking at the underlying code, I see that you have used saveRDS and readRDS for saving and reading the Giotto object in R. I was wondering if you would like to consider the qs package for this. I have been working with large spatial data and the qs package is faster compared to RDS for saving and loading data. Just something to consider if you are interested. Here is an example of how to use it.

## Saving the data
qs::qsave(
  fov_join,
  file = "CosMx_L12_Giotto_processed.qs",
  preset = "high",
  algorithm = "zstd",
  compress_level = 4L,
  shuffle_control = 15L,
  check_hash = TRUE,
  nthreads = 15
)

## Reading the data
fov_join <- qs::qread( "CosMx_L12_Giotto_processed.qs")

Thanks again! I will open a new issue for my feedback on the CosMx manual :-) Cheers.

[MomenehForoutan]

Just a short comment to say that while the two functions saveGiotto() and loadGiotto() solved the issue with spatInSituPlotPoints() on the saved and loaded data, I still get error when using other functions, like subsetGiotto() and subsetGiottoLoc() after loading the saved data into R. I have opened a new issue for this here. Thanks!

[RubD]

Thank you so much @RubD for the quick reply! The two functions you have defined did indeed solve the issue! Looking at the underlying code, I see that you have used saveRDS and readRDS for saving and reading the Giotto object in R. I was wondering if you would like to consider the qs package for this. I have been working with large spatial data and the qs package is faster compared to RDS for saving and loading data. Just something to consider if you are interested. Here is an example of how to use it.

## Saving the data
qs::qsave(
  fov_join,
  file = "CosMx_L12_Giotto_processed.qs",
  preset = "high",
  algorithm = "zstd",
  compress_level = 4L,
  shuffle_control = 15L,
  check_hash = TRUE,
  nthreads = 15
)

## Reading the data
fov_join <- qs::qread( "CosMx_L12_Giotto_processed.qs")

Thanks again! I will open a new issue for my feedback on the CosMx manual :-) Cheers.

Thanks again for the excellent suggestion @MomenehForoutan. We are always looking for ways to improve the speed and memory usage of Giotto and the qs and fst packages have been on our radar. I've implemented the option to use qs instead of RDS. It seems to work after a quick test, but let us know if you run into issues.

[MomenehForoutan]

Thanks for updating this so quickly! I have just tried the CosMx manual again after package update, and it works with no issue. Really appreciate it! Cheers.