CosMx mini dataset cannot reconnect images

[rbutleriii]

Describe the Error

Looking through the GiottoData package, I can locate images that look like the ones for the mini cosmx dataset in loadGiottoMini. But when I try to reconnectGiottoImage so I can plot the images, I get a strange error

To Reproduce

library(Giotto)
library(GiottoData)

x = loadGiottoMini("cosmx")

# Reconnect some images ----------------------------------------------------
img_path = c(
  "~R/x86_64-pc-linux-gnu-library/4.2/GiottoData/Mini_datasets/CosMx/Raw/CellLabels/CellLabels_F002.tif",
  "~R/x86_64-pc-linux-gnu-library/4.2/GiottoData/Mini_datasets/CosMx/Raw/CellLabels/CellLabels_F002.tif"
)
lrg_path = c(
  "~/R/x86_64-pc-linux-gnu-library/4.2/GiottoData/Mini_datasets/CosMx/Raw/CellComposite/CellComposite_F002.jpg",
  "~/R/x86_64-pc-linux-gnu-library/4.2/GiottoData/Mini_datasets/CosMx/Raw/CellComposite/CellComposite_F003.jpg"
)
x <- reconnectGiottoImage(x,
  image_name = names(x@images),
  image_path = img_path,  
  largeImage_name = names(x@largeImages),
  largeImage_path = lrg_path
)
Attempting automatic reconnection...

image(s) discovered...
--> fov002-image : filepath NOT found
--> fov003-image : filepath NOT found

largeImage(s) discovered...
--> fov002-composite : filepath foundbut file is missing
Error in img_path[[image_type]][[image_i]] : subscript out of bounds

If I try to manually specify them I get an issue still with plotting:

x@images$`fov002-image`@file_path = img_path[1]
x@images$`fov003-image`@file_path = img_path[2]
x@largeImages$`fov002-composite`@file_path = lrg_path[1]
x@largeImages$`fov003-composite`@file_path = lrg_path[2]

spatInSituPlotPoints(x,
  feats = list("COL1A1", "IGHG1", "KITLG", "ITK", "TPSB2"),
  point_size = 1,
  polygon_line_size = 0.1,
  show_polygon = TRUE,
  polygon_color = 'white',
  show_image = TRUE,
  largeImage_name = names(x@largeImages),
  show_legend = TRUE
)
select image done
Error in rgb(t(x[, , 1L]), t(x[, , 2L]), t(x[, , 3L]), maxColorValue = max) :
  color intensity NA, not in [0,1]

Expected behavior

Ideally the mini object would load without needing reconnection, but per #541 that is perhaps a longer solve. In the interim, am I pointing at the right images in the GiottoData object?

System Information

> sessionInfo()
R version 4.2.2 (2022-10-31)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: CentOS Linux 7 (Core)

Matrix products: default
BLAS/LAPACK: /home/rrbutler/.local/share/r-miniconda/envs/giotto_env/lib/libopenblasp-r0.3.25.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C
 [9] LC_ADDRESS=C               LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

other attached packages:
[1] BiocSingular_1.14.0   shiny_1.8.0           microbenchmark_1.4.10
[4] data.table_1.14.10    GiottoData_0.2.6.2    GiottoUtils_0.1.3
[7] Giotto_4.0.2          GiottoClass_0.1.3

loaded via a namespace (and not attached):
 [1] MatrixGenerics_1.10.0 Biobase_2.58.0        sass_0.4.8
 [4] jsonlite_1.8.8        gtools_3.9.5          bslib_0.6.1
 [7] BiocManager_1.30.22   stats4_4.2.2          GiottoVisuals_0.1.2
[10] globals_0.16.2        backports_1.4.1       pillar_1.9.0
[13] lattice_0.22-5        glue_1.7.0            beachmat_2.14.2
[16] reticulate_1.34.0     digest_0.6.34         RColorBrewer_1.1-3
[19] promises_1.2.1        XVector_0.38.0        checkmate_2.3.1
[22] colorspace_2.1-0      htmltools_0.5.7       httpuv_1.6.13
[25] Matrix_1.6-4          pkgconfig_2.0.3       listenv_0.9.0
[28] zlibbioc_1.44.0       xtable_1.8-4          scales_1.3.0
[31] ScaledMatrix_1.6.0    terra_1.7-55          later_1.3.2
[34] BiocParallel_1.32.6   tibble_3.2.1          generics_0.1.3
[37] farver_2.1.1          dbscan_1.1-12         IRanges_2.32.0
[40] ggplot2_3.4.4         ellipsis_0.3.2        cachem_1.0.8
[43] withr_2.5.2           BiocGenerics_0.44.0   cli_3.6.2
[46] magrittr_2.0.3        mime_0.12             memoise_2.0.1
[49] future_1.33.0         fansi_1.0.6           parallelly_1.36.0
[52] tools_4.2.2           lifecycle_1.0.4       matrixStats_1.2.0
[55] S4Vectors_0.36.2      munsell_0.5.0         DelayedArray_0.24.0
[58] irlba_2.3.5.1         jquerylib_0.1.4       compiler_4.2.2
[61] rsvd_1.0.5            rlang_1.1.3           grid_4.2.2
[64] rappdirs_0.3.3        igraph_1.6.0          miniUI_0.1.1.1
[67] tcltk_4.2.2           labeling_0.4.3        gtable_0.3.4
[70] codetools_0.2-19      R6_2.5.1              dplyr_1.1.4
[73] fastmap_1.1.1         future.apply_1.11.0   utf8_1.2.4
[76] colorRamp2_0.1.0      parallel_4.2.2        Rcpp_1.0.12
[79] vctrs_0.6.5           png_0.1-8             tidyselect_1.2.0```

[jiajic]

Thanks for reporting. The plotting issue shown here should be fixed in GiottoVisuals v0.1.4 #60 I am now getting this:

library(Giotto)
x <- GiottoData::loadGiottoMini("cosmx")
spatInSituPlotPoints(x,
                     feats = list("COL1A1", "IGHG1", "KITLG", "ITK", "TPSB2"),
                     point_size = 1,
                     polygon_line_size = 0.1,
                     show_polygon = TRUE,
                     polygon_color = 'white',
                     show_image = TRUE,
                     largeImage_name = names(x@largeImages),
                     show_legend = TRUE
)

Some additional details: The mini objects do attempt automatic image reconnections on load. It looks like there were some issues with the image saving during the creation of the mini object that resulted in the high values being treated as NA values. The object in GiottoData will have to be updated to rectify this.

y <- array(x@largeImages$`fov002-composite`@raster_object)
sum(is.na(test))
[1] 4332

That was causing this error message when converting to an annotation raster for ggplot to use

Error in rgb(t(x[, , 1L]), t(x[, , 2L]), t(x[, , 3L]), maxColorValue = max) :
  color intensity NA, not in [0,1]

Giotto is now treating NA values in the images as 0 when using the plotting functions.

[jiajic]

For the reconnection issue with the giottoImage objects, we are working on a solution. I would need to take a closer look, but my guess is that the problem is because of issues with our older giottoImage class that we are in the process of deprecating in favor of the newer giottoLargeImage. Before we can fully do that, we need to remove them in favor of just giottoLargeImage creation from the convenience functions for Visium (#850) and CosMx (in progress).

With the CosMx convenience function, we convert the composite giottoLargeImage to giottoImage, intending it as a way of caching it to be faster to load, but that means they do not actually exist on disk. This behavior will also be removed pending the convenience function update mentioned above. The workaround for the moment is to use the giottoLargeImage versions of the composite images for visualizations.

If a giottoLargeImage needs to be made, then that can be done by checking the giottoImage for spatial bounds information

a <- x@images$`fov002-image`
a

relevant portion of the return is below:

Boundaries are: 
 Image x-axis max boundary:  5966.444 
 Image x-axis min boundary:  5294.444 
 Image y-axis max boundary:  -156585.3 
 Image y-axis min boundary:  -157233.3 

a giottoLargeImage can then be created and the spatial extent can be set based on these values before being set back into the giotto object

library(GiottoClass)
im_path <- system.file("Mini_datasets/CosMx/Raw/CellComposite/CellComposite_F002.jpg", package = "GiottoData")
b <- createGiottoLargeImage(im_path)
ext(b) <- c(5294.444, 5966.444, -157233.3,-156585.3) # xmin, xmax, ymin, ymax
plot(b)

[rbutleriii]

Yep, updating to GiottoVisuals 0.1.4 and updating terra to 1.7-65 fixed the plotting error for me. Can probably close this or link it to the reconnection issue #