`read_opus()` fails with opusreader2_v0.1.0 on Linux, but main v0.5.0.900 OK

[davidbenncsiro]

If I install opusreader2_v0.1.0 on Linux with R 3.6.3 or Windows 10 with R 4.2.1:

devtools::install_github("spectral-cockpit/opusreader2@opusreader2_v0.1.0")

reading this file (in zip) works fine: example.zip

But with opusreader2_v0.1.0 on Linux with R 4.2.1 or 4.3.1 in GitHub Actions, the file load has errors, e.g.

result$acquisition$parameters$PLF$parameter_value

returns NULL instead of AB and data values (wavenumbers and intensities) are either 0 or NULL.

However, if I install spectral-cockpit/opusreader2 (i.e. HEAD of master branch), everything's fine.

Have you seen this particular problem?

[philipp-baumann]

If I install opusreader2_v0.1.0 on Linux with R 3.6.3 or Windows 10 with R 4.2.1:

devtools::install_github("spectral-cockpit/opusreader2@opusreader2_v0.1.0")

reading this file (in zip) works fine: example.zip

But with opusreader2_v0.1.0 on Linux with R 4.2.1 or 4.3.1 in GitHub Actions, the file load has errors, e.g.

result$acquisition$parameters$PLF$parameter_value

returns NULL instead of AB and data values (wavenumbers and intensities) are either 0 or NULL.

However, if I install spectral-cockpit/opusreader2 (i.e. HEAD of master branch), everything's fine.

Have you seen this particular problem?

Hi @davidbenncsiro , I guess since we are currently at v0.9.0.5000 on main, it is really the version difference. I can tag another release, but I'll test first to confirm.

[philipp-baumann]

philipp@philipp-private:~/git/spectral-cockpit/opusreader2_tests$ radian
R version 4.1.3 (2022-03-10) -- "One Push-Up"
Platform: x86_64-pc-linux-gnu (64-bit)

r$> library("opusreader2")

r$> file <- "example.0"

r$> data_list <- read_opus(dsn = file)

r$> data_list$acquisition$parameters$PLF$parameter_value
NULL

r$> data_list[[1]]$acquisition$parameters$PLF$parameter_value
[1] "AB"

r$> sessioninfo::session_info()
─ Session info ─────────────────────────────────────────────────────────────────────────────────────
 setting  value
 version  R version 4.1.3 (2022-03-10)
 os       Ubuntu 22.04.3 LTS
 system   x86_64, linux-gnu
 ui       X11
 language
 collate  en_US.UTF-8
 ctype    en_US.UTF-8
 tz       Europe/Zurich
 date     2023-09-12
 pandoc   2.9.2.1 @ /usr/bin/pandoc

─ Packages ─────────────────────────────────────────────────────────────────────────────────────────
 package     * version    date (UTC) lib source
 cli           3.6.1      2023-03-23 [1] CRAN (R 4.1.3)
 opusreader2 * 0.5.0.9000 2023-09-12 [1] Github (spectral-cockpit/opusreader2@337a2d4)
 sessioninfo   1.2.2      2021-12-06 [1] CRAN (R 4.1.2)

 [1] /home/philipp/R/x86_64-pc-linux-gnu-library/4.1
 [2] /opt/R/4.1.3/lib/R/library

[philipp-baumann]

@davidbenncsiro my guess is it is the fixes done in v0.5.0, see also https://github.com/spectral-cockpit/opusreader2/blob/main/NEWS.md

[philipp-baumann]

I also tagged a new release, so you can do like this:

devtools::install_github("spectral-cockpit/opusreader2@opusreader2_v0.5.0.9000")

[philipp-baumann]

What I don't get, however, why it should make a difference when you read in Win10 vs. Linux using the presumably identical version/hash of opusreader2.

[philipp-baumann]

Currently, I don't have access to a Win environment (made an Azure subscription, but ran out of my trial time). @davidbenncsiro what does you sessionInfo() return on Win vs. on Linux?

[philipp-baumann]

Guess you are using GH actions as sort of ETL pipeline to read spectra?

[davidbenncsiro]

Guess you are using GH actions as sort of ETL pipeline to read spectra?

Actually, simply for CI, i.e. running unit tests on commit.

[davidbenncsiro]

I also tagged a new release, so you can do like this:

devtools::install_github("spectral-cockpit/opusreader2@opusreader2_v0.5.0.9000")

I changed the GH Actions workflow YAML to install this version and also installed it in Windows and the other Linux env I mentioned and it works in all three now.

Thanks! As to why this is different from master, a good question.

[davidbenncsiro]

Currently, I don't have access to a Win environment (made an Azure subscription, but ran out of my trial time). @davidbenncsiro what does you sessionInfo() return on Win vs. on Linux?

Here's two envs:

> sessionInfo()
R version 3.6.3 (2020-02-29)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.2 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.7.1
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.7.1

locale:
 [1] LC_CTYPE=en_AU       LC_NUMERIC=C         LC_TIME=en_AU       
 [4] LC_COLLATE=en_AU     LC_MONETARY=en_AU    LC_MESSAGES=en_AU   
 [7] LC_PAPER=en_AU       LC_NAME=C            LC_ADDRESS=C        
[10] LC_TELEPHONE=C       LC_MEASUREMENT=en_AU LC_IDENTIFICATION=C 

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] soilspec.format_0.0.0.9001 testthat_3.0.4            

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.7             lattice_0.20-41        prettyunits_1.1.1     
 [4] png_0.1-7              ps_1.6.0               hyperSpec_0.100.0     
 [7] rprojroot_2.0.2        digest_0.6.28          foreach_1.5.1         
[10] utf8_1.2.2             R6_2.5.1               evaluate_0.14         
[13] ggplot2_3.3.5          pillar_1.6.3           rlang_0.4.11          
[16] lazyeval_0.2.2         curl_4.3.2             rstudioapi_0.13       
[19] callr_3.7.0            rmarkdown_2.18         mathjaxr_1.0-1        
[22] desc_1.4.0             devtools_2.4.2         opusreader2_0.5.0.9000
[25] stringr_1.4.0          munsell_0.5.0          compiler_3.6.3        
[28] xfun_0.35              pkgconfig_2.0.3        pkgbuild_1.2.0        
[31] htmltools_0.5.4        tidyselect_1.1.1       tibble_3.1.4          
[34] codetools_0.2-18       fansi_0.5.0            crayon_1.4.1          
[37] dplyr_1.0.7            withr_2.4.2            waldo_0.3.1           
[40] grid_3.6.3             jsonlite_1.7.2         gtable_0.3.0          
[43] lifecycle_1.0.1        magrittr_2.0.1         scales_1.1.1          
[46] cli_3.0.1              stringi_1.7.4          cachem_1.0.6          
[49] asdreader_0.1-3        fs_1.5.0               remotes_2.4.1         
[52] latticeExtra_0.6-29    xml2_1.3.2             ellipsis_0.3.2        
[55] generics_0.1.0         vctrs_0.3.8            prospectr_0.2.1       
[58] RColorBrewer_1.1-2     iterators_1.0.13       tools_3.6.3           
[61] glue_1.4.2             purrr_0.3.4            jpeg_0.1-9            
[64] processx_3.5.2         pkgload_1.2.2          fastmap_1.1.0         
[67] colorspace_2.0-2       opusreader_0.4.2       sessioninfo_1.1.1     
[70] memoise_2.0.1          knitr_1.30             usethis_2.0.1         

and

> sessionInfo()
R version 4.2.1 (2022-06-23 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19045)

Matrix products: default

locale:
[1] LC_COLLATE=English_Australia.utf8  LC_CTYPE=English_Australia.utf8    LC_MONETARY=English_Australia.utf8 LC_NUMERIC=C                      
[5] LC_TIME=English_Australia.utf8    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] soilspec.format_0.0.0.9001 testthat_3.1.4            

loaded via a namespace (and not attached):
 [1] opusreader_0.4.2       pkgload_1.3.0          jsonlite_1.8.0         foreach_1.5.2          opusreader2_0.5.0.9000 brio_1.1.3             shiny_1.7.2           
 [8] latticeExtra_0.6-30    remotes_2.4.2          sessioninfo_1.2.2      pillar_1.8.1           lattice_0.20-45        glue_1.6.2             digest_0.6.29         
[15] RColorBrewer_1.1-3     promises_1.2.0.1       colorspace_2.0-3       htmltools_0.5.3        httpuv_1.6.5           asdreader_0.1-3        pkgconfig_2.0.3       
[22] devtools_2.4.4         purrr_0.3.4            xtable_1.8-4           scales_1.2.1           processx_3.7.0         jpeg_0.1-9             later_1.3.0           
[29] prospectr_0.2.6        tibble_3.1.8           generics_0.1.3         ggplot2_3.3.6          usethis_2.1.6          ellipsis_0.3.2         cachem_1.0.6          
[36] withr_2.5.0            hyperSpec_0.100.0      lazyeval_0.2.2         cli_3.3.0              magrittr_2.0.3         crayon_1.5.1           mime_0.12             
[43] deldir_1.0-6           evaluate_0.16          memoise_2.0.1          ps_1.7.1               fs_1.5.2               fansi_1.0.3            xml2_1.3.3            
[50] pkgbuild_1.3.1         profvis_0.3.7          tools_4.2.1            prettyunits_1.1.1      lifecycle_1.0.1        stringr_1.4.1          interp_1.1-3          
[57] munsell_0.5.0          callr_3.7.2            compiler_4.2.1         rlang_1.0.4            grid_4.2.1             iterators_1.0.14       rstudioapi_0.14       
[64] htmlwidgets_1.5.4      miniUI_0.1.1.1         rmarkdown_2.16         waldo_0.4.0            codetools_0.2-18       gtable_0.3.1           curl_4.3.2            
[71] R6_2.5.1               knitr_1.40             dplyr_1.0.9            fastmap_1.1.0          utf8_1.2.2             mathjaxr_1.6-0         rprojroot_2.0.3       
[78] desc_1.4.1             stringi_1.7.8          Rcpp_1.0.9             vctrs_0.4.1            png_0.1-7              tidyselect_1.1.2       xfun_0.32             
[85] urlchecker_1.0.1

I don't have the GH Actions sessionInfo(). Let me know if you want that also.

[philipp-baumann]

I also tagged a new release, so you can do like this:

devtools::install_github("spectral-cockpit/opusreader2@opusreader2_v0.5.0.9000")

I changed the GH Actions workflow YAML to install this version and also installed it in Windows and the other Linux env I mentioned and it works in all three now.

Thanks! As to why this is different from master, a good question

Currently, I don't have access to a Win environment (made an Azure subscription, but ran out of my trial time). @davidbenncsiro what does you sessionInfo() return on Win vs. on Linux?

Here's two envs:

> sessionInfo()
R version 3.6.3 (2020-02-29)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.2 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.7.1
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.7.1

locale:
 [1] LC_CTYPE=en_AU       LC_NUMERIC=C         LC_TIME=en_AU       
 [4] LC_COLLATE=en_AU     LC_MONETARY=en_AU    LC_MESSAGES=en_AU   
 [7] LC_PAPER=en_AU       LC_NAME=C            LC_ADDRESS=C        
[10] LC_TELEPHONE=C       LC_MEASUREMENT=en_AU LC_IDENTIFICATION=C 

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] soilspec.format_0.0.0.9001 testthat_3.0.4            

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.7             lattice_0.20-41        prettyunits_1.1.1     
 [4] png_0.1-7              ps_1.6.0               hyperSpec_0.100.0     
 [7] rprojroot_2.0.2        digest_0.6.28          foreach_1.5.1         
[10] utf8_1.2.2             R6_2.5.1               evaluate_0.14         
[13] ggplot2_3.3.5          pillar_1.6.3           rlang_0.4.11          
[16] lazyeval_0.2.2         curl_4.3.2             rstudioapi_0.13       
[19] callr_3.7.0            rmarkdown_2.18         mathjaxr_1.0-1        
[22] desc_1.4.0             devtools_2.4.2         opusreader2_0.5.0.9000
[25] stringr_1.4.0          munsell_0.5.0          compiler_3.6.3        
[28] xfun_0.35              pkgconfig_2.0.3        pkgbuild_1.2.0        
[31] htmltools_0.5.4        tidyselect_1.1.1       tibble_3.1.4          
[34] codetools_0.2-18       fansi_0.5.0            crayon_1.4.1          
[37] dplyr_1.0.7            withr_2.4.2            waldo_0.3.1           
[40] grid_3.6.3             jsonlite_1.7.2         gtable_0.3.0          
[43] lifecycle_1.0.1        magrittr_2.0.1         scales_1.1.1          
[46] cli_3.0.1              stringi_1.7.4          cachem_1.0.6          
[49] asdreader_0.1-3        fs_1.5.0               remotes_2.4.1         
[52] latticeExtra_0.6-29    xml2_1.3.2             ellipsis_0.3.2        
[55] generics_0.1.0         vctrs_0.3.8            prospectr_0.2.1       
[58] RColorBrewer_1.1-2     iterators_1.0.13       tools_3.6.3           
[61] glue_1.4.2             purrr_0.3.4            jpeg_0.1-9            
[64] processx_3.5.2         pkgload_1.2.2          fastmap_1.1.0         
[67] colorspace_2.0-2       opusreader_0.4.2       sessioninfo_1.1.1     
[70] memoise_2.0.1          knitr_1.30             usethis_2.0.1         

and

> sessionInfo()
R version 4.2.1 (2022-06-23 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19045)

Matrix products: default

locale:
[1] LC_COLLATE=English_Australia.utf8  LC_CTYPE=English_Australia.utf8    LC_MONETARY=English_Australia.utf8 LC_NUMERIC=C                      
[5] LC_TIME=English_Australia.utf8    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] soilspec.format_0.0.0.9001 testthat_3.1.4            

loaded via a namespace (and not attached):
 [1] opusreader_0.4.2       pkgload_1.3.0          jsonlite_1.8.0         foreach_1.5.2          opusreader2_0.5.0.9000 brio_1.1.3             shiny_1.7.2           
 [8] latticeExtra_0.6-30    remotes_2.4.2          sessioninfo_1.2.2      pillar_1.8.1           lattice_0.20-45        glue_1.6.2             digest_0.6.29         
[15] RColorBrewer_1.1-3     promises_1.2.0.1       colorspace_2.0-3       htmltools_0.5.3        httpuv_1.6.5           asdreader_0.1-3        pkgconfig_2.0.3       
[22] devtools_2.4.4         purrr_0.3.4            xtable_1.8-4           scales_1.2.1           processx_3.7.0         jpeg_0.1-9             later_1.3.0           
[29] prospectr_0.2.6        tibble_3.1.8           generics_0.1.3         ggplot2_3.3.6          usethis_2.1.6          ellipsis_0.3.2         cachem_1.0.6          
[36] withr_2.5.0            hyperSpec_0.100.0      lazyeval_0.2.2         cli_3.3.0              magrittr_2.0.3         crayon_1.5.1           mime_0.12             
[43] deldir_1.0-6           evaluate_0.16          memoise_2.0.1          ps_1.7.1               fs_1.5.2               fansi_1.0.3            xml2_1.3.3            
[50] pkgbuild_1.3.1         profvis_0.3.7          tools_4.2.1            prettyunits_1.1.1      lifecycle_1.0.1        stringr_1.4.1          interp_1.1-3          
[57] munsell_0.5.0          callr_3.7.2            compiler_4.2.1         rlang_1.0.4            grid_4.2.1             iterators_1.0.14       rstudioapi_0.14       
[64] htmlwidgets_1.5.4      miniUI_0.1.1.1         rmarkdown_2.16         waldo_0.4.0            codetools_0.2-18       gtable_0.3.1           curl_4.3.2            
[71] R6_2.5.1               knitr_1.40             dplyr_1.0.9            fastmap_1.1.0          utf8_1.2.2             mathjaxr_1.6-0         rprojroot_2.0.3       
[78] desc_1.4.1             stringi_1.7.8          Rcpp_1.0.9             vctrs_0.4.1            png_0.1-7              tidyselect_1.1.2       xfun_0.32             
[85] urlchecker_1.0.1

I don't have the GH Actions sessionInfo(). Let me know if you want that also.

looks actually ok to me. Do we need to take further action or can we close?

[philipp-baumann]

@davidbenncsiro I am not so sure, now as all is fine in v 0.5.0.9000, I suggest we close? Just thinking if it's worth fiddling about old versions since we have the full changelog.

[philipp-baumann]

@davidbenncsiro feel free to reopen the issue.