ternaustralia / ausplotsR

R package to interact with TERN AusPlots data
GNU General Public License v3.0
10 stars 6 forks source link

HTTP 503 error trying to download soil data with `get_ausplots` #66

Closed rubysaltbush closed 1 year ago

rubysaltbush commented 1 year ago

Hello ausplots!

I'm interested in checking out the soil data available for ausplots sites, but every time I try to download the data I get an error message. At first I thought the servers must be down but I've tried a few times now on different computers so I think maybe there's an error in the API function or the database configuration?

Function I've run:

soil <- get_ausplots(site_info = FALSE, soil_subsites = TRUE, 
                     soil_bulk_density = TRUE, soil_character = TRUE)

Returns error message:

Error in .ausplots_api(path, query) : 
  Service Unavailable (HTTP 503). Failed to {"details":"no connection to the server\n","message":"Database client error"}.

And session info in case that's relevant:

sessionInfo()
R version 4.1.3 (2022-03-10)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS Monterey 12.6.5

Matrix products: default
LAPACK: /Library/Frameworks/R.framework/Versions/4.1/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] GGally_2.1.2        raster_3.6-20       sp_1.6-0            Hmisc_5.0-1         sf_1.0-12           ggpubr_0.6.0        ausflora_0.0.0.9000
 [8] lubridate_1.9.2     forcats_1.0.0       stringr_1.5.0       dplyr_1.1.1         purrr_1.0.1         readr_2.1.4         tidyr_1.3.0        
[15] tibble_3.2.1        ggplot2_3.4.2       tidyverse_2.0.0     ausplotsR_1.2.8     mapdata_2.3.1       maps_3.4.1          vegan_2.6-4        
[22] lattice_0.21-8      permute_0.9-7       austraits_1.1.1     RefManageR_1.4.0   

loaded via a namespace (and not attached):
  [1] colorspace_2.1-0   ggsignif_0.6.4     class_7.3-21       htmlTable_2.4.1    base64enc_0.1-3    rstudioapi_0.14    proxy_0.4-27      
  [8] bit64_4.0.5        fansi_1.0.4        xml2_1.3.3         codetools_0.2-19   splines_4.1.3      R.methodsS3_1.8.2  knitr_1.42        
 [15] itertools_0.1-3    Formula_1.2-5      jsonlite_1.8.4     betapart_1.6       broom_1.0.4        cluster_2.1.4      R.oo_1.25.0       
 [22] compiler_4.1.3     httr_1.4.5         backports_1.4.1    Matrix_1.5-1       fastmap_1.1.1      cli_3.6.1          htmltools_0.5.5   
 [29] tools_4.1.3        gtable_0.3.3       glue_1.6.2         fastmatch_1.1-3    Rcpp_1.0.10        carData_3.0-5      vctrs_0.6.1       
 [36] ape_5.7-1          nlme_3.1-162       iterators_1.0.14   xfun_0.38          timechange_0.2.0   lifecycle_1.0.3    gtools_3.9.4      
 [43] rstatix_0.7.2      terra_1.7-23       MASS_7.3-58.3      scales_1.2.1       rcdd_1.5           vroom_1.6.1        hms_1.1.3         
 [50] doSNOW_1.0.20      parallel_4.1.3     RColorBrewer_1.1-3 curl_5.0.0         gridExtra_2.3      jose_1.2.0         rpart_4.1.19      
 [57] reshape_0.8.9      stringi_1.7.12     maptools_1.1-6     foreach_1.5.2      e1071_1.7-13       checkmate_2.1.0    bibtex_0.5.1      
 [64] geometry_0.4.7     rlang_1.1.0        pkgconfig_2.0.3    evaluate_0.20      htmlwidgets_1.6.2  bit_4.0.5          tidyselect_1.2.0  
 [71] plyr_1.8.8         magrittr_2.0.3     R6_2.5.1           snow_0.4-4         generics_0.1.3     picante_1.8.2      DBI_1.1.3         
 [78] pillar_1.9.0       foreign_0.8-84     withr_2.5.0        mgcv_1.8-42        units_0.8-1        abind_1.4-5        nnet_7.3-18       
 [85] crayon_1.5.2       car_3.1-2          KernSmooth_2.23-20 utf8_1.2.3         rmarkdown_2.21     tzdb_0.3.0         grid_4.1.3        
 [92] data.table_1.14.8  minpack.lm_1.2-3   digest_0.6.31      classInt_0.4-9     R.utils_2.12.2     openssl_2.0.6      munsell_0.5.0     
 [99] magic_1.6-1        askpass_1.1       
GregGuerin commented 1 year ago

Hello and thanks for flagging this issue.

We are having some server capacity issues that we are working to resolve, due to the increasing size of the database.

Can I suggest you try including the argument veg.PI=FALSE in your above call to get_ausplots? The vegetation point intercept table is the largest and is likely causing the problem. If you do need the point intercept data, for now you can try reducing the size by using the bounding box option or extracting the site info table and using the site names to subset a subsequent call with veg.PI.

Hope that helps a little.

Greg

rubysaltbush commented 1 year ago

Hi Greg!

Thanks for replying so fast! Setting veg.PI = FALSE and also veg.vouchers = FALSE did the trick for getting soil data, I forgot that these are fetched by default by get_ausplots.

I've tested and you're right, running get_ausplots with just veg.PI = TRUE and everything else set to FALSE returns the same HTTP 503 error so it must just be the huge point-intercept table causing trouble. Lucky I have some cached veg.PI data to play around with.

I'll mark this issue as closed for now with this workaround, good luck with your servers!

Best, Ruby