search

weecology / portalcasting

Model development, deployment, and evaluation for forecasting Portal ecosystem dynamics
https://weecology.github.io/portalcasting
Other
8 stars 10 forks source link

"must be a data.frame" error when running `setup_dir()` #192

Closed ethanwhite closed 3 years ago

ethanwhite commented 3 years ago 
> library(portalcasting)
Registered S3 method overwritten by 'quantmod':
  method            from
  as.zoo.data.frame zoo 
> setup_dir(main = "./portalcast")
------------------------------------------------------------
This is portalcasting v0.19.0
------------------------------------------------------------
Establishing portalcasting directory at
 ./portalcast
------------------------------------------------------------
Filling directory with standard content
 -Downloading raw files
  -PortalData
  -portalPredictions
 -Filling casts folder with files from archive
 -Writing model scripts
  -AutoArima
  -ESSS
  -NaiveArima
  -nbGARCH
  -nbsGARCH
  -pevGARCH
  -simplexEDM
  -GPEDM
  -jags_RW
 -Adding data files to data subdirectory
  -lunar data file
  -rodents data files
   -all
   -controls
   -exclosures
   -all_interp
   -controls_interp
   -exclosures_interp
  -covariate data files
Error: in remove_incompletes: `df` must be a data.frame
> sessionInfo()
R version 3.6.3 (2020-02-29)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Pop!_OS 20.04 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3
LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] portalcasting_0.19.0

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.5        ltsa_1.4.6        urca_1.3-0        pillar_1.4.7     
 [5] compiler_3.6.3    viridis_0.5.1     runjags_2.0.4-6   tseries_0.10-48  
 [9] xts_0.12.1        tools_3.6.3       viridisLite_0.3.0 lubridate_1.7.9.2
[13] jsonlite_1.7.2    lifecycle_0.2.0   tibble_3.0.4      gtable_0.3.0     
[17] nlme_3.1-144      lattice_0.20-40   pkgconfig_2.0.3   rlang_0.4.10     
[21] yaml_2.2.1        curl_4.3          parallel_3.6.3    gridExtra_2.3    
[25] coda_0.19-4       lunar_0.1-04      dplyr_1.0.2       httr_1.4.2       
[29] hms_0.5.3         generics_0.1.0    vctrs_0.3.6       lmtest_0.9-38    
[33] grid_3.6.3        nnet_7.3-13       forecast_8.13     tidyselect_1.1.0 
[37] glue_1.4.2        R6_2.5.0          tidyr_1.1.2       readr_1.4.0      
[41] portalr_0.3.6     ggplot2_3.3.3     purrr_0.3.4       TTR_0.24.2       
[45] magrittr_2.0.1    scales_1.1.1      ellipsis_0.3.1    tscount_1.4.3    
[49] quantmod_0.4.18   timeDate_3043.102 colorspace_2.0-0  fracdiff_1.5-1   
[53] quadprog_1.5-8    munsell_0.5.0     crayon_1.3.4      zoo_1.8-8
ethanwhite commented 3 years ago

Same error after upgrading to R 4.0.3.

> sessionInfo()
R version 4.0.3 (2020-10-10)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Pop!_OS 20.04 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3
LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] portalcasting_0.19.0

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.5        ltsa_1.4.6        urca_1.3-0        pillar_1.4.7     
 [5] compiler_4.0.3    viridis_0.5.1     runjags_2.0.4-6   tseries_0.10-48  
 [9] xts_0.12.1        tools_4.0.3       viridisLite_0.3.0 lubridate_1.7.9.2
[13] jsonlite_1.7.2    lifecycle_0.2.0   tibble_3.0.4      gtable_0.3.0     
[17] nlme_3.1-151      lattice_0.20-41   pkgconfig_2.0.3   rlang_0.4.10     
[21] yaml_2.2.1        curl_4.3          parallel_4.0.3    gridExtra_2.3    
[25] coda_0.19-4       lunar_0.1-04      dplyr_1.0.2       httr_1.4.2       
[29] hms_0.5.3         generics_0.1.0    vctrs_0.3.6       lmtest_0.9-38    
[33] grid_4.0.3        nnet_7.3-14       forecast_8.13     tidyselect_1.1.0 
[37] glue_1.4.2        R6_2.5.0          tidyr_1.1.2       readr_1.4.0      
[41] portalr_0.3.6     ggplot2_3.3.3     purrr_0.3.4       TTR_0.24.2       
[45] magrittr_2.0.1    scales_1.1.1      ellipsis_0.3.1    tscount_1.4.3    
[49] quantmod_0.4.18   timeDate_3043.102 colorspace_2.0-0  fracdiff_1.5-1   
[53] quadprog_1.5-8    munsell_0.5.0     crayon_1.3.4      zoo_1.8-8
ethanwhite commented 3 years ago

Appears to have been fixed in #179 (though there are some warnings):

> setup_dir(main = "./test")
------------------------------------------------------------
This is portalcasting v0.20.0
------------------------------------------------------------
Establishing portalcasting directory at
 ./test
------------------------------------------------------------
Filling directory with standard content
 -Downloading raw files
  -PortalData
  -portalPredictions
 -Filling casts folder with files from archive
 -Writing model scripts
  -AutoArima
  -ESSS
  -NaiveArima
  -nbGARCH
  -nbsGARCH
  -pevGARCH
  -simplexEDM
  -GPEDM
  -jags_RW
 -Adding data files to data subdirectory
  -lunar data file
  -rodents data files
   -all
   -controls
   -exclosures
   -dm_controls
   -all_interp
   -controls_interp
   -exclosures_interp
  -covariate data files
  -metadata file
------------------------------------------------------------
Directory successfully instantiated
------------------------------------------------------------
Warning messages:
1: In min(.data$mintemp, na.rm = TRUE) :
  no non-missing arguments to min; returning Inf
2: In max(.data$maxtemp, na.rm = TRUE) :
  no non-missing arguments to max; returning -Inf
3: In min(x$mintemp[moons_in], na.rm = TRUE) :
  no non-missing arguments to min; returning Inf
4: In max(x$maxtemp[moons_in], na.rm = TRUE) :
  no non-missing arguments to max; returning -Inf
> sessionInfo()
R version 4.0.3 (2020-10-10)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Pop!_OS 20.04 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3
LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] portalcasting_0.20.0

loaded via a namespace (and not attached):
 [1] zoo_1.8-8          tidyselect_1.1.0   xfun_0.20          purrr_0.3.4       
 [5] urca_1.3-0         lattice_0.20-41    colorspace_2.0-0   vctrs_0.3.6       
 [9] generics_0.1.0     viridisLite_0.3.0  yaml_2.2.1         rlang_0.4.10      
[13] pillar_1.4.7       tscount_1.4.3      glue_1.4.2         forecast_8.13     
[17] TTR_0.24.2         lifecycle_0.2.0    quantmod_0.4.18    lunar_0.1-04      
[21] timeDate_3043.102  rEDM_0.7.5         munsell_0.5.0      gtable_0.3.0      
[25] codetools_0.2-18   coda_0.19-4        knitr_1.30         tseries_0.10-48   
[29] lmtest_0.9-38      parallel_4.0.3     curl_4.3           xts_0.12.1        
[33] Rcpp_1.0.5         scales_1.1.1       jsonlite_1.7.2     fracdiff_1.5-1    
[37] gridExtra_2.3      ggplot2_3.3.3      dplyr_1.0.2        runjags_2.0.4-6   
[41] portalr_0.3.6      grid_4.0.3         quadprog_1.5-8     tools_4.0.3       
[45] magrittr_2.0.1     ltsa_1.4.6         tibble_3.0.4       scoringRules_1.0.1
[49] crayon_1.3.4       tidyr_1.1.2        pkgconfig_2.0.3    ellipsis_0.3.1    
[53] MASS_7.3-53        lubridate_1.7.9.2  httr_1.4.2         viridis_0.5.1     
[57] R6_2.5.0           nnet_7.3-14        nlme_3.1-151       compiler_4.0.3
skmorgane commented 3 years ago

I got the same error when running setup_dir: "Error: in remove_incompletes: df must be a data.frame"

ethanwhite commented 3 years ago

@juniperlsimonis - you mentioned yesterday that this is upstream. Can you point me in the right direction?

juniperlsimonis commented 3 years ago

@ethanwhite the issue is in the data being generated by portalr::weather having NAs at the end

gmyenni commented 3 years ago

It seems like this specific error is actually a problem with arg_checks in remove_incompletes. The portalr output is a data.frame, and the desired behavior is to return the data.frame with the last row still included (there are NAs, but newmoonnumber is not NA).

portalr::weather(level = "newmoon", fill = TRUE) %>% 
ungroup() %>%
select(all_of(cols)) %>%
remove_incompletes("newmoonnumber", arg_checks = FALSE)

and 

dat <- portalr::weather(level = "newmoon", fill = TRUE) %>% 
ungroup() %>%
select(all_of(cols))
new <- remove_incompletes(dat,"newmoonnumber")

both still work.

I think the portalr issue is separate. The NAs are 'correct.' But maybe we don't want weather to ever give us NAs when fill=TRUE, no matter what. So we have to decide how to fill those in. I can start an issue for that over in portalr.

juniperlsimonis commented 3 years ago

is anyone getting the error when working on the juniper_active branch ? i think that got addressed already, but want to verify

ethanwhite commented 3 years ago

Confirming that juniper_active is working for me. Do we have an ETA on when #179 might go in? If it might be a while is the fix for this cherry pickable into master? If so I can help with the associated git juggling.

juniperlsimonis commented 3 years ago

great! im hoping to get it wrapped up and merged in by the start of next week.

ethanwhite commented 3 years ago

Fixed in #179