search

NEONScience / NEON-utilities

Utilities and scripts for working with NEON data. Currently: an R package with functions to join (stack) the month-by-site files in downloaded NEON data, to convert data to geoCSV format, and to download data from the API.
GNU Affero General Public License v3.0
57 stars 36 forks source link

Stacking tables failure #115

Closed walleo860 closed 3 years ago

walleo860 commented 3 years ago

Function loadByProduct()

Describe the bug The function downloads the data but then fails durring stacking

To Reproduce

portaldata <- loadByProduct( dpID = 'DP1.10081.001',

site = site,

                            startdate = '2018-01',
                            enddate = '2019-12',
                            check.size = FALSE,
                            token = Sys.getenv('NEON_TOKEN'),
                            package = 'expanded'

)
produces the error: Error in if ((datafl.splitName[[1]][4] == "DP4") && (datafl.splitName[[1]][5] == : missing value where TRUE/FALSE needed

Expected behavior It should stack the tables so I can use the data however it just fails durring stacking. This seems to work OK when doing the basic package.

System (please complete the following information):

cklunch commented 3 years ago

A couple of versions back, I added functionality to attempt to guess the table type (site-date, site-all, lab-all, etc) when table type was unknown, and it looks like that had the side effect that now it's attempting to stack the per sample files in the microbe community composition data products (also affects DP1.30012.001, Field spectra), and failing. This will take some time to fix, but I'll make sure to include it in the next version.

In the meantime, you can get around this either by using the basic package, which only includes URLs to the per sample files, instead of the files themselves, or by downloading with zipsByProduct(), moving the per sample files to another directory, and then stacking using stackByTable().

cklunch commented 3 years ago

Stacking of per sample files is enabled, expecting to send to CRAN next week.

cklunch commented 3 years ago

Stacking of per sample files is included in neonUtilities 2.1.0, now on CRAN.