DataONEorg / rdataone

R package for reading and writing data at DataONE data repositories
http://doi.org/10.5063/F1M61H5X
36 stars 19 forks source link

add the limit argument to `getDataObject` for resource maps #280

Closed laijasmine closed 2 years ago

laijasmine commented 2 years ago

When trying to get larger data packages (hundreds to thousand objects) I was getting this error:

library(dataone)
d1c <- D1Client("PROD", "urn:node:ARCTIC")
packageId <- "resource_map_urn:uuid:609466e6-28f8-447c-acc7-2de5c01dee0b"
dp <- getDataPackage(d1c, identifier=packageId, lazyLoad=TRUE, quiet=FALSE, limit = "1TB")

#Error in rawToChar(resMapBytes) : argument 'x' must be a raw vector
#In addition: There were 50 or more warnings (use warnings() to see the first 50)

We were limited by the 1MB default in getDataObject as this argument was not specified for the resource map in getDataPackage. The limit argument was added to be consistent.

This bug needs to be fixed in order for Dataone users to edit large data packages

laijasmine commented 2 years ago

Not sure why the checks failed - any ideas @mbjones or @gothub ?

gothub commented 2 years ago

Thanks for the fix @laijasmine!

Also, I'll look into why the checks are failing

laijasmine commented 2 years ago

@gothub That sounds reasonable - I've edited file to use that

laijasmine commented 2 years ago

@mbjones I think having a -1 as a limit would be helpful especially for resource maps.