rmendels / rerddapXtracto

xtractomactic using rerddap
Other
14 stars 4 forks source link

rxtracto_3D error downloading #31

Closed elizeng closed 1 year ago

elizeng commented 1 year ago

I am trying to map the global mean chlorophyll a levels while following the example here: https://coastwatch.pfeg.noaa.gov/projects/r/timeseries-chl.html I was successful in recreating the example presented in the link and wanted to extend it to a global scale and that was where it failed. I am wondering if it is due to memory issues. I have tried clearing the cache with rerddap::cache_delete_all(force = TRUE) to no avail.

Would like to seek some help regarding whether it is possible to continue pursuing this avenue, or should I look elsewhere.

> xcoord<-c(-179, 179)
> ycoord<-c(-89,89)
> ##Format Box Coordinates for cosmetics, to make a nice map title
> ttext<-paste(paste(abs(xcoord), collapse="-"),"W, ", paste(ycoord, collapse="-"),"N")
> # Use rerddap to get information about the dataset
> # if you encouter an error reading the nc file clear the rerrdap cache: 
> # rerddap::cache_delete_all(force = TRUE)
> dataInfo <- rerddap::info('erdMH1chlamday')
> dataInfo
<ERDDAP info> erdMH1chlamday 
 Base URL: https://upwell.pfeg.noaa.gov/erddap 
 Dataset Type: griddap 
 Dimensions (range):  
     time: (2003-01-16T00:00:00Z, 2022-05-16T00:00:00Z) 
     latitude: (-89.97917, 89.97916) 
     longitude: (-179.9792, 179.9792) 
 Variables:  
     chlorophyll: 
         Units: mg m-3 
> 
> # Extract the parameter name from the metadata in dataInfo
> parameter <- dataInfo$variable$variable_name
> 
> #Extract the start and end times of the dataset from the metadata in dataInfo
> global <- dataInfo$alldata$NC_GLOBAL
> 
> # Populate the time vector with the time_coverage_start from dataInfo
> # Use the "last" option for the ending date
> tt <- global[ global$attribute_name %in% c('time_coverage_end','time_coverage_start'), "value", ]
> tcoord <- c(tt[2],"last")
> 
> # Run rxtracto_3D
> chlMODIS<-rxtracto_3D(dataInfo,parameter=parameter,
+                       tcoord=tcoord,
+                       xcoord=xcoord,ycoord=ycoord)
 0s 0sinfo() output passed to x; setting base url to: https://upwell.pfeg.noaa.gov/erddap                                                               0s
info() output passed to x; setting base url to: https://upwell.pfeg.noaa.gov/erddap
info() output passed to x; setting base url to: https://upwell.pfeg.noaa.gov/erddap
info() output passed to x; setting base url to: https://upwell.pfeg.noaa.gov/erddap
info() output passed to x; setting base url to: https://upwell.pfeg.noaa.gov/erddap
info() output passed to x; setting base url to: https://upwell.pfeg.noaa.gov/erddap
info() output passed to x; setting base url to: https://upwell.pfeg.noaa.gov/erddap
info() output passed to x; setting base url to: https://upwell.pfeg.noaa.gov/erddap
info() output passed to x; setting base url to: https://upwell.pfeg.noaa.gov/erddap
info() output passed to x; setting base url to: https://upwell.pfeg.noaa.gov/erddap
info() output passed to x; setting base url to: https://upwell.pfeg.noaa.gov/erddap
[1] "error in trying to download the subset"
[1] "check your settings"
$x
<ERDDAP info> erdMH1chlamday 
 Base URL: https://upwell.pfeg.noaa.gov/erddap 
 Dataset Type: griddap 
 Dimensions (range):  
     time: (2003-01-16T00:00:00Z, 2022-05-16T00:00:00Z) 
     latitude: (-89.97917, 89.97916) 
     longitude: (-179.9792, 179.9792) 
 Variables:  
     chlorophyll: 
         Units: mg m-3 

$longitude
[1] -179.0208  178.9792

$latitude
[1] -88.97917  89.02084

$time
[1] "2003-01-16T00:00:00Z" "2022-05-16T00:00:00Z"

$fields
[1] "chlorophyll"

$read
[1] FALSE

[1] "stopping execution  - will return what has been downloaded so far"
[1] "There was an error in the url call, perhaps a time out. See message on screen and URL called"
Error in rxtracto_3D(dataInfo, parameter = parameter, tcoord = tcoord,  : 
  stopping download
rmendels commented 1 year ago

@elizeng - Each global file is roughly 60MB, so one year would be about 720MB, over the period you are requesting the file size would be way too big. Write a script to use the "files" command (https://coastwatch.pfeg.noaa.gov/erddap/files/documentation.html) the files you are after I believe are at https://coastwatch.pfeg.noaa.gov/erddap/files/erdMH1chlamday/ but understand that unless you have a very fast lie this will be a slow process, and our line s not that fast.

I am closing this issue.