Error 404 for any rerddap function

[AThielecke]

Hi, I am trying to download OISST data, using the following code:

# The information for the NOAA OISST data
OISST_info<-rerddap::info(datasetid = "ncdcOisst21Agg_LonPM180", url = "https://coastwatch.pfeg.noaa.gov/erddap/")

#get erddap gridded data
OISST_datA <- griddap(x = "ncdcOisst21Agg_LonPM180", 
                     url = "https://coastwatch.pfeg.noaa.gov/erddap/", 
                     time = c("2016-11-12", "2017-11-12"),
                     store = disk(),
                     zlev = c(0, 0),
                     latitude = c(40, 87),
                     longitude = c(-75, 80),
                     fields = "sst")

OISST_datB <- griddap(x = "ncdcOisst21Agg_LonPM180", 
                      url = "https://coastwatch.pfeg.noaa.gov/erddap/", 
                      time = c("2017-11-13", "2018-11-12"),
                      store = disk(),
                      zlev = c(0, 0),
                      latitude = c(40, 87),
                      longitude = c(-75, 80),
                      fields = "sst")

The code ran fine the first time for OISST_datA, but when trying to run the griddap code for the second time for OISST_datB I received the following error message:

Error: 'Error {
    code=404;
    message="Not Found: Currently unknown datasetID=ncdcOisst21Agg_LonPM180";
}
' does not exist in current working directory ('/Volumes/.../Thesis_Data').

(current working directory anonymised) Trying again I am now receiving the same error when running the rerddap::info line. Restarting R and my computer as well as downloading erddap again hasn't made a difference. Nor has removing the OISST_datA file and trying to reload again.

My sessionInfo is the following:

> sessionInfo()
R version 4.1.0 (2021-05-18)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS Catalina 10.15.7

Matrix products: default
BLAS:   /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/4.1/Resources/lib/libRlapack.dylib

locale:
[1] en_GB.UTF-8/en_GB.UTF-8/en_GB.UTF-8/C/en_GB.UTF-8/en_GB.UTF-8

attached base packages:
[1] parallel  stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] doParallel_1.0.16 iterators_1.0.13  foreach_1.5.1     rerddap_0.7.4     tidync_0.2.4     
[6] ggplot2_3.3.3     lubridate_1.7.10  dplyr_1.0.6      

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.6          lattice_0.20-44     assertthat_0.2.1    digest_0.6.27       utf8_1.2.1         
 [6] R6_2.5.0            plyr_1.8.6          RSQLite_2.2.7       pillar_1.6.1        rlang_0.4.11       
[11] curl_4.3.1          ncmeta_0.3.0        data.table_1.14.0   raster_3.4-10       blob_1.2.1         
[16] urltools_1.7.3      stringr_1.4.0       triebeard_0.3.0     bit_4.0.4           munsell_0.5.0      
[21] tinytex_0.31        compiler_4.1.0      xfun_0.23           pkgconfig_2.0.3     shape_1.4.6        
[26] marmap_1.0.5        tidyselect_1.1.1    tibble_3.1.2        httpcode_0.3.0      codetools_0.2-18   
[31] fansi_0.4.2         crayon_1.4.1        hoardr_0.5.2        withr_2.4.2         rappdirs_0.3.3     
[36] crul_1.1.0          grid_4.1.0          jsonlite_1.7.2      gtable_0.3.0        lifecycle_1.0.0    
[41] DBI_1.1.1           adehabitatMA_0.3.14 magrittr_2.0.1      scales_1.1.1        ncdf4_1.17         
[46] stringi_1.6.2       cachem_1.0.5        reshape2_1.4.4      sp_1.4-5            xml2_1.3.2         
[51] ellipsis_0.3.2      generics_0.1.0      vctrs_0.3.8         tools_4.1.0         forcats_0.5.1      
[56] bit64_4.0.5         glue_1.4.2          RNetCDF_2.4-2       purrr_0.3.4         fastmap_1.1.0      
[61] colorspace_2.0-1    memoise_2.0.0  

I'll be grateful for any ideas, since I am relatively inexperienced in R. Best, A

[AThielecke]

Just realised it's the same issue as #98. However I am keeping it open, since #98 remains unsolved

[sckott]

Thanks @AThielecke for the issue.

There was an issue with constructing URLs. I've just pushed a fix. Please try again after reinstalling like remotes::install_github("ropensci/rerddap") and let me know if it's fixed.

[rmendels]

@AThielecke @grwhumphries @sckott

I have taken over maintenance of 'rerddap'. I think I have a fix for the issue or trailing slashes in all cases, which I just pushed to Github. If possible could you try reinstalling using:

remotes::install_github("ropensci/rerddap")

and let me know if you see any problems

[DarrenLCY]

Hello,

Sorry to re-open this old thread. I have the same problem that AThielecke had. I'm following this tutorial to download OISST data, but I get the same error message. I've tried reinstalling using:

remotes::install_github("ropensci/rerddap")

But I still get the same error message. Have there been any changes made recently?

Here is my sessionInfo():

> sessionInfo()
R version 4.2.1 (2022-06-23 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 22621)

Matrix products: default

locale:
[1] LC_COLLATE=English_Australia.utf8  LC_CTYPE=English_Australia.utf8    LC_MONETARY=English_Australia.utf8 LC_NUMERIC=C                      
[5] LC_TIME=English_Australia.utf8    

attached base packages:
[1] parallel  stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] doParallel_1.0.17 iterators_1.0.14  foreach_1.5.2     rerddap_1.0.2     tidync_0.3.0      ggplot2_3.4.0     lubridate_1.9.0   timechange_0.2.0 
[9] dplyr_1.0.10     

loaded via a namespace (and not attached):
 [1] tidyselect_1.2.0  remotes_2.4.2     purrr_1.0.1       colorspace_2.0-3  vctrs_0.5.1       generics_0.1.3    utf8_1.2.2        rlang_1.0.6      
 [9] pkgbuild_1.4.0    pillar_1.8.1      httpcode_0.3.0    glue_1.6.2        withr_2.5.0       DBI_1.1.3         rappdirs_0.3.3    RNetCDF_2.6-2    
[17] lifecycle_1.0.3   stringr_1.5.0     munsell_0.5.0     gtable_0.3.1      codetools_0.2-18  forcats_0.5.2     callr_3.7.3       ps_1.7.2         
[25] curl_5.0.0        hoardr_0.5.3      fansi_1.0.3       triebeard_0.3.0   urltools_1.7.3    Rcpp_1.0.9        scales_1.2.1      desc_1.4.2       
[33] jsonlite_1.8.4    digest_0.6.31     stringi_1.7.12    processx_3.8.0    ncdf4_1.21        grid_4.2.1        rprojroot_2.0.3   cli_3.6.0        
[41] tools_4.2.1       magrittr_2.0.3    tibble_3.1.8      crul_1.3          crayon_1.5.2      pkgconfig_2.0.3   data.table_1.14.6 xml2_1.3.3       
[49] prettyunits_1.1.1 assertthat_0.2.1  rstudioapi_0.14   R6_2.5.1          ncmeta_0.3.5      compiler_4.2.1 

Thank you so much, Darren

[rmendels]

@DarrenLCY - I am confused. The tutorial you point to tries tp access the NCEI ERDDAP, the OP here was for the WCN ERDDAP. If it is for the WCN ERDDAP, we had some equipment issues and had to restart that ERDDAP, and it will take awhile for it to reload. But the only way I can determine anything is if you give me the actual steps in your R attempt

[DarrenLCY]

Hello,

Thanks for the prompt response! I'm not trying to download the global data through the NCEI ERDDAP, sorry for the confusion. It's exactly the same website used above. Here is the code that I used from the tutorial:

# This function downloads and prepares data based on user provided start and end dates
OISST_sub_dl <- function(time_df){
  OISST_dat <- rerddap::griddap(datasetx = "ncdcOisst21Agg_LonPM180",
                                url = "https://coastwatch.pfeg.noaa.gov/erddap/", 
                                time = c(time_df$start, time_df$end), 
                                zlev = c(0, 0),
                                latitude = c(40, 45),
                                longitude = c(2, 5),
                                fields = "sst")$data %>% 
    dplyr::mutate(time = base::as.Date(stringr::str_remove(time, "T12:00:00Z"))) %>% 
    dplyr::rename(t = time, temp = sst, lon = longitude, lat = latitude) %>% 
    dplyr::select(lon, lat, t, temp) %>% 
    stats::na.omit()
}

# Date download range by start and end dates per year
dl_years <- data.frame(date_index = 1:5,
                       start = as.Date(c("1982-01-01", "1990-01-01", 
                                         "1998-01-01", "2006-01-01", "2014-01-01")),
                       end = as.Date(c("1989-12-31", "1997-12-31", 
                                       "2005-12-31", "2013-12-31", "2023-01-20")))

# Download all of the data with one nested request
# The time this takes will vary greatly based on connection speed
base::system.time(
  OISST_data <- dl_years %>% 
    dplyr::group_by(date_index) %>% 
    dplyr::group_modify(~OISST_sub_dl(.x)) %>% 
    dplyr::ungroup() %>% 
    dplyr::select(lon, lat, t, temp)
) # 518 seconds, ~100 seconds per batch

Basically, the first time I ran the last chunk of code was fine. But after I ran it again (because probably all data did not finish downloading, as mentioned in the tutorial), I encountered the same error as above:

> base::system.time(
+   OISST_data <- dl_years %>% 
+     dplyr::group_by(date_index) %>% 
+     dplyr::group_modify(~OISST_sub_dl(.x)) %>% 
+     dplyr::ungroup() %>% 
+     dplyr::select(lon, lat, t, temp)
+ ) # 518 seconds, ~100 seconds per batch
Error: 'Error {
code=404;
message="Not Found: Currently unknown datasetID=ncdcOisst21Agg_LonPM180";
}
' does not exist in current working directory ('C:/Users/myusername/OneDrive/Code').Timing stopped at: 0.01 0.02 1.89

Trying to run the following also gives the same error, when at first it was working fine:

> rerddap::info(datasetid = "ncdcOisst21Agg_LonPM180", 
+               url = "https://coastwatch.pfeg.noaa.gov/erddap/")
Error: 'Error {
    code=404;
    message="Not Found: Currently unknown datasetID=ncdcOisst21Agg_LonPM180";
}
' does not exist in current working directory ('C:/Users/myusername/OneDrive/Code').

Thanks so much, Darren

[rmendels]

@DarrenLCY - Yes that code points to the WCN ERDDAP. That ERDDAP is in the midst or reloading datasets because we had to do a restart due to equipment problems. It may be another couple of hours until everything reloads. Try again tomorrow morning.

[DarrenLCY]

Sounds good, thanks @rmendels!

[rmendels]

@DarrenLCY Did you try this morning? I would like close this.

[Aliya-Caldwell]

I am having the same issue this morning. Using basically the same code as @DarrenLCY. It was working fine two days ago.

[rmendels]

@Aliya-Caldwell we had to update our server yesterday, which meant wiping the server and installing a new OS. There are still a few hiccups, but things are generally back up and running. I will be working today to try and smooth things out. Our apologies, but it was necessary for us to do this.

[DarrenLCY]

Hi @rmendels, sorry for not following up. You can close the thread, it's been working for me. Thank you so much!

[Aliya-Caldwell]

@rmendels I am still getting the same error when I attempt to download data:

Error: 'Error { code=404; message="Not Found: Currently unknown datasetID=ncdcOisst21Agg_LonPM180"; } ' does not exist in current working directory (my working directory)

I am guessing you are just doing another round of updates to your server? Otherwise, is it possible I am doing something wrong considering @DarrenLCY isn't having issues anymore?

My code is as follows:

`rerddap::info(datasetid = "ncdcOisst21Agg_LonPM180", url = "https://coastwatch.pfeg.noaa.gov/erddap/")

# function to loop through time periods

sst_grab <- function(time_df){ sst_data <- rerddap::griddap(datasetx = "ncdcOisst21Agg_LonPM180", url = "https://coastwatch.pfeg.noaa.gov/erddap/", time = c(time_df$start, time_df$end), zlev = c(0, 0), latitude = c(41.5, 44), longitude = c(-72, -69.5), fields = "sst")$data %>% dplyr::mutate(time = base::as.Date(stringr::str_remove(time, "T12:00:00Z"))) %>% dplyr::rename(t = time, temp = sst, lon = longitude, lat = latitude) %>% dplyr::select(lon, lat, t, temp) %>% stats::na.omit() }

# Date download range by start and end dates 

grab_years <- data.frame(date_index = 1:4, start = as.Date(c("1999-01-01", "2006-01-01","2013-01-01", "2020-01-01")), end = as.Date(c("2005-12-31", "2012-12-31", "2019-12-31", "2022-12-31")))

# Download data 

base::system.time( sst_data <- grab_years %>% dplyr::group_by(date_index) %>% dplyr::group_modify(~sst_grab(.x)) %>% dplyr::ungroup() %>% dplyr::select(lon, lat, t, temp) )`

[rmendels]

@Aliya-Caldwell We have been having equipment and power problems, The problem is with the ERDDAP server, not with R ERDDAP. In the future you should go to https://coastwatch.pfeg.noaa.gov/erddap/, first to see if it is running, and second do a search on the datasetid to see of to is available. This probably isn't the appropriate place for this issue (nor the the previous posts), because as I said rerddap is working correctly, the dataset just is not available. Each ERDDAP instance has contact information and that is who you should contact about issues with that server.

[DarrenLCY]

@Aliya-Caldwell Sorry for the confusion. What I meant is that it's been working for me during the times I tried it. I tried it just now again, and it's giving the same error. I guess what you can do is to keep trying until it works, and once it works just download and save the data so that you don't have to download them again. As @rmendels said, the problem is on the server's end.

[katsmimba]

@rmendels thanks for the updates the last few days - it has saved me a lot of stress trying to work out what the problems were with my R code!

I see the dataset is now back on the server, but I am still having trouble calling it.

This part of my code is working fine and I'm getting the correct information in response (previously I was getting the 404 error):

The information for the NOAA OISST data

rerddap::info(datasetid = "ncdcOisst21Agg_LonPM180", url = "https://coastwatch.pfeg.noaa.gov/erddap/")

RESPONSE: ncdcOisst21Agg_LonPM180 Base URL: https://coastwatch.pfeg.noaa.gov/erddap Dataset Type: griddap Dimensions (range):
time: (1981-09-01T12:00:00Z, 2023-02-11T12:00:00Z) zlev: (0.0, 0.0) latitude: (-89.875, 89.875) longitude: (-179.875, 179.875) Variables:
anom: Units: degree_C err: Units: degree_C ice: Units: 1 sst: Units: degree_C

But, when I get to the next step of my code, it seems unable to pull the necessary information. I have run this code hundreds of times over the past couple of years but have never had this particular problem, so any help would be very appreciated!

My code:

This function downloads and prepares data based on user provided start and end dates

OISST_sub_dl <- function(time_df){ OISST_dat <- griddap(x = "ncdcOisst21Agg_LonPM180", url = "https://coastwatch.pfeg.noaa.gov/erddap/", time = c(time_df$start, time_df$end), zlev = c(0, 0), latitude = c(-34.01018, -34.01018), longitude = c(151.23199, 151.23199), fields = "sst")$data %>% mutate(time = as.Date(stringr::str_remove(time, "T00:00:00Z"))) %>% dplyr::rename(t = time, temp = sst) %>% select(lon, lat, t, temp) %>% na.omit() }

Date downloaded range by start and end dates per year

dl_years <- data.frame(date_index = 1:5, start = as.Date(c("1982-01-01", "1990-01-01", "1998-01-01", "2006-01-01", "2014-01-01")), end = as.Date(c("1989-12-31", "1997-12-31", "2005-12-31", "2013-12-31", "2021-11-30")))

May need to complete this step multiple times (until it stops saying timed out) to get all the data

Download all of the data with one nested request

The time this takes will vary greatly based on connection speed

system.time( OISST_data_Sydney <- dl_years %>% group_by(date_index) %>% group_modify(~OISST_sub_dl(.x)) %>% ungroup() %>% select(lon, lat, t, temp)'

RESPONSE: Error in griddap(x = "ncdcOisst21Agg_LonPM180", url = "https://coastwatch.pfeg.noaa.gov/erddap/", : argument "datasetx" is missing, with no default

[rmendels]

@katsmimba

 Oisst_info <- rerddap::info(datasetid = "ncdcOisst21Agg_LonPM180", url = "https://coastwatch.pfeg.noaa.gov/erddap/")

  OISST_dat <- rerddap::griddap(Oisst_info,
                              time = c("2023-01-01T12:00:00Z", "2023-02-11T12:00:00Z"),
                              zlev = c(0, 0),
                              latitude = c(-34.01018, -34.01018),
                              longitude = c(151.23199, 151.23199),
                              fields = "sst",
                              url = "https://coastwatch.pfeg.noaa.gov/erddap/"
)
str(OISST_dat$data)
'data.frame':   41 obs. of  5 variables:
 $ longitude: num [1:41(1d)] 151 151 151 151 151 ...
 $ latitude : num [1:41(1d)] -34.1 -34.1 -34.1 -34.1 -34.1 ...
 $ zlev     : num [1:41(1d)] 0 0 0 0 0 0 0 0 0 0 ...
 $ time     : chr  "2023-01-01T12:00:00Z" "2023-01-02T12:00:00Z" "2023-01-03T12:00:00Z" "2023-01-04T12:00:00Z" ...
 $ sst      : num  22 22.2 22.3 22.5 22.7 ...

If I had to guess, check your times. They may look like you are passing characters. but if they are dates, in fact will not be, need to pass as.character(your-date).

HTH

[katsmimba]

Thanks @rmendels - will take a look!

[Aliya-Caldwell]

@katsmimba have you had any luck?

[rmendels]

@Aliya-Caldwell @katsmimba I just ran the example above no problem. I cleared my cache before doing so to be certain that it was a new download

[Aliya-Caldwell]

it's been working for me as well.

[katsmimba]

Seems to be working again for me - turns out my new, apparently high speed internet is massively slowing the process down!

[NikiDiogou]

Hi, I have been using the same code more or less as in the previous messages for this dataset "noaacwLEOACSPOSSTL3SCDaily", to download data for a fairly large region and time period: latitude= c(68.5, 72.5), longitude= c(-139.5, -116), time= c("2014-01-01", "2022-12-31"). Every time I run the code I get an Error and says Error:Timing stopped at: 979 16.7 1060. Then I run it again. Should I keep doing that, is it just a lot of data that needs several runs to download everything or the error means that I need to change something in the code?

I would really appreciate some feedback! I have many variables that I need to download for this region and I am uncertain if this is even working.

Best, Niki

Here is the code: sst_grab <- function(time_df){ sst_data <- rerddap::griddap(datasetx = "noaacwLEOACSPOSSTL3SCDaily", url = "https://coastwatch.pfeg.noaa.gov/erddap/", time = c(time_df$start, time_df$end),

zlev = c(0, 0),

                           latitude = c(68.5, 72.5),
                           longitude = c(-139.5, -116),
                           fields = "sea_surface_temperature")$data %>%
dplyr::mutate(time = base::as.Date(stringr::str_remove(time, "T12:00:00Z"))) %>%
dplyr::rename(t = time, sst = sea_surface_temperature, lon = longitude, lat = latitude) %>%
dplyr::select(lon, lat, t, sst) %>%
stats::na.omit()

}

Date download range by start and end dates

grab_years <- data.frame(date_index = 1:14, start = as.Date(c("2014-01-01", "2014-08-02","2015-04-02", "2016-01-01", "2016-08-02","2017-04-02", "2018-01-01", "2018-08-02","2019-04-02", "2020-01-01", "2020-08-02","2021-04-02", "2022-01-01", "2022-08-02")), end = as.Date(c("2014-08-01", "2015-04-01", "2015-12-31","2016-08-01", "2017-04-01", "2017-12-31", "2018-08-01", "2019-04-01", "2019-12-31","2020-08-01", "2021-04-01", "2021-12-31", "2022-08-01", "2023-01-01")))

Download data

loop through each time period and download the corresponding data using sst_grab() function.

base::system.time( sst_data <- grab_years %>% dplyr::group_by(date_index) %>% dplyr::group_modify(~sst_grab(.x)) %>% dplyr::ungroup() %>% dplyr::select(lon, lat, t, sst))

Here is my session info (perhaps my computer is not powerful enough for all this data?):

sessionInfo() R version 4.2.2 (2022-10-31 ucrt) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows 10 x64 (build 19044)

Matrix products: default

locale: [1] LC_COLLATE=English_Canada.utf8 LC_CTYPE=English_Canada.utf8 LC_MONETARY=English_Canada.utf8 LC_NUMERIC=C LC_TIME=English_Canada.utf8

attached base packages: [1] parallel stats graphics grDevices utils datasets methods base

other attached packages: [1] cmocean_0.3-1 plot3D_1.4 ncdf4_1.21 mapdata_2.3.1 maps_3.4.1 akima_0.6-3.4 doParallel_1.0.17 iterators_1.0.14 foreach_1.5.2
[10] rerddap_1.0.2 tidync_0.3.0 ggplot2_3.4.0 lubridate_1.9.0 timechange_0.1.1 dplyr_1.0.10

loaded via a namespace (and not attached): [1] Rcpp_1.0.10 lattice_0.20-45 prettyunits_1.1.1 ps_1.7.2 assertthat_0.2.1 rprojroot_2.0.3 digest_0.6.30 utf8_1.2.2 R6_2.5.1
[10] pillar_1.9.0 rlang_1.0.6 curl_4.3.3 misc3d_0.9-1 ncmeta_0.3.5 rstudioapi_0.14 data.table_1.14.4 callr_3.7.3 labeling_0.4.2
[19] urltools_1.7.3 stringr_1.4.1 triebeard_0.4.1 munsell_0.5.0 compiler_4.2.2 pkgconfig_2.0.3 pkgbuild_1.3.1 tcltk_4.2.2 tidyselect_1.2.0 [28] tibble_3.1.8 httpcode_0.3.0 codetools_0.2-18 fansi_1.0.3 crayon_1.5.2 hoardr_0.5.3 withr_2.5.0 rappdirs_0.3.3 crul_1.3
[37] grid_4.2.2 jsonlite_1.8.3 gtable_0.3.1 lifecycle_1.0.3 DBI_1.1.3 magrittr_2.0.3 scales_1.2.1 stringi_1.7.8 cli_3.4.1
[46] farver_2.1.1 remotes_2.4.2 sp_1.6-0 xml2_1.3.3 generics_0.1.3 vctrs_0.5.0 tools_4.2.2 forcats_0.5.2 glue_1.6.2
[55] RNetCDF_2.6-2 purrr_1.0.1 processx_3.8.0 colorspace_2.0-3

[Aliya-Caldwell]

@NikiDiogou From my understanding, if you are getting that error the code is broken somewhere. It won't help to continue running it.

[rmendels]

@NikiDiogou do a small test with 3 time periods and the time bounds are given explicitly as character strings

[NikiDiogou]

@rmendels, thank you kindly, splitting the dates periods worked. So, I will be downloading the data in 4 chunks and then combine them. I am new to spatial analysis, and this process is taking me ages to figure out. Will a simple rbind of the dataframes resulted work or should I transform the data to sf object then use st_join and then transform the data to a raster? I need to extract buffers around certain stations from the entire area and I should use a raster I guess for that. The final format that i need the data is a dataframe though, time series.

Thank you so much again for the support!

[rmendels]

@NikiDiogou you should be able to just bind the data frames you create in your scripts.

[NikiDiogou]

Thank you @rmendels!

[NikiDiogou]

Hi @rmendels,

I have been using the code above for a dataset of SST from 2014 to 2022, I had to break it down to 27 periods of about 3 months each to be able to get it. Like this and has been really inefficient:

grab_years17 <- data.frame(date_index = 1:1, start = as.Date(c( "2019-11-02" )), end = as.Date(c( "2020-04-01" )))

grab_years18 <- data.frame(date_index = 1:1, start = as.Date(c( "2020-04-02" )), end = as.Date(c( "2020-05-01" )))

grab_years19 <- data.frame(date_index = 1:1, start = as.Date(c( "2020-05-02" )), end = as.Date(c( "2020-09-01" )))

Then I read about rxrtacto_3D and was able to download very quickly and efficiently a large data set of chlorophyll, using this code:

xcoord<-c(-140.5, -116) ycoord<-c(68.5,72.5)

Use rerddap to get information about the dataset

dataInfo <- rerddap::info('erdMH1chla1day')

Extract the parameter name from the metadata in dataInfo

parameter <- dataInfo$variable$variable_name tcoord <- c("2014-01-01", "2022-07-15")

Extract the timeseries data using rxtracto_3D

chl<-rxtracto_3D(dataInfo,parameter=parameter, tcoord=tcoord, xcoord=xcoord,ycoord=ycoord)

But I am unsure how to use the object that results. I am trying to convert to a dataframe but I have not succeeded with this:

Convert the sst_data dataframe to an xyz format

chl_xyz <- as.data.frame(chl[c("chlorophyll", "longitude", "latitude", "time")])

I have also been unable to find info online on how to convert a rxtracto_3D object to dataframe.

Any chance you would have any suggestions?

Best, Niki

[rmendels]

@NikiDiogou please read the package vignette, section "Plotting crossing the dateline"

[Ruben7c]

Hi, I have been trying to download data with the below code but keep geting the following error:

"Error in griddap(x = "ncdcOisst21Agg_LonPM180", url = "https://coastwatch.pfeg.noaa.gov/erddap/", : argument "datasetx" is missing, with no default"

Code `rerddap::info(datasetid = "ncdcOisst21Agg_LonPM180", url = "https://coastwatch.pfeg.noaa.gov/erddap/")

OISST_sub_dl <- function(time_df){ OISST_dat <- griddap(x = "ncdcOisst21Agg_LonPM180", url = "https://coastwatch.pfeg.noaa.gov/erddap/", time = c(time_df$start, time_df$end), zlev = c(0, 0), latitude = c(1, 2), longitude = c(103, 104), fields = "sst")$data %>% mutate(time = as.Date(stringr::str_remove(time, "T00:00:00Z"))) %>% dplyr::rename(t = time, temp = sst) %>% select(lon, lat, t, temp) %>% na.omit() }

dl_years <- data.frame(date_index = 1:2, start = as.Date(c("2002-01-01", "2011-01-01" )), end = as.Date(c("2010-12-31", "2018-12-31" )))

system.time( OISST_data <- dl_years %>% group_by(date_index) %>% group_modify(~OISST_sub_dl(.x)) %>% ungroup() %>% select(lon, lat, t, temp) )`

Any idea what be going on?

Thanks

[rmendels]

@Ruben7c Please do the following commands:

 library(rerddap)
?griddap

and look at the present description of the function griddap().

[NikiDiogou]

@NikiDiogou please read the package vignette, section "Plotting crossing the dateline"

Thanks again, the map_frame function worked!

[NikiDiogou]

@rmendels, another quick question!

It has been difficult to download all the data I need in .nc files with these rerddap codes. Unless I am doing something wrong. R keeps crashing on me and gives me errors, since I am trying to convert them to data frames before I turn them into rasters and clip them in certain buffers.

Is there a way to download them in a series of daily .geotif formats? I have not seen this option in rerddap or rerddaprxtracto. Or in any other option through R.

[rmendels]

@NikiDiogou - from the get go you have been trying to download and use a very large amount of data, and then been surprised when there are issues. No matter what format you choose to download the data, once read into R the array size will be the same. So if you need to do that you either need to find a computer with more memory or think about what you are doing. Either way, this has nothing to do with rerddap, and I will not respond further here. If you want to discuss further contact me at my NOAA email