Cannot resolve host -- MOTU server issue?

[ericward-noaa]

Running the example in the vignette (below) I get the error message:

Already logged in
Preparing download...
Failed to collect information from Copernicus.
Could not resolve host: cmems-be.lobelia.earth

Code:

destination <- tempfile("copernicus", fileext = ".nc")

copernicus_download_motu(
  destination   = destination,
  product       = "GLOBAL_ANALYSISFORECAST_PHY_001_024",
  layer         = "cmems_mod_glo_phy-cur_anfc_0.083deg_P1D-m",
  variable      = "sea_water_velocity",
  output        = "netcdf",
  region        = c(-1, 50, 10, 55),
  timerange     = c("2021-01-01", "2021-01-02"),
  verticalrange = c(0, 2),
  sub_variables = c("uo", "vo")
)

[pepijn-devries]

The server seems to be offline at the moment, the URL (https://cmems-be.lobelia.earth) also won't open in my browser. This does not seem to be an issue with the package. Maybe you could try contacting the people at Copernicus (https://marine.copernicus.eu)

[pepijn-devries]

I was too soon with my response. Apparently Copernicus has migrated the server providing product details without a forwarding address. I'm working on a fix as we speak. I'm hoping to have it pushed to github by tomorrow

[pepijn-devries]

Done. It should be fixed in the latest version on github. Please install using devtools::install_github('pepijn-devries/CopernicusMarine') and let me know if this fixes the issue for you.

[ericward-noaa]

Great, thanks for the fast fix. I just tested the example in the helpfile, and it works as intended.

[ericward-noaa]

Actually I'm re-opening this. I have still run into issues accessing some of the data products that were accessible a couple weeks ago -- things like the 'GLOBAL_MULTIYEAR_PHY_001_030' product.

After getting some feedback from the folks that run Copernicus Marine, I think this is a larger issue. They are in the process of generating a new version of the Copernicus Marine Service, with new clients. The current release is in beta testing, and I'm able to get all the data products I need from it (using Python) -- but the R package will need to be updated (I am not sure whether MOTU will continue to serve the next generation of products)

[jatalah]

I am experiencing a similar issue with the latest Github version. It worked for the product IBI_ANALYSISFORECAST_PHY_005_001, but when attempting downloading MEDSEA_MULTIYEAR_PHY_006_004-TDS I get:

Already logged in Preparing download... Copernicus reported a client error (status 404). You may have requested information that is not available, please check your input. Error in copernicus_download_motu(destination = dest, product = "MEDSEA_MULTIYEAR_PHY_006_004-TDS", : No services available, please check if you specified the 'product' and 'layer' name correctly, and whether the layer has a MOTU service.

[pepijn-devries]

Thanks for notifying. I will have a look at this and see if I can reproduce the issue and provide a fix for this. It could take some more time.

[pepijn-devries]

Without a reprex it is hard to figure out what is going on. Note that renaming products or migration of servers by Copernicus is outside of my sphere of influence. It is always wise to document the DOI of the product you are using such that you can always recover it.

@ericward-noaa I've prepared the following reprex for your product which works without any problems

dest <- tempfile()
copernicus_download_motu(
  destination = dest,
  product = "GLOBAL_MULTIYEAR_PHY_001_030",
  layer = "cmems_mod_glo_phy_my_0.083_P1D-m",
  variable = "bottomT",
  region = c(0, 50, 1, 52),
  verticalrange = c(0.5, 1),
  timerange = c("2020-01-01", "2020-01-02")
)
#> Already logged in
#> Preparing download...
#> Downloading file...
#> Done

^{Created on 2023-11-09 with reprex v2.0.2}

Do you have a specific example that doesn't work?

@jatalah I can't find the product you refer to, do you have a DOI? I did find MEDSEA_MULTIYEAR_PHY_006_004 which works fine.

[jatalah]

Hi, yes that's the one. DOI I think I needed to delete the "-TDS" from the end of the product name provided in the MOTU API request to make it work. Thanks

[pepijn-devries]

@jatalah Note that only products (product_id) listed in copernicus_products_list() can be requested. I plan to update the documentation to make this more clear. Furthermore, I'm thinking of writing a graphical user interface using Shiny in an supplemental package to facilitate data discovery in R.

[pepijn-devries]

@ericward-noaa Please let me know if you can provide a reprex, such that I can reopen and investigate the issue. Otherwise, I'm assuming that the issue is ressolved.

[ericward-noaa]

Sure, here's a reproducible example that doesn't currently work

copernicus_download_motu(
  destination   = 'GLORYS-Monthly-SSH-1994.nc',
  product       = "GLOBAL_MULTIYEAR_PHY_001_030",
  layer         = "cmems_mod_glo_phy_my_0.083_P1M-m",
  variable      = "Sea surface height above geoid",
  output        = "netcdf",
  region        = c(-129,40,-116,48),
  timerange     = c("1994-01-01", "1994-12-31"),
  verticalrange = c(0, 100),
  sub_variables = c("zos"),
  overwrite=TRUE
)

I get

Already logged in
Preparing download...
Downloading file...
Failed to collect information from Copernicus.
Could not resolve host: NA

But I can access the same data via Python and the new beta release of Copernicus

[pepijn-devries]

Hi @ericward-noaa ,

This is strange, I cannot reproduce. From the messages you get, it seems that the download is being prepared, but you don't get a working download URL (given the Could not resolve host: NA). For me, your code works without problems. Does this only happen for this particular product+variable or does this happen on your system for all products?

What happens if you specify zos directly as variable (for me both implemenations work):

copernicus_download_motu(
  destination   = 'GLORYS-Monthly-SSH-1994.nc',
  product       = "GLOBAL_MULTIYEAR_PHY_001_030",
  layer         = "cmems_mod_glo_phy_my_0.083_P1M-m",
  variable      = "zos",
  output        = "netcdf",
  region        = c(-129,40,-116,48),
  timerange     = c("1994-01-01", "1994-12-31"),
  verticalrange = c(0, 100),
  overwrite=TRUE
)

Furthermore, could you send me your sessionInfo()? That might help to deduce the problem. Thanks in advance!

[ericward-noaa]

Specifying zos directly as a variable seemed to solve the problem, thanks!

My sessionInfo is below -- this same code was also not working for colleagues on a PC.

R version 4.3.1 (2023-06-16) Platform: x86_64-apple-darwin20 (64-bit) Running under: macOS Monterey 12.6.7

Matrix products: default BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib LAPACK: /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/lib/libRlapack.dylib; LAPACK version 3.11.0

[pepijn-devries]

Ah thanks. I suspect that this is because there are spaces in the variable name that are not escaped by the package. This issue seems to be platform specific as it occurs on MacOS and not on Windows. I will adjust my code such that variable names are escaped before concatenating them into an URL. Hopefully, this will solve the issue. For now you can use the approach listed above as a workaround

[jatalah]

Hi, I am having problems downloading the MEDSEA_MULTIYEAR_PHY_006_004 data. It gets stuck on:

Already logged in Preparing download...

Any ideas?

copernicus_download_motu(
  destination = tempfile(),
  product = "MEDSEA_MULTIYEAR_PHY_006_004",
  layer = "med-cmcc-tem-rean-d",
  variable = "thetao",
  output = "netcdf",
  region = c(-0.5, 37, 1, 38),
  timerange = c("2022-03-01", "2022-03-11"),
  verticalrange = c(0, 10),
  overwrite = TRUE
)

[pepijn-devries]

@jatalah with your code I get:

Logging in onto MOTU server...
Preparing download...
010-6 : The date range is invalid. fr.cls.atoll.motu.web.bll.exception.MotuInvalidDateRangeException: Invalid date range: [2022-03-01 00:00:00,2022-03-11 00:00:00]. Valid range is: [1987-01-01 12:00:00,2021-06-30 12:00:00].

You are requesting a date range that is not valid. If you select a valid date range, your code will work.