USGS EROS API 1.4.0 expired - requests invalid

[jakobjassmann]

The USGS EROS API 1.4.0 has expired and all requests by getSpatialData to the USGS EROS inventory are now invalid. The API has now been replaced by M2M API version 1.5.0. Documentation and migration guidelines can be found here: https://m2m.cr.usgs.gov/api/docs/json/#section-migration

We are using getSpatialData in one of our projects and it has been a real big help so far! Thanks for the hard work that has already gone into this.

@16EAGLE Are you planning on integrating the new EROS API?

No worries if not. I would also be more than happy to facilitate the transfer of the code to achieve compatibility with the new API. However, for this it would be good to know whether there are long term plans to maintain the package. I noticed there has been little activity from the project maintenance team recently and a few pull requests are still outstanding. Are you intending to continue development of this package?

[jakobjassmann]

Here is a bit more detail...

The new URL for the API access is: "https://m2m.cr.usgs.gov/api/api/json/stable/".

All requests now need to be made as POST request with a JSON body, the authentication key needs to in the X-Auth-Token header rather than in the main body and all requests need to carry a User-Agent header. There are a few changes in the endpoints, detailed here, but overall it seems fairly straight forward.

For example, here is a quick suggestion on how to make the POST request in the login function complaint with the new API:

.ERS_login <- function(username, password, n_retry = 3){
  x <- POST(url = paste0(getOption("gSD.api")$ee, "login"),
            body = list(username = username, password = password),
            encode = "json",
            user_agent("httr"))
  stop_for_status(x, "connect to server.")
  warn_for_status(x)
  v <- content(x)$data
  if(is.null(v)) out("Login failed. Please retry later or call services() to check if USGS services are currently unavailable.", type = 3)
  return(v)
}

[16EAGLE]

Thank you for raising the issue and providing an example. I will work on integrating this change ASAP to restore the EROS functionalities and am happy for your contributions in case you are still interested!

Regarding your question: I definitely want to keep getSpatialData alive though I had too much to do over the last 6-8 months to be able to allocate enough time to ensure appropriate maintenance. I cannot guarantee quick fixes at all times at the moments but am hoping that this is just a phase and that I get to the point again where I have time to integrate new sources and products.

Any contributions in the form of bug reports, suggestions or direct solutions as PRs are always very much welcome.

---

Edit (2021-05-26, 11:22): Functions that are migrated to v1.5.0 by now (soon to be pushed): login_USGS, get_products

[16EAGLE]

The EROS API v.1.5.0 client is implemented now. Here a little reprex for MODIS:

library(getSpatialData)
login_USGS(username = Sys.getenv("gSD_user"), password = Sys.getenv("gSD_pass"))
login_earthdata(username = Sys.getenv("gSD_user"), password = Sys.getenv("gSD_pass"))

# get products
products <- get_products(update_online = T)

# session settings
set_archive(dir_archive = "/my/dir")
set_aoi(aoi_data[[1]])

# get records
records <- get_records(
                       time_range = c("2019-05-15", "2019-05-25"),
                       products = c("modis_mod13a1_v6", "modis_myd13q1_v6"))
records <- records[c(1,5),]

# view previews and footprints
records <- get_previews(records)
view_previews(records)
view_records(records)

# check availability, order if needed
records <- check_availability(records)
records <- order_data(records, wait_to_complete = T)
records <- get_data(records)

Working on fixing problems with individual products (see #91).

[FReddig]

Hi Jakob, could you help me out? I still can't log in into the EROS API (login_USGS).

Error message: "Error in .post(url = paste0(getOption("gSD.api")$ee, "login"), username = NULL, : Not Found (HTTP 404). Failed to connect to server.."

Login outside of R is working fine. What might be the issue here?

Thanks in advance, Fabian

[s-young8]

Hi FReddig, have you found a way to make it work? It worked once for me and then invalid since then.

Thanks, Sherry

[AmoxiDylan]

Just wanting to bump this thread. Experiencing the same issue as described. Thanks 16EAGLE

[henrykironde]

Any updates on this issue?

[Aviankind]

Hi, for the HTTP 404 or 500 error: do you also get an error on the test page at https://m2m.cr.usgs.gov/ ? If yes, to be able to use the USGS API you need to head to your profile at https://ers.cr.usgs.gov/ , choose "Access request", click "Request Access", select access type machine and fill out the form. Takes about a week to be activated. HTH