bluegreen-labs / ecmwfr

Interface to the public ECMWF API Web Services
https://bluegreen-labs.github.io/ecmwfr/
Other
106 stars 30 forks source link

Error when requesting data #117

Closed pedroalencar1 closed 1 year ago

pedroalencar1 commented 1 year ago

Hi, I am just getting started with ecmwfr and maybe this is a rather basic issue.

I have the following request:

request <- list(
  class   = "od",
  date    = "1985-01-01",
  expver  = "1",
  levtype = "sfc",
  param   = "167.128",
  step    = "12/24/36/48/60/72/84/96/108/120/132/144/156/168/180/192/204/216/228/240",
  stream  = "oper",
  time    = "12:00:00",
  type    = "fc",
  target  = "output"
)

Which should return the 2m temperature forecast from 1st of Jan. 1985 for the following 10 days. When I run

wf_request(request = request, user = "myemailadress")

I get no results and a single message "Error:" with no further information about the error. Is there something that I am missing out?

Many thanks for the help and great work.

Cheers

khufkens commented 1 year ago

Most common errors are badly formatted requests (use the addin, don't format by hand!). Unit checks still pass so check your request first (e.g. there is no format specified in your target make it: output.nc - or reflect the expected output format / zip file if so).

Note, this is a WebAPI request see #114 , this service will be shut down as of June. If you don't want to rewrite your workflow and ensure replicability I suggest to move to CDS right now.

pedroalencar1 commented 1 year ago

Thanks! I fixed my request and added some filed that are not automatically generated by MARS. I managed to get the access to the data, but every so often I get an error: Error in self$update_status() : attempt to apply non-function

e.g.: When I tried to run this example that is in the README

my_request <- list(
  stream = "oper",
  levtype = "sfc",
  param = "165.128/166.128/167.128",
  dataset = "interim",
  step = "0",
  grid = "0.75/0.75",
  time = "00/06/12/18",
  date = "2014-07-01/to/2014-07-31",
  type = "an",
  class = "ei",
  area = "73.5/-27/33/45",
  format = "netcdf",
  target = "tmp.nc"
)

wf_request(
  user = "pedro.alencar@campus.tu-berlin.de",
  request = my_request,
  transfer = TRUE,
  path = "~")
khufkens commented 1 year ago

Just for clarification, do you get the specified error each time on the query above or only intermittent?

pedroalencar1 commented 1 year ago

I always get the error in this query (and other).

On the other hand, this here works fine but I can't see the difference

my_request <- list(stream = "oper",
                levtype = "sfc",
                param = "168.128",
                dataset = "interim",
                step = "120",
                grid = "0.05/0.05",
                time = "00",
                date = "2014-07-01/to/2014-07-04",
                type = "an",
                class = "ei",
                area = "52/13/53/14",
                format = "netcdf",
                target = "tmp.nc")

wf_request(
  user = "pedro.alencar@campus.tu-berlin.de",
  request = my_request,
  transfer = TRUE,
  path = "~")
khufkens commented 1 year ago

The only difference I see is in the order of the bounding box coordinates of area.

Try this in the README request:

area = "33/-27/73.5/45"
khufkens commented 1 year ago

Irrespective, the error should be trapped but seemingly isn't.

pedroalencar1 commented 1 year ago

Indeed, it still doesn't work. It does work if I request a single parameter at each time.

However, if I try to get forecast data it never works (and here maybe I am missing something, like the dataset?)


my_request <- list(
  class   = "od",
  date    = "2023-03-01",
  expver  = "1",
  levtype = "sfc",
  param   = "228.128",
  grid    = "0.25/0.25",
  step    = "0",
  stream  = "oper",
  time    = "00:00:00",
  type    = "fc",
  area    = "52/13/53/13",
  format  = "netcdf",
  target  = "output.nc"
)

wf_request(
  user = "pedro.alencar@campus.tu-berlin.de",
  request = my_request,
  transfer = TRUE,
  path = "~")
khufkens commented 1 year ago

param defines the dataset. Now, some of the products have restricted access, especially forecasts. Do you have access to the data itself?

pedroalencar1 commented 1 year ago

I see, this might be the problem for forecasts then.

To summarise: to download era interim, I manage to download reliably, but one variable at each time. For forecasts, I probably lack the permissions (indeed I think that is the case).

When I check it here: https://www.ecmwf.int/en/forecasts/dataset/operational-archive I get a locker in the MARS data and can only see the archive. I thought I could download data in the archive, but it is likely not true. Screenshot 2023-03-13 at 13 44 29

Thank you so much for your support!

If it's not an imposition, could you tell me how to request access to the forecast data?

khufkens commented 1 year ago

For academic licenses see bottom left here: https://www.ecmwf.int/en/forecasts/accessing-forecasts

Note that the WebAPI service will close, so you will have to move to CDS. Not sure how and in what modalities they will move.

pedroalencar1 commented 1 year ago

Thanks again!

I already experimented ecmwfr for cds to download ERA5 data and it was fine (maybe a bit slow?).

I'll try and get the proper licenses to access the data. Once again, thanks for the great help.

khufkens commented 1 year ago

Closing this for now.