how to get the complete data for the whole forecast period

[shuang816]

Hello, I want to download the "Met Office Global Atmospheric Hi-Res Model - 10km Deterministic" dataset, and I don't subscribe the AWS SNS yet. Instead, I got the keys on the bucket: https://s3.eu-west-2.amazonaws.com/aws-earth-mo-atmospheric-mogreps-g-prd, then according to the bucket and keys. I have downloaded the data every hour by URL(bucket+key) for a week, but today when I analyze the data I downloaded, I found that the dataset is not complete, for certain variable, such as "wind speed at 10m", I only got two files that forecasted at 2018-11-18 12:00, the time is not consequent for one forecasting reference time, I was wondering that have you uploaded all or part of the keys on "https://s3.eu-west-2.amazonaws.com/aws-earth-mo-atmospheric-mogreps-g-prd", and how can I get the complete hourly data for the whole forecast period?

Thanks He Shuangshuang

[TomDufall]

Can you give me an example of a key that had the data you wanted? I may be able to look up the forecast name and find you some keys for it. I'm only involved in the technical side, so I don't know how often they should be available. In general, it's recommended to subscribe to the SNS feed as it sends out notifications that give you info such as the forecast name, forecast reference time, forecast period, and key, which makes it easier to filter out just the forecasts you want to download.

[shuang816]

@TomDufall Thanks for your kindly reply. I download more than 1700 files, and then I read the variable name, time, dimension, forecast_reference_time from files, and output the information as a CSV file, partially shown as below: Example: key | var | dim | forecast_reference_time | time 00ea0d41d2e8a21dfc241f1b15503fa8ea719fce.nc | wind_speed | latitude: 1920; longitude: 2560 | [2019-11-18 00:00:00] | [2019-11-24 00:00:00] 0185d9707d8b4d8d8e2037abadcf6a43717229fe.nc | wind_speed | latitude: 1920; longitude: 2560 | [2019-11-18 00:00:00] | [2019-11-20 15:00:00] So, the forecast reference time is 2019-11-18 00:00:00, I want to get all of the files for windspeed or other variables in the forecast period, I guess it should be at least 48 hours, but I only get two files. That's the problem I met. Thanks again.

[TomDufall]

I just re-read your first message and realised you're after the global deterministic forecasts, but you're using the ensemble (MOGREPS) bucket. The 10km global det bucket is aws-earth-mo-atmospheric-global-prd

[shuang816]

I'm sorry I posted the wrong bucket in my first comment, I used the right bucket in my python script.