skgrange / saqgetr

Import Air Quality Monitoring Data in a Fast and Easy Way
GNU General Public License v3.0
9 stars 3 forks source link

Can't download BC data from 2008 to 2012, 2019 and part of 2018 from 31 BC monitoring sites. #6

Closed t0270293 closed 3 years ago

t0270293 commented 3 years ago

Hi Dr. Grange,

By using the get_saq_observations() function, I could manage to download BC data from 31 BC monitoring sites (including those closed ones) between 2013 and 2017. However, as I was trying to download BC data between 2008 and 2012, 2018 to 2019; only part of 2018 could be downloaded. Please see the following code:

BC_Data <- get_saq_observations( site = c("gb1044a","gb1028a","gb0048r","gb1067a","gb1097a","gb1055r","gb0620a", "gb0682a","gb0886a","gb0567a","gb1023a","gb0723a","gb0934a","gb0580a", "gb0960a","gb0851a","gb0105a","gb0146a","gb0991a","gb0839a","gb0931a", "gb0641a","gb0036r","gb0706a","gb0613a","gb0995a","gb1059a","gb0182a", "gb0658a","gb0234a","gb0135a"), start = 2011, end = 2012, variable = "bc", verbose = TRUE ) %>% saq_clean_observations (summary = " hour " , valid_only = TRUE, spread = TRUE ) %>% arrange(site)**

Is there anything I have done wrong?

Thank you so much in advance.

Sam

t0270293 commented 3 years ago

Sorry, here's the error message when I set 'start = 2011, end =2012':

Error: arrange() failed at implicit mutate() step.

Thanks a lot.

Sam

skgrange commented 3 years ago

Hello Sam, I do not think I can help out much on this question. I have had a look the process table, and it shows that black carbon observations are not available for the 2011--2012 time period for your sites of interest. An extract from the process table is here:

| process|site    |site_name                  |variable |data_source |date_start          |date_end            |
|-------:|:-------|:--------------------------|:--------|:-----------|:-------------------|:-------------------|
|  130695|gb0036r |Harwell                    |bc       |aqer:e1a    |2013-01-03 12:00:00 |2015-12-31 23:00:00 |
|  130878|gb0048r |Auchencorth Moss           |bc       |aqer:e1a    |2013-01-01 00:00:00 |2018-12-31 23:00:00 |
|  131058|gb0146a |Cardiff 12                 |bc       |aqer:e1a    |2013-01-01 00:00:00 |2014-06-09 08:00:00 |
|  131059|gb0146a |Cardiff 12                 |bc       |aqer:e1a    |2013-01-01 00:00:00 |2013-12-31 23:00:00 |
|  131128|gb0567a |Belfast Centre             |bc       |aqer:e1a    |2013-01-01 00:00:00 |2018-12-31 23:00:00 |
|  131175|gb0580a |Cardiff Centre             |bc       |aqer:e1a    |2015-03-13 11:00:00 |2018-12-31 23:00:00 |
|  131306|gb0620a |London N. Kensington       |bc       |aqer:e1a    |2013-01-01 00:00:00 |2018-12-31 23:00:00 |
|  131465|gb0682a |London Marylebone Road     |bc       |aqer:e1a    |2013-01-01 00:00:00 |2018-12-31 23:00:00 |
|  131601|gb0723a |Strabane 2                 |bc       |aqer:e1a    |2013-01-01 00:00:00 |2018-12-31 23:00:00 |
|  131901|gb0851a |Birmingham Tyburn          |bc       |aqer:e1a    |2013-01-01 00:00:00 |2017-02-28 04:00:00 |
|  132063|gb0886a |Detling                    |bc       |aqer:e1a    |2013-01-01 00:00:00 |2018-12-31 23:00:00 |
|  132190|gb0934a |Ballymena Ballykeel        |bc       |aqer:e1a    |2013-01-01 00:00:00 |2018-12-31 23:00:00 |
|  132337|gb0960a |Birmingham Tyburn Roadside |bc       |aqer:e1a    |2013-01-01 00:00:00 |2016-09-07 04:00:00 |
|  132415|gb0995a |Norwich Lakenfields        |bc       |aqer:e1a    |2013-01-01 00:00:00 |2013-05-14 04:00:00 |
|  132568|gb1023a |Kilmakee Leisure Centre    |bc       |aqer:e1a    |2013-01-02 16:00:00 |2018-12-31 23:00:00 |
|  132621|gb1028a |Glasgow Townhead           |bc       |aqer:e1a    |2013-12-01 00:00:00 |2018-12-31 23:00:00 |
|  132702|gb1044a |Glasgow High Street        |bc       |aqer:e1a    |2015-03-04 16:00:00 |2018-12-31 23:00:00 |
|  132753|gb1055r |Chilbolton Observatory     |bc       |aqer:e1a    |2016-01-14 14:00:00 |2018-12-31 23:00:00 |
|  132920|gb1067a |Birmingham A4540 Roadside  |bc       |aqer:e1a    |2016-10-05 11:00:00 |2018-12-31 23:00:00 |

This means that the British reporting authorities have not submitted BC data for your period of interest and will have to be gained elsewhere, if they in fact exist.

On your second point regarding the error, this occurs because if no observations are available, an empty tibble with zero rows and columns is returned. Therefore, when you try to arrange by site, there is no site variable in the tibble and the function will error. Sorry I cannot be more help. Stuart.

t0270293 commented 3 years ago

Hi Dr. Grange,

Thanks for your reply.

I see. Yes, the BC data from 2008 do exist and can be downloaded directly from the Defra AURN website. So it's quite weird that they cannot be downloaded by the get_saq_observations() function. Anyway, thanks again for this useful package and data can be obtained effectively. Seems like I need to download the rest of them manually through the website, that's sad. :)

Thanks and regards,

Sam

skgrange commented 3 years ago

I'm glad you were able to get your data in the end. I can have a look at including another data source, but this will have to wait a while because it is not a priority for my current research. Good luck with your research!