MattCowgill / readabs

Download and tidy time series data from the Australian Bureau of Statistics in R
https://mattcowgill.github.io/readabs/
Other
101 stars 22 forks source link

Add Household Spending Indicator function #194

Open AEld2602 opened 2 years ago

AEld2602 commented 2 years ago

Hi,

Is there anyway to download data that doesn't appear to be listed in the ABS Time Series Directory concordance. For example "5682.0 Monthly Household Spending Indicator" doesn't seem to be in the concordance table in the directory and readabs is giving me an error.

Can I pass the function an url or something?

Thanks

Adam

MattCowgill commented 2 years ago

Hi @AEld2602, To my eternal frustration, the ABS sometimes decides not to release time series data using their standard time series spreadsheets and the Time Series Directory. This makes it hard to work with programmatically. The read_abs() function does not work with data that is not in the standard format and not in the TSD.

I will add a function to get this data specifically, like read_payrolls() for the ABS Weekly Payroll Jobs data.

In the meantime, you can download the data with download_abs_data_cube(). You will need to manually wrangle it. Here is an example:

library(readabs)
library(tidyverse)

download_abs_data_cube("monthly-household-spending-indicator",
                       "National") %>% 
  readxl::read_excel("Data 1",
                     skip = 4) %>% 
  janitor::clean_names() %>% 
  filter(!is.na(food)) %>% 
  rename(date = x1) %>% 
  mutate(date = janitor::excel_numeric_to_date(as.numeric(date)))
AEld2602 commented 2 years ago

Thanks Matt!

"monthly-household-spending-indicator" doesn't seem to be in the show_available_catalogues() list. Is it a matter of just manually adding this to here?

The more general problem I am hoping to solve is some small tweaks to available functions in readabs that allow for download (not necessarily tidy, although it would be nice) any ABS data.

Thanks again for your help!

MattCowgill commented 2 years ago

If you run show_available_catalogues(refresh = TRUE) the returned vector will include "monthly-household-spending-indicator". Alternatively, if you update to the current version (0.4.12, now on both GitHub and CRAN) then show_available_catalogues() will include "monthly-household-spending-indicator".

The download_abs_data_cube() function already works to download any ABS spreadsheet. Please see example in my previous comment.

I would love to have a function that would import a tidy version of any ABS spreadsheet. However, that is not a trivial task. The time series spreadsheets that can be imported using read_abs() are all formatted in a consistent way. Other spreadsheets are formatted in myriad different ways - you would almost need bespoke code for each spreadsheet, which would then break if the format changed. I have done this for the ABS Weekly Payrolls data (read_payrolls()) and plan to do it for the monthly household spending indicator too. In the meantime you can use the code I posted above.