expersso / OECD

Reproducible and programmatic access to OECD data
126 stars 20 forks source link

Issue when importind data with get_dataset using filter #26

Open Nerd1ngxG opened 1 year ago

Nerd1ngxG commented 1 year ago

I get the following error: "Bad Request (HTTP 400).Error in rsdmx::readSDMX(url) : HTTP request failed with status: 400" while trying to import data through get_dataset() function and filtering before the data using filter and a simple list with just one member. I used the following code

dataset = "TABLE2A"

filter_list <- list(c("20001")) df <- get_dataset(dataset = dataset, filter = filter_list)

Bad Request (HTTP 400).Error in rsdmx::readSDMX(url) : HTTP request failed with status: 400

luifrancgom commented 1 year ago

Hello Nerd1ngxG

Please check out the data structure of the data set using:

OECD::get_data_structure(dataset = "TABLE2A") |> str()
data_set_structure <- OECD::get_data_structure(dataset = "TABLE2A")

data_set_structure$VAR_DESC |> tibble::as_tibble()
data_set_structure$RECIPIENT |> tibble::as_tibble()
data_set_structure$DONOR |> tibble::as_tibble()
data_set_structure$PART |> tibble::as_tibble()
data_set_structure$AIDTYPE |> tibble::as_tibble()
data_set_structure$DATATYPE |> tibble::as_tibble()

The variable filter_list that you create don't have the necessary dimensions to apply the filter expression to create a data query. If you want to see the necessary variables check the following link

For example this will work:

OECD::get_dataset(dataset = "TABLE2A", 
            filter = "10100.20001", 
            start_time = 2021, end_time = 2021,
            pre_formatted = TRUE)

Or in an equivalent way

new_filter_list <- list(c("10100"),
                        c("20001"))

OECD::get_dataset(dataset = "TABLE2A", 
                  filter = new_filter_list, 
                  start_time = 2021, end_time = 2021,
                  pre_formatted = FALSE)

Where you are getting the following information:

for the year 2021.

Also an alternative solution mention in #14 can be:

OECD::get_dataset(dataset = "TABLE2A", 
                  filter = ".20001", 
                  start_time = 2021, end_time = 2021,
                  pre_formatted = TRUE)

by leaving the first dimension empty in the query or in an equivalent way:

new_filter_list_2 <- list(c(""),
                        c("20001"))

OECD::get_dataset(dataset = "TABLE2A", 
                  filter = new_filter_list_2, 
                  start_time = 2021, end_time = 2021,
                  pre_formatted = FALSE)

Finally as it is mentioned in #6 a possible enhacement in the future will be to specify in your case OECD::get_dataset(dataset = "TABLE2A", DONOR = "20001") but this is not yet develop by the author.

I hope this will help you.