sboysel / fredr

An R client for the Federal Reserve Economic Data (FRED) API
https://sboysel.github.io/fredr/
Other
92 stars 21 forks source link

Trouble downloading multiple FRED series #87

Closed Islascuevas closed 3 years ago

Islascuevas commented 3 years ago

Hi! I'm having an issue when I try to download multiple series.

The following error appears:

Request failed [504]. Retrying in 1.7 seconds... Request failed [504]. Retrying in 1.8 seconds... Error: 404: Not Found. The series does not exist. Call rlang::last_error() to see a backtrace

I'm using this code to download multiple series:

fredr_set_key("9a0529b046b504306865a494d1ce7a6b")

params <- list( series_id = c("ICSA", "IC4WSA","PAYEMS", "NPPTTL", "UNRATE", "LNS12032194", "LNS13023706", "LNS12032194", "AWHAETP", "CEU3000000002", "CSUSHPINSA", "PERMIT", "HOUST", "HOUST1F", "HOUST5F", "HSN1F", "EXHOSLUSM495S", "NHSDPTS", "MORTGAGE30US", "MRTSSM44X72USS", "RSFSXMV","ECOMSA", "UMCSENT", "PSAVERT", "HNONWPDPI", "TOTALSA", "INDPRO", "IPMAN", "DGORDER", "NEWORDER", "CEFDFSA066MSFRBPHI", "ISRATIO", "BOPTEXP", "BOPTIMP", "TWEXB", "CPROFIT", "PNFI", "PPIACO", "PCUOMFGOMFG", "CPILFESL", "PCEPI", "PCEPILFE", "PCETRIM1M158SFRBDAL", "T5YIFR", "CES0500000003", "IREXFUELS", "W875RX1", "PCE", "SP500", "CPIAUCSL", "USD3MTD156N", "FEDFUNDS", "GDPC1", "PCEC96", "FPIC1", "EXPGSC1", "IMPGSC1", "GCEC1", "A261RX1Q020SBEA"), frequency = c("m", "m","m", "m", "m", "m", "m", "m", "m", "m", "m", "m", "m", "m", "m", "m", "m", "m", "m", "m", "m", "q", "m", "m","q", "m", "m", "m", "m", "m", "m", "m", "m", "m", "m", "q", "q", "m", "m", "m", "m", "m", "m", "m", "m", "m", "m", "m", "m", "m", "m", "m", "q", "q", "q", "q", "q", "q", "q") )

fred.data <- pmap_dfr( .l= params, .f = ~ fredr(series_id = .x, frequency = .y))

Islascuevas commented 3 years ago

I have already used the rlang:: last_error() and the rlang:: last_trace() with this results:

> rlang::last_error()
<error>
message: 404: Not Found.  The series does not exist.
class:   `rlang_error`
backtrace:
 1. purrr::pmap_dfr(.l = params, .f = ~fredr(series_id = .x, frequency = .y))
 2. purrr::pmap(.l, .f, ...)
 3. global::.f(...)
 4. fredr::fredr(series_id = .x, frequency = .y)
Call `rlang::last_trace()` to see the full backtrace
> rlang::last_trace()
    x
 1. \-purrr::pmap_dfr(.l = params, .f = ~fredr(series_id = .x, frequency = .y))
 2.   \-purrr::pmap(.l, .f, ...)
 3.     \-global::.f(...)
 4.       \-fredr::fredr(series_id = .x, frequency = .y)
 5.         +-base::do.call(fredr_request, c(fredr_args, user_args))
 6.         \-(function (endpoint, ..., to_frame = TRUE, print_req = FALSE, ...
sboysel commented 3 years ago

Unfortunately this backtrace given here doesn't really explain which ID fredr() returns the 404 for. Can you try just writing a simple for loop for your list of series IDs? Then you can identify which IDs are causing the error.

list_of_ids <- c("ICSA", "IC4WSA","PAYEMS", "NPPTTL", "UNRATE", "LNS12032194", "LNS13023706",
  "LNS12032194", "AWHAETP", "CEU3000000002", "CSUSHPINSA", "PERMIT", "HOUST",  "HOUST1F", 
  "HOUST5F", "HSN1F", "EXHOSLUSM495S", "NHSDPTS", "MORTGAGE30US",  "MRTSSM44X72USS", 
  "RSFSXMV","ECOMSA", "UMCSENT", "PSAVERT", "HNONWPDPI", "TOTALSA",  "INDPRO", "IPMAN", 
  "DGORDER", "NEWORDER", "CEFDFSA066MSFRBPHI", "ISRATIO", "BOPTEXP", "BOPTIMP", "TWEXB", 
  "CPROFIT", "PNFI", "PPIACO", "PCUOMFGOMFG", "CPILFESL", "PCEPI", "PCEPILFE", "PCETRIM1M158SFRBDAL", 
  "T5YIFR", "CES0500000003", "IREXFUELS", "W875RX1", "PCE", "SP500", "CPIAUCSL", "USD3MTD156N", 
  "FEDFUNDS", "GDPC1", "PCEC96", "FPIC1", "EXPGSC1", "IMPGSC1", "GCEC1", "A261RX1Q020SBEA")

for (i in list_of_ids) {
    fredr_series(series_id = i)
}
Islascuevas commented 3 years ago

Thanks, I've run the loop, and apparently, the problem is the LIBOR image

DavisVaughan commented 3 years ago

Likely this restriction https://github.com/sboysel/fredr#restrictions

Islascuevas commented 3 years ago

Likely this restriction https://github.com/sboysel/fredr#restrictions

Thanks, that was the whole problem. Keep the good work guys!