x1o / moexer

A thin wrapper around MOEX ISS REST interface
GNU General Public License v3.0
2 stars 1 forks source link

error "414 Request-URI Too Long" whet getting the full list of securities #4

Open sash-kan opened 2 months ago

sash-kan commented 2 months ago

по примеру из документации попробовал получить полный список ценных бумаг с помощью запроса fetching_fully(query_iss)('securities').

но после нескольких минут работы (или ожидания?) сервер вернул ошибку http 414 (вероятно, "Request-URI Too Long").

может быть, я неправильно использовал декоратор fetching_fully?

> sec <- fetching_fully(query_iss)('securities')
No encoding supplied: defaulting to UTF-8.
Error in `map2()`:
ℹ In index: 1.
ℹ With name: securities.
Caused by error:
! lexical error: invalid char in json text.
                                       <html>  <head><title>414 Reques
                     (right here) ------^
Run `rlang::last_trace()` to see where the error occurred.
> rlang::last_trace()
<error/purrr_error_indexed>
Error in `map2()`:
ℹ In index: 1.
ℹ With name: securities.
Caused by error:
! lexical error: invalid char in json text.
                                       <html>  <head><title>414 Reques
                     (right here) ------^
---
Backtrace:
     ▆
  1. └─fetching_fully(query_iss)("securities")
  2.   └─purrr::imap(...)
  3.     └─purrr::map2(.x, vec_index(.x), .f, ...)
  4.       └─purrr:::map2_("list", .x, .y, .f, ..., .progress = .progress)
  5.         ├─purrr:::with_indexed_errors(...)
  6.         │ └─base::withCallingHandlers(...)
  7.         ├─purrr:::call_with_cleanup(...)
  8.         └─moexer (local) .f(.x[[i]], .y[[i]], ...)
  9.           ├─base::do.call(query_iss_fn, c(fn_params, list(params = query_params)))
 10.           └─moexer (local) `<fn>`("securities", params = `<named list>`)
 11.             ├─purrr::map(...)
 12.             │ └─purrr:::map_("list", .x, .f, ..., .progress = .progress)
 13.             │   └─purrr:::vctrs_vec_compat(.x, .purrr_user_env)
 14.             └─jsonlite::fromJSON(...)
 15.               └─jsonlite:::parse_and_simplify(...)
 16.                 └─jsonlite:::parseJSON(txt, bigint_as_char)
 17.                   └─jsonlite:::parse_string(txt, bigint_as_char)
Run rlang::last_trace(drop = FALSE) to see 4 hidden frames.
> rlang::last_trace(drop = FALSE)
<error/purrr_error_indexed>
Error in `map2()`:
ℹ In index: 1.
ℹ With name: securities.
Caused by error:
! lexical error: invalid char in json text.
                                       <html>  <head><title>414 Reques
                     (right here) ------^
---
Backtrace:
     ▆
  1. ├─fetching_fully(query_iss)("securities")
  2. │ └─purrr::imap(...)
  3. │   └─purrr::map2(.x, vec_index(.x), .f, ...)
  4. │     └─purrr:::map2_("list", .x, .y, .f, ..., .progress = .progress)
  5. │       ├─purrr:::with_indexed_errors(...)
  6. │       │ └─base::withCallingHandlers(...)
  7. │       ├─purrr:::call_with_cleanup(...)
  8. │       └─moexer (local) .f(.x[[i]], .y[[i]], ...)
  9. │         ├─base::do.call(query_iss_fn, c(fn_params, list(params = query_params)))
 10. │         └─moexer (local) `<fn>`("securities", params = `<named list>`)
 11. │           ├─purrr::map(...)
 12. │           │ └─purrr:::map_("list", .x, .f, ..., .progress = .progress)
 13. │           │   └─purrr:::vctrs_vec_compat(.x, .purrr_user_env)
 14. │           └─jsonlite::fromJSON(...)
 15. │             └─jsonlite:::parse_and_simplify(...)
 16. │               └─jsonlite:::parseJSON(txt, bigint_as_char)
 17. │                 └─jsonlite:::parse_string(txt, bigint_as_char)
 18. └─base::.handleSimpleError(...)
 19.   └─purrr (local) h(simpleError(msg, call))
 20.     └─cli::cli_abort(...)
 21.       └─rlang::abort(...)
sash-kan commented 2 months ago

почему url слишком длинный - становится понятно при запуске с debug_output = TRUE:

> sec <- fetching_fully(query_iss)('securities',debug_output = TRUE)
http://iss.moex.com/iss/securities.json
http://iss.moex.com/iss/securities.json?iss.only=securities&start=100
http://iss.moex.com/iss/securities.json?iss.only=securities&start=100&iss.only=securities&start=200
http://iss.moex.com/iss/securities.json?iss.only=securities&start=100&iss.only=securities&start=200&iss.only=securities&start=300
http://iss.moex.com/iss/securities.json?iss.only=securities&start=100&iss.only=securities&start=200&iss.only=securities&start=300&iss.only=securities&start=400
http://iss.moex.com/iss/securities.json?iss.only=securities&start=100&iss.only=securities&start=200&iss.only=securities&start=300&iss.only=securities&start=400&iss.only=securities&start=500
...

а как же избежать такого дублирования http-параметров?