stephhazlitt
commented
1 year ago This is due to the default setting of download_audience = "Public" in the bcdata::bcdc_search() function. The records above that were not surfaced with bcdata::bcdc_search("Metadata for ICBC") are all download_audience = "Not downloadable" so they are excluded from the query.
Unfortunately, bcdata::bcdc_search("Metadata for ICBC", download_audience = "Not downloadable") does not work.
bcdata::bcdc_search("Metadata for ICBC",`
download_audience = "Not downloadable")
#> List of B.C. Data Catalogue Records
#> Number of records: 0
#> Titles:
#> Error in `map2()`:
#> ℹ In index: 1.
#> Caused by error in `x[[.y]]`:
#> ! subscript out of bounds
#> Backtrace:
#> ▆
#> 1. ├─base::tryCatch(...)
#> 2. │ └─base (local) tryCatchList(expr, classes, parentenv, handlers)
#> 3. │ ├─base (local) tryCatchOne(...)
#> 4. │ │ └─base (local) doTryCatch(return(expr), name, parentenv, handler)
#> 5. │ └─base (local) tryCatchList(expr, names[-nh], parentenv, handlers[-nh])
#> 6. │ └─base (local) tryCatchOne(expr, names, parentenv, handlers[[1L]])
#> 7. │ └─base (local) doTryCatch(return(expr), name, parentenv, handler)
#> 8. ├─base::withCallingHandlers(...)
#> 9. ├─base::saveRDS(...)
#> 10. ├─base::do.call(...)
#> 11. ├─base (local) `<fn>`(...)
#> 12. ├─global `<fn>`(input = base::quote("bad-grub_reprex.R"))
#> 13. │ └─rmarkdown::render(input, quiet = TRUE, envir = globalenv(), encoding = "UTF-8")
#> 14. │ └─knitr::knit(knit_input, knit_output, envir = envir, quiet = quiet)
#> 15. │ └─knitr:::process_file(text, output)
#> 16. │ ├─base::withCallingHandlers(...)
#> 17. │ ├─knitr:::process_group(group)
#> 18. │ └─knitr:::process_group.block(group)
#> 19. │ └─knitr:::call_block(x)
#> 20. │ └─knitr:::block_exec(params)
#> 21. │ └─knitr:::eng_r(options)
#> 22. │ ├─knitr:::in_input_dir(...)
#> 23. │ │ └─knitr:::in_dir(input_dir(), expr)
#> 24. │ └─knitr (local) evaluate(...)
#> 25. │ └─evaluate::evaluate(...)
#> 26. │ └─evaluate:::evaluate_call(...)
#> 27. │ ├─evaluate (local) handle(...)
#> 28. │ │ └─base::try(f, silent = TRUE)
#> 29. │ │ └─base::tryCatch(...)
#> 30. │ │ └─base (local) tryCatchList(expr, classes, parentenv, handlers)
#> 31. │ │ └─base (local) tryCatchOne(expr, names, parentenv, handlers[[1L]])
#> 32. │ │ └─base (local) doTryCatch(return(expr), name, parentenv, handler)
#> 33. │ ├─base::withCallingHandlers(...)
#> 34. │ ├─base::withVisible(value_fun(ev$value, ev$visible))
#> 35. │ └─knitr (local) value_fun(ev$value, ev$visible)
#> 36. │ └─knitr (local) fun(x, options = options)
#> 37. │ ├─base::withVisible(knit_print(x, ...))
#> 38. │ ├─knitr::knit_print(x, ...)
#> 39. │ └─knitr:::knit_print.default(x, ...)
#> 40. │ └─evaluate (local) normal_print(x)
#> 41. │ ├─base::print(x)
#> 42. │ └─bcdata:::print.bcdc_recordlist(x)
#> 43. │ └─purrr::imap(...) at bcdata/R/utils-classes.R:142:2
#> 44. │ └─purrr::map2(.x, vec_index(.x), .f, ...)
#> 45. │ └─purrr:::map2_("list", .x, .y, .f, ..., .progress = .progress)
#> 46. │ ├─purrr:::with_indexed_errors(...)
#> 47. │ │ └─base::withCallingHandlers(...)
#> 48. │ ├─purrr:::call_with_cleanup(...)
#> 49. │ └─bcdata (local) .f(.x[[i]], .y[[i]], ...)
#> 50. │ ├─base::nrow(bcdc_tidy_resources(x[[.y]])) at bcdata/R/utils-classes.R:144:4
#> 51. │ └─bcdata::bcdc_tidy_resources(x[[.y]])
#> 52. └─purrr (local) `<fn>`(`<sbscOOBE>`) at bcdata/R/bcdc_search.R:324:2
#> 53. └─cli::cli_abort(...)
#> 54. └─rlang::abort(...)Looking at the API docs, it seems that our resulting {bcdata} call from the above function (https://catalogue.data.gov.bc.ca/api/3/action/package_search?q=Metadata%20for%20ICBC+download_audience:Not%20downloadable) fails because we (now?) need to wrap some values in quotes. For example, this query works in a browser: https://catalogue.data.gov.bc.ca/api/3/action/package_search?q=Metadata%20for%20ICBC+download_audience:"Not%20downloadable".
In addition to fixing the above (the quoting), I wonder if we should set download_audience = NULL as per the other facets. There are now a lot more records in the other categories compared to when this function was first developed.
bcdata::bcdc_search_facets(facet = "download_audience")
#> facet count display_name
#> 1 download_audience 2277 Public
#> 2 download_audience 226 Not downloadable
#> 3 download_audience 51 Named users
#> 4 download_audience 174 NA
#> 5 download_audience 8 Government and Business BCeID
#> 6 download_audience 496 Government
#> name
#> 1 Public
#> 2 Not downloadable
#> 3 Named users
#> 4 NA
#> 5 Government and Business BCeID
#> 6 GovernmentCreated on 2023-03-01 with reprex v2.0.2
Searching for "Metadata for ICBC" through the web interface yields 18 records, e.g.,
Using
bcdata::bcdc_search("Metadata for ICBC")returns 10 of the above records, but omits the eight where the record title begins with "Metadata for ICBC" and that all belong to one organization.Created on 2023-03-01 with reprex v2.0.2