dreamRs / datamods

Shiny modules to import and manipulate data into an application or addin
https://dreamrs.github.io/datamods/
GNU General Public License v3.0
138 stars 35 forks source link

Module to import from JSON APi #39

Closed eduardszoecs closed 2 years ago

eduardszoecs commented 2 years ago

Added a small module to import from an API that serve flat JSONs. Just paste the URL and your good to go. Have a look at tests for an example API (gapminder data)

codecov-commenter commented 2 years ago

Codecov Report

Merging #39 (64af1a1) into master (e4848c5) will increase coverage by 0.41%. The diff coverage is 83.01%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master      #39      +/-   ##
==========================================
+ Coverage   76.79%   77.20%   +0.41%     
==========================================
  Files          13       14       +1     
  Lines        1698     1799     +101     
==========================================
+ Hits         1304     1389      +85     
- Misses        394      410      +16     
Impacted Files Coverage Δ
R/utils-shiny.R 77.92% <0.00%> (ø)
R/import-modal.R 82.23% <80.00%> (-0.07%) :arrow_down:
R/import-api.R 84.70% <84.70%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update e4848c5...64af1a1. Read the comment docs.

pvictor commented 2 years ago

Thanks! That's interesting.

I have a couple of questions:

jsonlite::fromJSON("https://cdn.jsdelivr.net/gh/timruffles/gapminder-data-json@74aee1c2878e92608a6219c27986e7cd96154482/gapminder.min.json")
# -> GOOD
jsonlite::fromJSON("https://www.google.fr/")
# -> ERROR

It is less rigorous regarding what is provided as URL, but wrapping that in try() or tryCatch() + checking that output is a data.frame could do the job

# JSON
rio::import("https://raw.githubusercontent.com/dreamRs/datamods/master/inst/extdata/mtcars.json")
# CSV
rio::import("https://raw.githubusercontent.com/dreamRs/datamods/master/inst/extdata/mtcars.csv")

It will be more like the import_file module but with an URL as entrypoint.

Note: there's a %>% that isn't needed in read_json()

Victor

eduardszoecs commented 2 years ago

Great suggestions @pvictor! I'll refine it to a more general import_url (perhaps similar to a lightweight import_file)

eduardszoecs commented 2 years ago

@pvictor I've changed accordingly to a general import_url_*(), that supports all (flat) formats of rio. This also drop the httr dependency.

Addtionally, I've also extended insert_error to display the message from try() - which is used across all modules. This give the user a little bit more information on what might be wrong.

pvictor commented 2 years ago

That's perfect, thank you!

I just added an option to retrieve the data's name (file name).

Nice addition for insert_error !