search

hrbrmstr / wayback

:rewind: Tools to Work with the Various Internet Archive Wayback Machine APIs
https://hrbrmstr.github.io/wayback/index.html
54 stars 7 forks source link

remaining work #1

Open hrbrmstr opened 7 years ago

hrbrmstr commented 7 years ago

Finishing up the CDX API wrapper: https://github.com/internetarchive/wayback/tree/master/wayback-cdx-server

then error/edge-case handling + writing tests + code coverage

full-disclosure: seekrit project (i suck at seekrit projects) is a FOSS book on web scraping. Full credit in book and package to all who contribute.

jonocarroll commented 7 years ago

I'll have a play.

hrbrmstr commented 7 years ago

I added some is_* helpers today since I was waiting for an online student to arrive to class and got tired of poking at Microsoft.

jonocarroll commented 7 years ago

Looking good. I'll fiddle with internals later in the week, but I'm hoping things like

if ("timestamp" %in% colnames(res)) {
   res <- dplyr::mutate_(res, "timestamp" = lazyeval::interp(~anytime::anytime(t), t = quote(timestamp)))
}

replaced with 

res <- dplyr::mutate_at(res, dplyr::vars(dplyr::matches("timestamp")), dplyr::funs(anytime::anytime))

makes sense/is desired. The failing test (if there is no timestamp column) leaves res unchanged.

hrbrmstr commented 7 years ago

+1. I kinda get lazy in my pkgs and do a globalVariables() hack