ropensci-archive / bomrang

:warning: ARCHIVED :warning: Australian government Bureau of Meteorology (BOM) data client for R
Other
109 stars 26 forks source link

Custom useragent #132

Closed jonocarroll closed 3 years ago

jonocarroll commented 3 years ago

Closes #130

It appears the issues are twofold:

  1. BOM seems to be blocking RStudio-specific useragents as I am able to read the relevant file(s) in a terminal session or when setting the useragent to something non-RStudio.
  2. Some recent changes borked my zip-reading functionality and attempted to read a .zip with data.table::fread(), which won't work unless you mean to do something like data.table::fread(cmd = paste0('unzip -pq ', tmp)) but it doesn't appear to work with .zip files containing multiple files.

I've routed around the first by setting a custom useragent any time data is requested which includes the name and version of the package. This will better identify the package to anyone checking logs at BOM (presumably what has precipitated (:drum:) the restriction) and may result in further conversations.

I've fixed the second by reinstating .get_zip_and_load(). I don't know what other consequences this has, but it removes the calling of .get_url() so you may want to patch that back in.

Summary of tests:

before (devel):

══ Testing test-get_historical.R ═══════════════════════════════════════════════

── Error (test-get_historical.R:47:3): Query stationid = '023000' ───────────────────────────
Error: embedded nul in string: '\177\034\027\xfb\020\xbb\x8e\xab'\x8a0O\021\xb8B\xd6i\xfa\xf26\xab\xee̩\xbfU\xaf\x93\x84\xd9\xf5!\xe0T\xeb\xf5!\x9b\xaa\xb7\036\037\xf2\xa7\xa4\xfc\xb4q\006 To=>$U!\xb7\xde\xf8i\177|H\xa1ƪLJ\xf0\0\xacLJ\xb0\xef\xadLJ\xf0ȪLJ\xf0dT\xe5C\xb8\b=\xd1-\xeb\xf5!+\xf1ҙ3[gjc\xb3}\xcc`8\xd7\xe3C\xfe\x8cS\0302\xf5\xf8\x90?ՋMo\xb9V8>ĸ\xf7\x9e\xe0\xa7$\xccz\xc2E\xd7\xe3C\xca\xe6\xfev\x82\037\xf4H\xf5\xfa\020\fW\xea\t~\xfe\xf87\034Y'\xf8I4V=a\022\xbd\xb4y4Eݲ\035\xc7Uq\xa2lޗ\xe8\xdb\xdau\\\xd8C\xda\r~2/\035\xa7B\xa0[\xb6\033\xfcdJ\xba\022\xadЎ\xe32\x9c\001\xdbq\\\005c\xcbv\034W\xc1\x88\xab\035\xc7\xf5\xa7z\xb1B\x8e\xe3Boٮ\xe3\xc2ɨ\035\xc7\xf5\xe7\xa5X!e\xa4\x9a\xfe\x8f\xe3*\xe8\xb8\xdaq\\\177̰M\xff8\xae?fؑ\xec\x973;+?\xee!f\t\xe7Ю\xe3B\xffִjc\xb3;}`ӟ\xe0\a\xe7\xacv\034W\xc1\x95Q;\x8e\xeb\x8f\0316\xbd\xddV\0\xdfۮ\xe3\xaaT\xbd\xdd\027\xe8\xf4m\xddC_\xfa\xb6.\x8f\004M߽\003S\021\xfa\xf9\xe3qڏG*\x8dzo?\036)U\xd2u|\b\x9b\xed\xc4\xc8\xea\xd7#\xe1\032\xb0_\x8f\xd4

[ FAIL 4 | WARN 4 | SKIP 0 | PASS 9 ]

The embedded nul is where fread() tries to read a .zip.

after (custom_useragent):

══ Testing test-get_historical.R ═══════════════════════════════════════════════
[ FAIL 0 | WARN 0 | SKIP 0 | PASS 27 ] Done!

Test complete
jonocarroll commented 3 years ago

It would be entirely reasonable to request that this be split into 2 PRs to address the separable concerns, but I leave that decision to you.