GitHub rate limiting 403 on dasht-docsets-install

[AdamWagner]

Love what you've done with dasht - great concept!

Unfortunately, I haven't been able to test-drive it yet due to dasht-docsets-install resulting in a 403 due to a RateLimit issue on the feeds url. Here's the full error:

❯ dasht-docsets-install
  HTTP/1.1 403 Forbidden
  Server: GitHub.com
  Date: Wed, 02 Jan 2019 07:47:12 GMT
  Content-Type: application/json; charset=utf-8
  Content-Length: 247
  Status: 403 Forbidden
  X-RateLimit-Limit: 60
  X-RateLimit-Remaining: 0
  X-RateLimit-Reset: 1546415757
  X-GitHub-Media-Type: github.v3; format=json
  Access-Control-Expose-Headers: ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type
  Access-Control-Allow-Origin: *
  Strict-Transport-Security: max-age=31536000; includeSubdomains; preload
  X-Frame-Options: deny
  X-Content-Type-Options: nosniff
  X-XSS-Protection: 1; mode=block
  Referrer-Policy: origin-when-cross-origin, strict-origin-when-cross-origin
  Content-Security-Policy: default-src 'none'
  X-GitHub-Request-Id: EC95:6FB3:2E874DD:76A2D0E:5C2C6C80
https://api.github.com/repos/Kapeli/feeds/git/refs/heads/master:
2019-01-02 02:47:12 ERROR 403: Forbidden.
  HTTP/1.1 403 Forbidden
  Server: GitHub.com
  Date: Wed, 02 Jan 2019 07:47:12 GMT
  Content-Type: application/json; charset=utf-8
  Content-Length: 247
  Status: 403 Forbidden
  X-RateLimit-Limit: 60
  X-RateLimit-Remaining: 0
  X-RateLimit-Reset: 1546415757
  X-GitHub-Media-Type: github.v3; format=json
  Access-Control-Expose-Headers: ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type
  Access-Control-Allow-Origin: *
  Strict-Transport-Security: max-age=31536000; includeSubdomains; preload
  X-Frame-Options: deny
  X-Content-Type-Options: nosniff
  X-XSS-Protection: 1; mode=block
  Referrer-Policy: origin-when-cross-origin, strict-origin-when-cross-origin
  Content-Security-Policy: default-src 'none'
  X-GitHub-Request-Id: EC9A:6FB3:2E874E1:76A2D1F:5C2C6C80
https://api.github.com/repos/Kapeli/feeds/git/trees/%7Brefrefs/heads/masternode_idMDM6UmVmMTEyMzkyNzk6bWFzdGVyurlhttps//api.github.com/repos/Kapeli/feeds/git/refs/heads/masterobject%7Bf6640fea1ab1e9bd1544722cf3ac0ee1e9a7416ftypecommiturlhttps//api.github.com/repos/Kapeli/feeds/git/commits/f6640fea1ab1e9bd1544722cf3ac0ee1e9a7416f%7D%7D:
2019-01-02 02:47:12 ERROR 403: Forbidden.

The output includes the same block of text twice, although it's the result of running dasht-docsets-install only once.

Running on macOS Mojave in fish shell. Have tested in bash with the same result.

[sunaku]

Possible workarounds:

• Retry from a different IP address (VPN or ISP) that hasn't sent so much traffic to GitHub API service.
• Inherit already installed docsets from Dash for macOS by setting the DASHT_DOCSETS_DIR envar.
• Manually download the docsets you're interested in from the list at https://github.com/Kapeli/feeds
  • You'll find *.tgz links near the top of the file, which you can then download from your browser.
  • Run dasht-docsets-extract *.tgz on the downloaded files to install them into your dasht area.

A permanent solution:

• Add support for specifying a GitHub access token in API requests sent by dasht to avoid rate limiting.

[sunaku]

Closing due to lack of response and because the issue seems to be beyond the scope of this project. :policeman:

[eikendev]

In case anybody else faces the same problem and lands here. I had set an empty User-Agent in my wget configuration (meaning no User-Agent would be sent along the request at all). This caused dasht to fail to download anything via the API, always resulting in Forbidden. When I removed the setting, dasht worked fine again.