search

philschatz / octokat.js

:octocat: Github API Client using Promises or callbacks. Intended for the browser or NodeJS.
http://philschatz.com/2014/05/25/octokat/
MIT License
421 stars 133 forks source link

Cannot fetch files > 1 mb from GitHub API. #261

Open johncade opened 6 years ago

johncade commented 6 years ago

I get the following error message trying to fetch a .json file that has grown larger than 1mb:

{
  "message":"This API returns blobs up to 1 MB in size. The requested blob is too large to fetch via the API, but you can use the Git Data API to request blobs up to 100 MB in size.",
  "errors":[{"resource":"Blob","field":"data","code":"too_large"}]
}

Is there a workaround that exists for this? Any help would be greatly appreciated, thanks in advance!

johncade commented 6 years ago

Any help would be so greatly appreciated.

philschatz commented 6 years ago

It seems like https://developer.github.com/v3/git/blobs/#get-a-blob allows you to retrieve larger files.

In order to get the sha for the blob, https://developer.github.com/v3/git/ might be useful. I have not had to use Blobs directly, but it might look something like:

  1. Get the branch to find the commit sha
  2. Get the commit for the branch to find the tree sha
  3. Get the tree corresponding to the commit to find subtrees or the blob sha
  4. recurse through subtrees if the JSON file is not at the root of the repository
  5. Get the blob corresponding to the file
  6. base64decode the blob (or maybe use octokat's .readBinary() method)

There are methods on octokat that correspond to each of the GitHub API calls. Examples can be found here: https://github.com/philschatz/octokat.js/blame/master/examples/README.md#L43-L50

Hope that helps!