Github rate limit error WITH authentication

[sergei-maertens]

Hi, I'm not sure if this is on jspm's end or Github's, but in Travis I'm running into broken builds because of the dreaded Rate limit errors.

Before today, this was working perfectly. The error I'm seeing because of jspm install is:

GitHub rate limit reached, with authentication enabled.

The build log is publicly visible: https://travis-ci.org/modelbrouwers/modelbrouwers/builds/144385431

As I said, I have set up the token before, and jspm correctly detects that auth is enabled. What gives?

[guybedford]

This is a regression unfortunately.

When we removed usernames from the github authentication process in https://github.com/jspm/github/pull/91 this stopped private authentication from working as private github needs the username and the token as the password.

It looks like we will need to revert the previous PR work, or merge in the ability to work with usernames again.

//cc @tamird

[tamird]

Am I missing something? I don't see that error in the build log at all.

private github needs the username and the token as the password.

Now I'm completely lost. What does this have to do with private github? Everything here appears to be public.

[guybedford]

@tamird when configuring github credentials to use private repos, it is apparently necessary to use the username and token as the password in order for the authentication to work.

[guybedford]

And thanks for the quick response!

[tamird]

@guybedford how did you determine that? What does that have to do with this issue?

[guybedford]

@AdamBurgess reported this along with a PR at https://github.com/jspm/github/pull/95#issuecomment-232688931. This issue is exactly because the private repo authentication is failing, and thus hitting the rate limit.

[sergei-maertens]

For the record, all of this build should be public.

On Jul 15, 2016 17:31, "Tamir Duberstein" notifications@github.com wrote:

@guybedford https://github.com/guybedford how did you determine that? What does that have to do with this issue?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jspm/github/issues/97#issuecomment-232985247, or mute the thread https://github.com/notifications/unsubscribe-auth/AFQ01rWjZzPV-WLtZtlx9MS5YD_3mcEuks5qV6fagaJpZM4JLaqu .

[guybedford]

@sergei-maertens is it a private github account though, even if public?

[guybedford]

(I may just be showing my ignorance here though)

[tamird]

This seems to have absolutely nothing to do with whatever is being discussed in #94.

@sergei-maertens I don't see any JSPM-related error in the log you linked. @guybedford sounds to me like this error (wherever it occurred) was possibly a real case of hitting GH's API limit?

[sergei-maertens]

it's my personal github account, the very same that I'm using here to comment.

The build log in Travis seems to have changed because the build was triggered by something else, I'll see if I can dig it up somewhere again.

[sergei-maertens]

This one has the same issue: https://travis-ci.org/modelbrouwers/modelbrouwers/builds/143824722

Exerpt:

travis_time:start:2547ba90
$ jspm config registries.github.auth $GITHUBTOKEN

travis_time:end:2547ba90:start=1468224022243393123,finish=1468224022780825407,duration=537432284
travis_fold:end:before_script.3
travis_fold:start:before_script.4
travis_time:start:08f59710
$ jspm install
     Looking up npm:babel-core
     Looking up npm:babel-runtime
     Looking up npm:core-js
     Looking up npm:URIjs
     Looking up npm:aurelia-http-client
     Looking up npm:aurelia-pal
     Looking up npm:aurelia-pal-browser
     Looking up npm:perfect-scrollbar
     Looking up npm:q
     Looking up npm:string.prototype.startswith
     Creating registry cache...
     Looking up github:components/jquery
     Looking up github:components/jqueryui
     Looking up github:twbs/bootstrap
     Looking up github:components/handlebars.js
     Looking up github:systemjs/plugin-json
     Looking up github:systemjs/plugin-css
     Looking up github:jspm/nodelibs-url
     Looking up github:twitter/typeahead.js
     Looking up github:sergei-maertens/django-ponyjs
     Downloading github:components/handlebars.js@2.0.0
     Downloading github:systemjs/plugin-json@0.1.0
     Downloading github:systemjs/plugin-css@0.1.13
     Downloading github:jspm/nodelibs-url@0.1.0
     Downloading github:sergei-maertens/django-ponyjs@0.1.2
     Downloading github:components/jquery@2.2.1
     Downloading github:components/jqueryui@1.11.4
     Downloading github:twbs/bootstrap@3.3.5
     Downloading github:twitter/typeahead.js@0.11.1
     Looking up npm:url
     Downloading github:components/handlebars.js@3.0.3
     Looking up github:js-cookie/js-cookie
     Downloading github:js-cookie/js-cookie@2.1.0

warn Error on download for github:twbs/bootstrap

     GitHub rate limit reached, with authentication enabled.
     The rate limit will reset in 5 minutes.

err  Error downloading github:twbs/bootstrap.

warn Installation changes not saved.

travis_time:end:08f59710:start=1468224022785126821,finish=1468224025060958172,duration=2275831351

The command "jspm install" failed and exited with 1 during .

Your build has been stopped.

[tamird]

Have you confirmed that this is not a legit case of you having exceeded the API limit?

[sergei-maertens]

I could hardly imagine where, did no local jspm install and there were not multiple builds running at the same time. I'm not sure where to start diagnosing the problem, since I can't reproduce it locally either

On Jul 15, 2016 17:41, "Tamir Duberstein" notifications@github.com wrote:

Have you confirmed that this is not a legit case of you having exceeded the API limit?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/jspm/github/issues/97#issuecomment-232988040, or mute the thread https://github.com/notifications/unsubscribe-auth/AFQ01koEpwkVjUZ6heIaAhpiYtcKddzZks5qV6oZgaJpZM4JLaqu .

[guybedford]

@sergei-maertens perhaps try alternative GitHub credentials and see if that helps?

[tamird]

Seems suspicious that you can't reproduce it locally, doesn't it?

[GregDracoulis]

I'm seeing this start to happen much more often in my Travis builds as well. Not sure if something has changed with JSPM, or GitHub has changed their rate limits (I'm thinking it's probably the former?)

@tamird this issue would be tricky to reproduce locally, since if you have already run jspm install you have your dependencies cached. You could clear your registry cache and remove the jspm_packages directory, then re-run jspm install. When you're running in a CI environment, you're always starting from a blank slate.

@guybedford I think there's just some confusion in terminology since the API is being used with GitHub account credentials to check out all public_repos. I do believe that you still have to use usernames with the given access token to authenticate through the API?

[tamird]

Have you tried adding $HOME/.jspm to your Travis cache?

On Jul 19, 2016 18:21, "Greg Dracoulis" notifications@github.com wrote:

I'm seeing this start to happen much more often in my Travis builds as well. Not sure if something has changed with JSPM, or GitHub has changed their rate limits (I'm thinking it's probably the former?)

@tamird https://github.com/tamird this issue would be tricky to reproduce locally, since if you have already run jspm install you have your dependencies cached. You could clear your registry cache and remove the jspm_packages directory, then re-run jspm install. When you're running in a CI environment, you're always starting from a blank slate.

@guybedford https://github.com/guybedford I think there's just some confusion in terminology since the API is being used with GitHub account credentials to check out all public_repos. I do believe that you still have to use usernames with the given access token to authenticate through the API?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/jspm/github/issues/97#issuecomment-233784525, or mute the thread https://github.com/notifications/unsubscribe-auth/ABdsPIp9GXuD6Fi6WiQzah7w7FqtI2Xhks5qXU3pgaJpZM4JLaqu .

[GregDracoulis]

^ yeah, looks like adding that and/or jspm_packages to your .travis.yml like this would help:

cache:
  directories:
  - $HOME/.jspm
  - jspm_packages

I worry that this might be obscuring a deeper issue though. On my end, we can't use the Travis cache because we have an enterprise deployment without S3 set up to store the cache artifacts.

[sergei-maertens]

I have tried with a new token and adding the cache - however I can't get my deps installed in the first place, so the cache is doing nothing at this point.

Build log: https://travis-ci.org/modelbrouwers/modelbrouwers/builds/146775809

[guybedford]

@sergei-maertens is this on jspm 0.17 or jspm 0.16? If using jspm 0.16, try using jspm-github 0.13.13, rerunning the auth and see if that alters the rate limit behaviour. (this involves ensure jspm 0.16 is using jspm-github 0.13.13).

That will at least confirm if this is a regression or not.

[sergei-maertens]

This is 0.16.

After carefully picking the window to restart the build, it was successfull, and adding $HOME/.jspm to the Travis cache seems to work well enough.

I will try the suggestions later tonight (hopefully).

[guybedford]

This should be resolved in https://github.com/jspm/github/pull/112 now.

[guybedford]

Released in 0.13.17.