Lets hook Ghee up to Travis CI and tested on multiple rubies.

huboard / ghee

This is an unofficial ruby client for the Github API. The end goal is a complete, simple, and intuitive ruby API for all things Github.

http://documentup.com/huboard/ghee

MIT License

160 stars 51 forks source link

Lets hook Ghee up to Travis CI and tested on multiple rubies. #7

Closed jonmagic closed 12 years ago

jonmagic commented 12 years ago

Doh, that wasn't supposed to pick up your commits from the other PR. Oh well... Anyways, first thought was, what if we just start including fixtures now? We'd have to make sure they scrubbed for any sensitive data, and if the GitHub API changes we'd have to update them, but it would make it 100x easier for someone just starting out with the library to run the tests and make Travis CI a synch.

rauhryan commented 12 years ago

What do you mean by "including fixtures"?

rauhryan commented 12 years ago

That would be awesome! Couple things off the top of my head we will need to make that happen.

A fake github account (gheeci) that has a fork of ghee_test with issues enabled
The specs will need to support ENV variables as well as the settings.yml
We'll need read/write access to the disk for VCR to work (unless there is a way to disable it)

I'll make an issue for 1.8.7 support, I think the only thing not compatible is the named capture groups we are using to parse out the pagination link header

jonmagic commented 12 years ago

Include all of the fixtures created by VCR that get stuck in the spec/responses dir...

Its going to take a bit of work to get that done, making sure they don't have sensitive data (tokens/passwords/etc), but it would make easier for someone to get started and Travis CI wouldn't need storage or need to make calls out to GitHub.

rauhryan commented 12 years ago

I wonder if there is a way to just disable VCR in CI and force it to make fresh API calls on the build server.

That way you don't have to deal with cleaning up the workspace or storing sensitive data.

If we had a separate github account, the only sensitive data to worry about is the OAuth token for that account that would hopefully be set in an ENV variable.

Something like this:

~> ACCESS_TOKEN=AFoue287429lokjal32 GH_USER=gheeci GH_REPO=ghee_test rake ci

gheeci commented 12 years ago

How about using a fake account and a generated token

28ae41160e2a0493e0f58e1f3f968f28c2c8398b

Check out this commit and tell me your thoughts

rauhryan commented 12 years ago

ooops gheeci is me

rauhryan commented 12 years ago

Holy crap! TravisCI is pretty freaking bad ass!

http://travis-ci.org/#!/rauhryan/ghee

I've almost got all the kinks worked out.

So interestingly...

The api will not allow you to create new Authorizations when authenticated with an OAuth token you most provide your username and password (i had to disable those specs)
travis kicks off the entire build matrix all at once and some of the test aren't itempotent, so I'm getting false failures because certain tests collide with each other

rauhryan commented 12 years ago

How bout this!

http://travis-ci.org/#!/rauhryan/ghee/builds/1044206

The build is very inconsistent tho, I'm not sure its the best idea to live test against the API

jonmagic commented 12 years ago

Working on a commit for this that uses cached responses so it doesn't have to live test against the API... will push as soon as I can, have it almost done.

jonmagic commented 12 years ago

Got it :) Looks like just 1.8.7 is failing...

http://travis-ci.org/#!/jonmagic/ghee

rauhryan commented 12 years ago

I may actually know why this is happening....

If you install 1.8.7 locally and run all the test you will notice that VCR stores the responses as a JSON string instead of the binary string it uses in 1.9*

Not sure why it behaves differently in 1.9.+ than 1.8 but I'll bet thats the problem

jonmagic commented 12 years ago

So I'm not sure I like the approach I took on this PR, but I thought it was worth a try. @rauhryan what do you think?

rauhryan commented 12 years ago

For some reason VCR doesn't output the same files for 1.8.7 that is does for 1.9.2 so for now I'm going to table travis-ci.

Currently I'm running specs locally on 1.8 and 1.9 just switching between them with RVM.

A few notes:

I'm not really comfortable with an extra step to scrub sensitive data from the response files, its too easy to forget and then bam... theres your password in plain text on github forever.

I don't like the extra hash you have to pass to vcr for every test, it perks up my spidie sense for too much ceremony and repetition. Its already annoying to have to wrap every test in a block

jonmagic commented 12 years ago

I agree to all your points. It felt dirty while I was doing it :)