search

Galooshi / happo

Visual diffing in CI for user interfaces
505 stars 16 forks source link

Travis CI integration example #44

Open lencioni opened 8 years ago

lencioni commented 8 years ago

I think it would be really helpful for a lot of people if we had an example of how to use this with Travis CI somewhere. I plan on working on this soon, but I wanted to open this issue to track my progress.

Things I think I will need to conquer:

Probably need something like:

before_script:
  - export DISPLAY=:99.0
  - sh -e /etc/init.d/xvfb start

Taken from http://karma-runner.github.io/0.8/plus/Travis-CI.html

Checking out different commits

I'm not sure if the travis environment has the repo in it like we do in Jenkins, so I'm not sure what will be required to jump around different versions of the code.

For PRs, we'd probably want to check out whatever branch the PR was being submitted against.

Posting comments back to GitHub

This will probably require the administrator of the repo to set up an environment variable in the travis interface or add an encrypted variable to the .travis.yml file with a GitHub API token.

http://docs.travis-ci.com/user/environment-variables/

This will allow us to use the GitHub API to post comments.

lencioni commented 8 years ago

I was hoping that I could make it so that instead of uploading things to S3, it would just post all of the diffs in the comment directly, since GitHub supports comments with images. However, it appears as though the API does not support uploading images, so I may need to figure out the best thing to do there.

One option would be to use a different service like S3 or imgur to host the images, and then just post the comment to GitHub with those images referenced. The bummer about this is that it will require more work for people to get set up, which I was hoping to avoid.

https://api.imgur.com/

lencioni commented 8 years ago

I wonder if I could work around this by using data URIs in GitHub comment posts instead of uploading images. It looks like this currently won't work but I'm hopeful that this pull request will enable this feature.

trotzig commented 8 years ago

Running in Firefox happens in our internal Travis setup, where we indeed do exactly what you suggest:

before_install:
  - "export DISPLAY=:99.0"
  - "sh -e /etc/init.d/xvfb start"

We then use the Headless gem to make a display available. Instead of using Headless, I think you can wrap commands in an xvfb-run call, e.g. xfvb-run diffux_ci. I actually thought I went for that solution instead of using Headless, but it appears I didn't. The Travis docs has a section on using xvfb-run: http://docs.travis-ci.com/user/gui-and-headless-browsers/#Using-xvfb-to-Run-Tests-That-Require-GUI-(e.g.-a-Web-browser)

Regarding checking out a different commit in Travis, this is the closest I can find to something similar: http://stackoverflow.com/questions/32580821/how-can-i-customize-override-the-git-clone-step-in-travis-ci . It looks like you can git checkout just as we do in Jenkins, but I wouldn't be surprised if there are dragons lurking here.

Interesting about using base64 images in github comments. It looks like you came around this at the right time, the PR to html-pipeline was submitted just 8 hours after you started this issue!

Thanks for working on this! When we have a POC, we can start diffuxing selected popular github projects. I'm thinking material-ui, fixed-data-table and others.

trotzig commented 8 years ago

A first version could just use what we have with S3 today. Post a link to the results. We can treat the inline image posting as an enhancement.

trotzig commented 8 years ago

It would be awesome if we could get a diffux run for diffux-ci components. I think if we work on https://github.com/diffux/diffux_ci/issues/61 we might actually have something to diff.

lencioni commented 8 years ago

Another reason to stick with S3 as a requirement over just posting the images in comments on GitHub would be that it would allow us to provide a better UI (see #61) for viewing the results. We could always add images to the GitHub comments as an enhancement once that is possible, but for now I don't think we should look at it as a requirement.

trotzig commented 7 years ago

@lencioni: have you learned anything you could share in this issue since you last looked into it?

lencioni commented 7 years ago

Yes, I have learned a lot but I haven't had the time to put something coherent together yet.