Shippable builds very slow compared to Codeship

[daveblake]

We're trialling Shippable and Codeship side by side - at the moment Shippable is taking approx 25 mins to build / test whereas the equivalent Codeship build tasks < 4 mins.

I'm really interested if we have a mistake in our configuration or if there is anything we should be doing to improve speeds.

For the following build we had the following times: https://app.shippable.com/runs/587671feed52f70f00e148fc/1/console

Codeship: 3 min 9 sec Shippable: 26 mins

The yml file looks like:

language: php

php:

• 5.6

branches: except:

• dev*

addons: hosts:

• example.org

environment variable

env: global:

• MAIL_DRIVER=log
• DB_CONNECTION=sqlite
• DB_DATABASE=testing.sqlite
• secure: ### HIDDEN SECURE ###

before_install:

• composer config -g github-oauth.github.com $SPARK_GITHUB_TOKEN #load the encrypted GitHub token from shippable to gain access to Laravel/Spark

before_script:

• sudo apt-get update
• sudo npm cache clean -f #clear npm cache
• sudo npm install -g n #install npm and update
• sudo n stable
• npm install --silent #install node modules silently
• npm install -g gulp #install gulp
• composer self-update -n
• composer install --prefer-source --no-interaction --no-progress #downloads vendor
• cp -v .env.example .env # creates .env file
• php artisan key:generate # generates secure key
• touch database/testing.sqlite
• php artisan migrate --seed #creates and seeds database table
• gulp --production # run gulp

script:

• phpunit --log-junit shippable/testresults/junit.xml --coverage-xml shippable/codecoverage

after_success:

hidden deploy hooks

[rageshkrishna]

@daveblake Does turning off code coverage change things? We've seen issues in the past with phpunit trying to run coverage on everything instead of just your code, but I'm not 100% sure if that's what's causing the slowness that you're seeing. It would be helpful if you could try a build with coverage turned off. Thanks.

[daveblake]

Thanks - it's down to about 9 minutes not, considerably better but still not as fast. I've turned on caching and will see if that makes any difference.

[manishas]

@daveblake We spin up a machine for your builds, and this spin up time adds up to 4 mins for most builds. If a machine is already available in your account, we do reuse it so some builds will seem faster.

To avoid the machine spin up time, you can try using our BYON feature which lets you run builds on your own machines. This means you can attach your VMs from AWS or DO to your Shippable account and your builds run there. There are many advantages to BYON as explained here: http://docs.shippable.com/ci/advancedOptions/byon/

Ragesh can help with any config improvements but I definitely think BYON will cut down your time by half or more.

[avinci]

I think we need to understand if there are pre-reqs in the php composer and npm that cadetship builds machines have preinstalled and when @daveblake is running it on codeship it just skips and we are actually installing them.

One option is also you could build a custom build image that has all the pre-reqs you need and use that for your build.

@daveblake do you know how much of the time is install and how much is actually running the tests?

[gaumire]

@manishas I was looking into the BYON option, a quick question ... how would the pricing work when I have my own infrastructure hooked up with shippable.

• Can I have multiple nodes attached ?
• Where do I configure the resources I hook up to the containers in the node ?
• Are there any limitations on the number of builds I can run concurrently?
• Would the artifacts still be stored in your s3 ?

[manishas]

@gaumire

• how would the pricing work when I have my own infrastructure hooked up with shippable. Pricing remains the same at $25/mo per parallel build.

• Can I have multiple nodes attached ? Yes, you can have as many nodes as you have in your pricing plan for the subscription.

• Where do I configure the resources I hook up to the containers in the node ? Not sure what you mean by this.... you attach the machines to your Shippable subscription and during the initialization phase, we will install some pre-reqs on the machines, including a Shippable agent. At that point, your machines will start listening to a queue where our SaaS service will add triggered builds. Your machines will pick the builds up and run them. Let me know if that answers your qs.

• Are there any limitations on the number of builds I can run concurrently? The limit is number of minions in your Subscription.

• Would the artifacts still be stored in your s3 ? Yes.

If you're interested in running Shippable behind your firewall and pushing artifacts to your own S3, we also have a Server version of Shippable. Let me know if you want more details on that.

[daveblake]

@avinci

See below images for a breakdown of times from Shippable / Codeship. What is a bit interesting is Codeship total says 14 minutes but if you add up the individual times it comes to about 10 minutes.

@manishas I will look further into BYON but one of the reasons behind using SAAS is for the simplicity. What sort of size AWS box do you recommend? Can we use the free tier?

[manishas]

@daveblake our AWS boxes are c3.large so you will need at least that or better.

BYON requires very little management from your end. Attaching the nodes should take just a few minutes and you don't need to do anything after that. Your builds automatically run on your nodes. This saves the 3-4 min node spin up time.

[manishas]

Closing this issue.