Send Jenkins into retirement

[standage]

Now that we have Travis up and running, I think we should slate Jenkins for retirement soon.

• I keep getting notifications from Rackspace (used to host the Jenkins web interface and the Linux test slave) about charges, or expiring certificates, or any number of other things that I have absolutely no interest in worrying about.
• Now that we're automatically requiring reviews and passing CI builds for a merge, the fact that we frequently have to request Jenkins to re-test PRs/commits is frustrating.

[ctb]

Sounds good to me. Let's check with Luiz and Michael re rackspace stuff to make sure we appropriately deactivate there.

Are we testing on OS X with Travis now?

[standage]

Are we testing on OS X with Travis now?

Yep: https://travis-ci.org/dib-lab/khmer/jobs/161075734

[betatim]

Things to do when switching over:

• [x] configure one of the travis runs to upload to codecov (I vote OSX one)
• [x] decide if the OSX travis should fail if pep8 and friends are unhappy (I think yes)

[standage]

Also, we should extend the test matrix to include other versions of Python (at least 3.5, maybe other 3.x versions). Since the OSX VMs are take more time to reserve and launch, I'm ok if we only do this on Linux.

• [x] Add Python 3.5 on Linux to config matrix

[standage]

I just noticed that the Travis build does not fail on pep8 errors. I agree with @betatim that it should.

[ctb]

Could someone (@betatim? @standage?) update this issue, please?

[betatim]

Once #1464 is done I think this issue will be ready for closing.

[standage]

We are finalizing the Travis CI configuration (see #1464). We had already disabled Jenkins builds, but have not yet turned off the CI servers (they can be re-enabled with a few button clicks). If Travis has no issues during a trial period, we'll give @mr-c the go-ahead to decommission ci.oxli.org. I suggest Nov 1st as the date we pull the plug.

[mr-c]

Did the release test get ported to TravisCI as well?

[betatim]

No. Where do I go to read about what this is/how to implement it? Is this point 7 of https://khmer.readthedocs.io/en/latest/dev/release.html ?

[mr-c]

@betatim Jenkins was automatically running through (almost) all of the release tests: http://ci.oxli.org/job/khmer-releasetest/configure (log in to see the script)

[betatim]

Copying the config below. Travis currently only runs tests that are not huge. I'll look into changing that. I am quietly confident that running the stuff related to pip below are not needed in each travis run.

• [ ] run huge tests on travis

#!/bin/bash

set -e
set -x

rm -Rf testenv? || /bin/true

if [[ "${NODE_LABELS}" == *osx* ]]
then
attr="!known_failing,!huge,!linux"
else
attr="!known_failing"
fi

export HEAD=`git rev-parse HEAD`
virtualenv testenv1
virtualenv testenv2
virtualenv testenv3
virtualenv testenv4
# First we test the tag

source testenv1/bin/activate
make install-dependencies
make test
pip uninstall -y khmer || true; pip uninstall -y khmer || true; make install
mkdir testenv1/not-khmer # if there is a subdir named 'khmer' nosetest will execute tests
# there instead of the installed khmer module's tests
pushd testenv1/not-khmer; ../bin/nosetests khmer --attr ${attr}; popd

# Secondly we test via pip

cd testenv2
source bin/activate
pip install -U setuptools==3.4.1
pip install -e git+https://github.com/dib-lab/khmer.git@${HEAD}#egg=khmer
cd src/khmer
make install-dependencies
make dist
make test
cp dist/khmer*tar.gz ../../../testenv3/
pip uninstall -y khmer || true; pip uninstall -y khmer || true; make install
cd ../.. # no subdir named khmer here, safe for nosetesting installed khmer module
bin/nosetests khmer --attr ${attr}

# Is the distribution in testenv2 complete enough to build another
# functional distribution?

cd ../testenv3/
source bin/activate
pip install -U setuptools==3.4.1
pip install khmer*tar.gz
pip install nose
tar xzf khmer*tar.gz
cd khmer*
make dist
make test
pip uninstall -y khmer || true; pip uninstall -y khmer || true; make install
mkdir ../not-khmer
pushd ../not-khmer ; ../bin/nosetests khmer --attr ${attr} ; popd

[mr-c]

The point of the release checklist is to catch weird packaging errors, so I really recommend running the whole thing. Perhaps just on the master branch?

[betatim]

I'll put it in then, we can see how long it takes to run. I am -1 on only having it on master as you get the most annoying of all travis messages: your PR was green but now master is failing :)

[standage]

From Twitter (ht @luizirber and @ctb): https://docs.travis-ci.com/user/cron-jobs/

Doesn't solve the problem of long-running jobs though. We'd still have to find a way to make them run in the allotted time. But we could run daily/weekly rather than with every build.

[ctb]

I think: put it on the list of things to revisit after Daniel and I do a release.

[ctb]

Working on acceptance tests; see status update here: https://github.com/dib-lab/khmer/issues/1476#issuecomment-268409358