CODE-RADE solves the problem of reliably, continuously delivering scientific applicaitions to common research computing platforms. Uses Github repos, Jenkins-CI and CVMFS to automatically build, test and deliver your applications to Linux boxes almost everywhere. Grid, Cloud, HPC, your own :computer:, it's all the same to CODE-RADE.
Curious ? Ok, prepare to have your mind blown :fireworks:...
CODE-RADE will build your application
We build artifacts on for the expected execution environment :
No, you can't use Docker. Yet... :soon: Seriously, Foundation Release 3 should have Docker support.
Once we're sure that the application will actually run in the target environment, you ship that sucker :ship: !
Even though it sounds like you need to be permanently online for CODE-RADE to work, CVMFS does a good job of caching, so you can happily set up a local cache at a site and consume the apps from there.
CODE-RADE builds, tests and deploys user-requested applications using can be used on almost any Linux machine :
All you need in each case is access to the repository, via CVMFS. In order to use it you need
. /etc/profile.d/modules.sh
# cvmfs stuff
export SITE=generic
export OS=u1404
export ARCH=x86_64
export CVMFS_DIR=/cvmfs/code-rade.africa-grid.org/
# Add modules
module use $CVMFS_DIR/modules/compilers
module use $CVMFS_DIR/modules/libraries
module use ${CVMFS_DIR}/modules/bioinformatics
module use ${CVMFS_DIR}/modules/astro
<etc>
module add <your favourite application>
There are examples of how to use the artifacts in the repo in the relevant subdirectories. Get started with grid examples.
Are you still here ? Here are some more gory details...
CODE-RADE has also been developed in the context of academic projects. There are several aspects of the platform which need development, so if you would like to propose work, or are interested in undertaking development on the project in the context of postgraduate work at a university, please open an issue and request further information. See collaborating.
Are you one of those Open Science hippies that is all about
Well, then come on in... Here's what we've got in the roadmap.
Information about how to set up CVMFS mounts can be found in cvmfs
Docker containers are used by Travis to check whether the repo is working. The container Docker files are in containers and you can find them on Docker Hub
We use both ZenHub and Waffle for providing project boards to developers and colaborators.
CODE-RADE is an open infrastructure project, and aims to reduce the barrier to entry for users of distributed computing infrastructure by solving the delivery problem. The applications you see here come from the work of a few people, and the design of the platform has come from the intellectual contributions of innumerable chats over :beers: and other beverages. For a platform to succeed, it needs to evolve and adapt to the needs of the user community - so if anything you've read here piques your interest, please read the guidelines on contributing and let's build something awesome. Essentially, there are a few general ways to collaborate :
Let's face it, our documentation sucks. The same could probably be said about the tests we try to run. Only domain experts can really know how to test whether the applications they use are properly built. If you are a domain expert, you can help us to improve the quality of the application in the repository by sending pull requests on the relevant check-build
script.
If you want to work on applications with us, it's probably best to check out my-first-deploy and contents thereof.
We build, test, and deliver. All of those can be done better and differently. There are a few design aspects which should not be comprimised - atomic dependencies, continuous integration, automation, _etc. Beyond that, we can talk different ways of building, testing and delivering.
Do you a computing facility that wants to use CODE-RADE ? Does it have the funkiest of hardware and special libraries that make applications go vroom ? Is it the next big thing ? Or, some bohemoth Iron Man style beast from the '90s that everyone has forgetten but is actually the :bomb: ? We don't care ! Cycles is cycles, and we want to run every damn application everywhere.
CODE-RADE provides direct links between the code and method that was used to produce specific scientific results and the results themselves. Each build provides a unique and persistent link to the actual binary that was executed. Each build is linked back via a commit message to a change controlled repository which describes how that binary was built. No hiding from Jenkins !
Some people have harder to blow minds than others. That's ok, we don't judge. Here's some more README stuff - but at this point you're better off reading the website
CODE RADE solves the delivery problem. We wanted a "fire-and-forget" model, requiring once-off action on the part of a site administrator. In order to do that, it needs to also implement the build, integrate and test problems. The latter (ie, everything but the delivery problem) have also been recently solved, and solved well by awesome projects such as :