The Hadoop Ecosystem Builder -Buildoop- provides interoperable tools, metadata, and processes that enable the rapid, repeatable development of a Linux Hadoop based system.
With Buildoop you can build a complete set of Hadoop ecosystem components based on RPM or DEB packages, make integration tests for this tools on a RedHat/CentOS, or Debian/Ubuntu virtual system, and maintain a set of configuration files for baremetal deployment.
Buildoop have receive a reestructuration of the code to isolate the core (builder and packager) and the recipes. The previous recipes has been moved to https://github.com/keedio/buildoopRecipes With these detachment we obtain an easier recipes versions maintenance. Also brings to the project the possibility to be used for build no-hadoop tools.
The Buildoop is splitted in the following fundations:
From the technology point of view Buildoop is based on:
buildoop: Main folder for Buildoop main controler.
conf: Buildoop configuration folder, BOM definitions, targets definitions.
deploy: Folder for deploy in VM and Baremetal systems. Based on Puppet and Chef.
sit: System Integration Testing tests for VM pseudo-cluster system.
toolchain: Tools for cross-compiling for diferent targets.
Download Groovy binary:
wget http://dl.bintray.com/groovy/maven/groovy-binary-2.3.3.zip
Clone the project:
git clone https://github.com/keedio/buildoop.git
Set the enviroment:
cd buildoop && source set-buildoop-env
In order to build some packages you need install some dependecies:
less buildoop/doc/DEPENDENCIES
Usage examples:
List available distributions-versions in the external repository
buildoop -remoterepo https://github.com/keedio/buildoopRecipes
Select a distribution-version and download it
buildoop -downloadrepo https://github.com/keedio/buildoopRecipes openbus-v1.0
Build the whole ecosystem for the distribution openbus-v1.0:
buildoop openbus-v1.0 -build
Build the zookeeper package for the distribuion openbus-v1.0:
buildoop openbus-v1.0 zookeeper -build
For more commands:
less buildoop/doc/README
Clone the repository from your project fork:
$ git clone https://github.com/keedio/buildoop.git
The clone has as default active branch "buildoop-v1-dev"
`$ git branch
Yo have to make your changes in the "buildoop-v1-dev" branch.
$ git add .
$ git commit -m "...."
$ git push origin
1
When you are ready to purpose a change to the original repository, you have to use the "Pull Request" button from GitHub interface.
The point is the pull request have to go to the "buildoop-v1-dev" branch so the pull request revisor can check the change, pull to original "buildoop-v1-dev" branch, and the last step is to push this "development pull request" to the "buildoop-v1-master" branch.
So the project has two branches:
Feature | Desc | State |
---|---|---|
Core Engine | Core building engine | Done |
POM versioning | Simple BOM multi-versioning | Done |
Git repsotory | Download sources from GIT | Done |
Svn repsotory | Download sources from Subversion | Pending |
Code refactoring | More elegant code | Forever Pending |
Cross-Architecture | Cross build from different distributions | Pending |
DEB Support | Debian/Ubuntu Support | Pending |
Layers | Add/Modify features without modify the core folders | Pending |
SIT | System Integration Tests | Pending |
-- Javi Roman jromanes@redhat.com Marcelo Valle mvalle@keedio.com