xCAT and Confluent

[chulhokim]

---

Software Name

xCAT and Confluent (optional sub-component within xCAT but can be used independently)

---

Public URL

http://xcat.org/ - Main landing page for xCAT https://github.com/xcat2/xcat-core - new GITHUB repository

---

Technical Overview

From website:

xCAT is Extreme Cluster/Cloud Administration Toolkit, xCAT offers complete management for HPC clusters, RenderFarms, Grids, WebFarms, Online Gaming Infrastructure, Clouds, Datacenters, and whatever tomorrow's buzzwords may be. It is agile, extensible, and based on years of system administration best practices and experience. It enables you to:

Provision Operating Systems on physical or virtual machines: RHEL, CentOS, Fedora, SLES, Ubuntu, AIX, Windows, VMWare, KVM, PowerVM, PowerKVM, zVM. Provision using scripted install, stateless, statelite, iSCSI, or cloning. Remotely manage systems: lights-out management, remote console, and distributed shell support. Quickly configure and control management node services: DNS, HTTP, DHCP, TFTP, NFS

It has been deployed on small and large x86 clusters (multiple vendors), Power clusters, and have been enabled in prototype ARM cluster

---

Latest stable version number

xCAT 2.12.2

---

Open-source license type

xCAT - Eclipse Public License 1.0 (EPL-1.0) Confluent - Apache 2

---

Relationship to component?

• [X] contributing developer (company)
• [ ] user
• [ ] other

If other, please describe:

---

Build system

• [ ] autotools-based
• [ ] CMake
• [X] other

If other, please describe: xCAT have something a bit custom (just copying scripts from source to destination). Confluent uses python distutils.

Does the current build system support staged path installations? For example: make install DESTIR=/tmp/foo (or equivalent) Both support staged path installations.

• [X] yes
• [ ] no

---

Does component run in user space or are administrative credentials required?

• [X] user space
• [X] admin

---

Does component require post-installation configuration.

• [X] yes
• [ ] no

If yes, please describe briefly: Must declare managed nodes, number of other optional configuration.

---

If component is selected, are you willing and able to collaborate with OpenHPC maintainers during the integration process?

• [X] yes
• [ ] no

---

Does the component include test collateral (e.g. regression/verification tests) in the publicly shipped source?

• [ ] yes
• [X] no

If yes, please briefly describe the intent and location of the tests.

---

Does the component have additional software dependencies (beyond compilers/MPI) that are not part of standard Linux distributions?

• [X] yes
• [ ] no

If yes, please list the dependencies and associated licenses.

---

Does the component include online or installable documentation?

• [X] yes
• [ ] no

If available online, please provide URL. http://xcat-docs.readthedocs.io/en/stable/ <- New document repository - not all documents migrated https://sourceforge.net/p/xcat/wiki/XCAT_Documentation/ <- Older document repository

---

[Optional]: Would you like to receive additional review feedback by email?

- [X] yes - [ ] no

[koomie]

@chulhokim, you indicated that there are additional software dependencies, but did not list the dependencies and associated licenses. Can you please update accordingly?

[koomie]

@chulhokim: another follow up question on this submission. I am familiar with the fact that xCAT builds for a variety of distros are already being made available at:

http://xcat.org/files/xcat/repos/yum/xcat-dep/

In this request, are you suggesting to (1) leverage this existing distribution repository, or (2) repackage from src for distribution in the OpenHPC repositories?

[chulhokim]

I have been discussing with my team member who is our Lenovo's xCAT developer. We believe it would be best to leverage the xCAT version that Lenovo is testing for its x86 cluster solution. Here is an example of our best recipe page - search for 'xCAT' or 'Cluster Manager' section: https://support.lenovo.com/us/en/solutions/HT118877 The latest version that IBM puts in the distribution is first tested on Power platform and does not necessary line up with the version that Lenovo tests on x86 platform and there have been bugs in the past and as well as recently where newer version of xCAT had functional issues on x86. When someone reports the bug, it will be investigated and eventually fixed. Since for OpenHPC, we want stable tested version, I think this would be the safest way to go. Most of the packages are 'noarch' and not tied to platform. Here are the sub-list that we think should be included. elilo-xcat.noarch 3.14-4 @xcat-dep - gplv2 license ipmitool-xcat.x86_64 1.8.15-3 @xcat.org_files_xcat_repos_yum_xcat-dep_rh6_x8664 - BSD license syslinux-xcat.noarch 3.86-2 @xcat-dep - gplv2 license xnba-undi.noarch 1.0.3-131028 @xcat-dep - gpl with unmodified binary

perl-HTTP-Async.noarch 0.30-2 @xcat.org_files_xcat_repos_yum_xcat-dep_rh6_x8664 perl-JSON.noarch 2.50-1.el6.rfx @xcat-dep perl-Net-HTTPS-NB.noarch 0.14-2 @xcat.org_files_xcat_repos_yum_xcat-dep_rh6_x8664 "This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself. " Perl5 is Copyright (C) 1993-2005, by Larry Wall and others. It is free software; you can redistribute it and/or modify it under the terms of either: a) the GNU General Public License as published by the Free Software Foundation; either version 1, or (at your option) any later version, or b) the "Artistic License".

There is one additional component that enables automatic node discovery but we are not sure on the licensing terms right now - I guess this can be delayed until we sort that out. xCAT-genesis-base-x86_64.noarch 1:2.12-snap201607221057 @xcat.org_files_xcat_repos_yum_xcat-dep_rh6_x8664

[chulhokim]

So for all the parts except the xCAT-genesis-base-x86_64.noarch, we believe should be able to be built with source or use the binary that Lenovo has validated with its x86 testing. It should also work on ARM platform (would have to do separate build) but we have not validated on ARM management server. Currently we have used x86 XCAT management node to install/config on our internal ARM servers.

[koomie]

@chulhokim, thank you for the submission and responses to follow up questions. The TSC has recommended moving forward with an xCAT based recipe. Since there is a pre-existing repository for xCAT and it's dependencies, the TSC recommends referencing pre-built packages from this repository in a companion OpenHPC recipe. The expectation is that end-users needing provisioning services could access separate recipes (e.g. for xCAT and Warewulf).

A new tracker has been created in https://github.com/openhpc/ohpc/issues/323 for integration/documentation work. Closing this out.