macOS support

[pagrubel]

Unable to build on mac laptop with OSX version 10.12.6:

Below is version information of my compiler:

Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include-dir=/usr/include/c++/4.2.1 Apple LLVM version 9.0.0 (clang-900.0.37) Target: x86_64-apple-darwin16.7.0 Thread model: posix InstalledDir: /Library/Developer/CommandLineTools/usr/bin Here is the output from the make command: $ make printf '%s+%s%s\n' \ $(cat VERSION) \ $(git rev-parse --short HEAD) \ $(git diff-index --quiet HEAD || echo '.dirty') \

VERSION.full echo "#define VERSION \"$(cat VERSION.full)\"" > bin/version.h echo "version () { echo 1>&2 '$(cat VERSION.full)'; }" > bin/version.sh cd bin && /Library/Developer/CommandLineTools/usr/bin/make SETUID= all cc -std=c11 -Wall -c -o ch-run.o ch-run.c ch-run.c:11:10: fatal error: 'argp.h' file not found

include

^~~~ 1 error generated. make[1]: [ch-run.o] Error 1 make: [all] Error 2

[reidpr]

Patricia, thanks for the new issue!

We have not previously considered OS X because it doesn't have the namespace system calls we use to run containers.

Could you tell us more about your use case and the workflow you envision? If we can be persuaded it fits Charliecloud's vision, then we'd love to see a PR.

Regarding the specific compile failure above, that's because we use the argp functions in glibc. I don't know how one might make them available on OS X.

[pagrubel]

We'd like to be able to build the container on mac and run them on the HPC systems.

[reidpr]

OK. Can you explain this use case in more detail? I'm still a little confused because even if you can successfully build images, you won't be able to run them, even for a sanity check.

Alternately, you can run Charliecloud in a virtual machine on your Mac: https://hpc.github.io/charliecloud/virtualbox.html. It works quite well, and since you're at LANL I can provide a pre-built image file if you like (e-mail me internally and I'll send you the link).

[pagrubel]

I will use a platform like Darwin to test for use case. I will look into the virtual case as well as seeing if I can solve this problem.

[le-ander]

Hey there,

I was looking into installing charliecloud on mac as well in order to create tarballs from my docker images using ch-builder2tar which I can then upload to the server running charliecloud.

As the VM solution seems like quite a bit of overhead for this purpose, I was wondering if tarballs created with docker save would be compatible with charliecloud's ch-tar2dir?

Thanks a lot in advance!

[j-ogas]

Hello @le-ander! ch-run needs a flattened filesystem tree (directory or squashfs).

docker save produces a tarred repository, which contains all parent layers and corresponding meta data (hash directories, layer tarballs, json, etc.), which wouldn't work:

``` $ docker save mpihello-openmpi -o mpihello.tar $ ch-tar2dir mpihello.tar /tmp creating new image /tmp/foo 845MiB 0:00:01 [================================================================>] 100% [...] /tmp/mpihello unpacked ok $ ls -lh /tmp/mpihello drwxr-xr-x. 2 jogas jogas 50 Aug 5 12:16 09b4a904061cbcccd9ba90ad2f1755f61f4c3fa45a0ca8bc5ccbb60adf1c2a84 drwxr-xr-x. 2 jogas jogas 50 Aug 5 12:16 0a61e1722b1d139db28a3a20d6f9a3d6d1d9645d7844bc1ba8d72693b7cb685e drwxr-xr-x. 2 jogas jogas 50 Aug 5 12:16 27d7818c590444f92d885f0c01ca42ce5b1f1ef77c146e092c29c7063131de74 -rw-r--r--. 1 jogas jogas 7.6K Aug 5 12:16 32f94b26f508604afe81caf1bcba9944ed943eca690596b512a379fa21040162.json drwxr-xr-x. 2 jogas jogas 50 Aug 5 12:16 3e42179c6482b84dc0f5d9812be3ff01011a93b0a12873421463d5cdf0bb0637 drwxr-xr-x. 2 jogas jogas 50 Aug 5 12:16 424096783465c9032ef00f33ec6a8f08270d77764faa83fc69e77f7bbca54368 drwxr-xr-x. 2 jogas jogas 50 Aug 5 12:16 43a058b250cae44858c3e5f320fa7174f3a54515329c9930149a964e656034ce drwxr-xr-x. 2 jogas jogas 50 Aug 5 12:16 4a23901f0227b15b1251a9785013cec0b9ec5d0d9708dee25e29752f55f84c4c drwxr-xr-x. 2 jogas jogas 50 Aug 5 12:16 5f01a031b1e5eb27e34892ca394ab3f42cdcca67223b61734a5f76aad732b185 drwxr-xr-x. 2 jogas jogas 50 Aug 5 12:16 63a9ea169652012829470065feb7942179d81f91ae3425392a6f5f8ddeb976de drwxr-xr-x. 2 jogas jogas 50 Aug 5 12:16 6d0e8910d48c8e97857e732dd1f4d79d2fe10ae0921de15726952e8fdfffb109 drwxr-xr-x. 2 jogas jogas 50 Aug 5 12:16 793dcf1cfdbcc7df30f5fb3177e076aa1c27b683edd1360467cab132b607c0b1 drwxr-xr-x. 2 jogas jogas 50 Aug 5 12:16 90d211418820b7f7fb5d8b7d3c676740f5e344aeb7365e5e3135bf3f9f9ba7d0 drwxr-xr-x. 2 jogas jogas 50 Aug 5 12:16 9e29d33324e16165d74d8cb694abeaf67455c7da26365ecc8a55df9f789e2b8c drwxr-xr-x. 2 jogas jogas 50 Aug 5 12:16 ae1b39a037869d761dd41a3e389253b030b0ea5e765be8093bcc0627ca2a53fa drwxr-xr-x. 2 jogas jogas 50 Aug 5 12:16 b45bf41abcdc64d8239ee008416d229e156930d24c82b46e185b50f8efc4d369 drwxr-xr-x. 2 jogas jogas 50 Aug 5 12:16 b543d5dac9968560ebb880c108b0e47b8eea57b73e0803d7627681560990f548 drwxrwxr-x. 2 jogas jogas 6 Aug 14 08:01 dev drwxr-xr-x. 2 jogas jogas 50 Aug 5 12:16 e45ed4b43f57653d43d38404d0314787f68248694449fbf93c12a478657dbc84 drwxrwxr-x. 2 jogas jogas 38 Aug 14 08:01 etc drwxr-xr-x. 2 jogas jogas 50 Aug 5 12:16 fac42474ae32068a2f8103e0fde3be219392d90ec4f4d1c45de670e69ab68402 -rw-r--r--. 1 jogas jogas 1.5K Dec 31 1969 manifest.json drwxrwxr-x. 12 jogas jogas 96 Aug 14 08:01 mnt drwxrwxr-x. 2 jogas jogas 6 Aug 14 08:01 proc -rw-r--r--. 1 jogas jogas 99 Dec 31 1969 repositories drwxrwxr-x. 2 jogas jogas 6 Aug 14 08:01 sys -rw-rw-r--. 1 jogas jogas 40 Aug 14 08:01 WEIRD_AL_YANKOVIC ```

If you wanted to create charliecloud compatible tarballs (flattened filesystem images) , you'd want to use docker create in conjunction with docker export:

``` $ sudo docker export $(sudo docker create --read-only mpihello-openmpi) > mpihello.tar $ mkdir mpihello && mv mpihello.tar mpihello $ cd mpihello && tar xf mpihello.tar $ mv mpihello.tar ../ total 12K drwxr-xr-x. 2 jogas jogas 4.0K Aug 5 11:24 bin drwxr-xr-x. 2 jogas jogas 6 Mar 28 03:12 boot drwxr-xr-x. 4 jogas jogas 43 Aug 14 08:18 dev drwxr-xr-x. 48 jogas jogas 4.0K Aug 14 08:18 etc drwxr-xr-x. 2 jogas jogas 133 Aug 5 12:16 hello drwxr-xr-x. 2 jogas jogas 6 Mar 28 03:12 home drwxr-xr-x. 9 jogas jogas 125 Aug 5 11:24 lib drwxr-xr-x. 2 jogas jogas 34 Jul 7 21:30 lib64 drwxr-xr-x. 2 jogas jogas 6 Jul 7 21:30 media drwxr-xr-x. 3 jogas jogas 15 Aug 5 11:27 mnt drwxr-xr-x. 2 jogas jogas 6 Jul 7 21:30 opt drwxr-xr-x. 2 jogas jogas 6 Mar 28 03:12 proc drwx------. 2 jogas jogas 37 Jul 7 21:30 root drwxr-xr-x. 3 jogas jogas 30 Jul 7 21:30 run drwxr-xr-x. 2 jogas jogas 4.0K Aug 5 11:24 sbin drwxr-xr-x. 2 jogas jogas 6 Jul 7 21:30 srv drwxr-xr-x. 2 jogas jogas 6 Mar 28 03:12 sys drwxrwxr-x. 2 jogas jogas 6 Aug 5 12:16 tmp drwxr-xr-x. 10 jogas jogas 105 Jul 7 21:30 usr drwxr-xr-x. 11 jogas jogas 139 Jul 7 21:30 var $ ch-run . -- ./hello/hello 0: MPI version: Open MPI v3.1.4, package: Open MPI root@89edfcf5d38f Distribution, ident: 3.1.4, repo rev: v3.1.4, Apr 15, 2019 0: init ok, 1 ranks, userns 4026532427 0: send/receive ok 0: finalize ok ```

Hope this helps.

[le-ander]

This is great, thanks a ton!

[reidpr]

I'll add that there's no fundamental reason that the building-related scripts could not run on MacOS, so if you're interested in writing a patch, we should discuss. It is running containers (ch-run and the scripts that call it) that requires Linux system calls.

[le-ander]

I see, I think however that this beyond my capabilities for now.

[pagrubel]

This will only work for those who have sudo on their mac, unfortunately.

[rstyd]

@pagrubel You should be able to use ch-grow on mac. Though, I don't think that's been tested.

[tylerjereddy]

At the time of writing it looks like ./configure on the master branch will fail on mac OS because configure: error: shm_open(3) not found. It looks like the configure script assumes that -lrt is the linking source for shm_open, but that is only true on Linux, and not BSD/Mac. spack runs into the same issue rather than telling me that MacOS is not supported when I try to install charliecloud package. Probably useful to raise an informative error in spack recipe for now, when building on Mac.

If there are broader issues with performing basic builds on a Mac, I won't try to open a PR to fix that though.

[reidpr]

@tylerjereddy The namespace system calls we need aren't available on Mac, so Charliecloud definitely can't run containers on a Mac. Building containers may be plausible. I'd certainly be open to a PR that fails configure with a better error message on non-Linux systems, as well as the Spack recipe. A more extensive PR to enable building containers on a Mac would need more discussion, I think.

[reidpr]

It turns out that we could build images on MacOS (via Docker but not ch-grow because it needs ch-run, which uses Linux system calls), and all/most of the shell scripts should work. So, we can revisit this.

[rstyd]

The current plan is to add support for building images via docker and pushing images via ch-grow push on MacOS. Will also require modifying configure.ac to support MacOS builds.

[bio-la]

Hello, any update on this? is the suggested VM still the recommended route for creating a tarball from docker image on Mac? thanks!

[reidpr]

@bio-la, thanks for asking. At present a Linux VM is the only way to do Charliecloud on MacOS. The Python and shell parts of Charliecloud should work on Macs, aside from there surely being showstopper bugs that need to be fixed, but we haven't been able to prioritize it. Patches welcome, of course.