Install linux distribution-specific versions

[p-mongo]

This is needed for enterprise server versions to reference the correct libsnmp binary, for instance.

[stennie]

@p-mongo The Linux distro is generally useful to detect and use for both community & enterprise builds as the distro-specific MongoDB tarballs can include additional dependencies that are not in the generic 64-bit Linux legacy MongoDB builds (for example, TLS/SSL). I think it would be better to add distro detection with a fallback to generic builds if a distro tarball doesn't exist.

There may also be a more generic approach (perhaps an lsb_release incantation) to streamline detection.

Would you be interested in creating a more generic PR to detect the distro version?

Regards, Stennie

[p-mongo]

OK will work on that.

[p-mongo]

Neither suse 12 (via evergreen) nor opensuse 13 (via https://app.vagrantup.com/bento) nor amazon linux have lsb_release on them, apparently.

[p-mongo]

I am only going to handle distros listed on https://www.mongodb.com/download-center/enterprise/releases/archive. Other users can send PRs to handle their systems (for example, I'm not going to do anything for Fedora seeing how rhel and centos are detected in entirely different ways).

[p-mongo]

Handy reference courtesy of Brian: https://github.com/chef/ohai/blob/master/lib/ohai/plugins/linux/platform.rb

[p-mongo]

@stennie Tested this on:

• redhat62
• suse12
• amazon
• debian71

Added non-enterprise support.

I think this is usable as is. The one caveat is if there isn't an exact match between host OS and what mongodb is built for and e.g. suse11 package is installed on suse12, this isn't guaranteed to work.

[stennie]

Thanks @p-mongo .. looks great! Will sanity check on supported distros before merging.

Regards, Stennie

[p-mongo]

If you can make these changes that would be my preference, thanks.

[devkev]

Sure, no problem.

I carefully checked the version compatibility against a variety of Docker-based distros. Results summary, (alternate view). The testing mechanism was to use this check-all-mongodb-versions script:

#!/bin/bash

apt-get update
apt-get install -y openssl curl
apt-get install -y libcurl3
#apt-get install -y libcurl4
echo
echo

function runit {
    #"$v"/bin/mongod --version
    #"$v"/bin/mongod --version >/dev/null 2>&1
    output="$("$v"/bin/mongod --version 2>&1)"
}

cd /mnt/omnimongo/omnimongo/sw/linux/x86_64
for i in amazon* debian* rhel55 rhel62 rhel70 suse1* ubuntu1*; do
    for vbase in 3.0 3.2 3.4 3.6 4.0; do
        v="$(ls -dvr $i/"$vbase"*[0-9] 2>/dev/null | head -1)"
        if [ -x "$v"/bin/mongod ]; then
            if runit; then
                echo "$v YES"
            else
                echo "$v NO"
                echo "$output"
            fi
        else
            echo "$i/$vbase* -"
        fi
    done
    echo
done

and run it with:

for distro in amazonlinux:{1,2} debian:{7,8,9} centos:{5,6,7} opensuse:42.3 ubuntu:1{2,4,6,8}.04; do
    docker run --rm -v /mnt/omnimongo:/mnt/omnimongo:ro -v $PWD:/data "$distro" /data/check-all-mongodb-versions
done

Future versions tend to only work in older distros sporadically/rarely, so I've taken them out. The rhel70 default was just because there needed to be a default for Enterprise, I think (which has never had a "legacy" build). I've otherwise put in "backups" between rhel/amazon and ubuntu/debian where they seem to be reliable across MongoDB versions. I also added support for Amazon Linux 2.

@kevinadi, can you please test the updated code at https://github.com/devkev/m/tree/distros for me, and confirm that it works as expected?

[p-mongo]

The diff is looking good, it's clearer too except for the sed seas of quotes as would be expected.

If you are handling the ubuntu without lsb_release situation the comment saying it isn't handled should probably be removed.

[devkev]

Good catch, thanks - I've updated the comments accordingly.

[kevinadi]

I pushed a new branch on my fork at https://github.com/kevinadi/m/tree/distros which fixes some minor issues.

I tested the script and check to see if the installed mongod version matches the OS using docker images. Here are the results:

With lsb_release:

Distro	3.2.20	3.4.16	3.6.6	4.0.0	4.0.1
ubuntu-1204	debian71	generic	generic	generic	generic
ubuntu-1404	ubuntu1404	ubuntu1404	ubuntu1404	ubuntu1404	ubuntu1404
ubuntu-1604	ubuntu1604	ubuntu1604	ubuntu1604	(x) ubuntu1604	(x) ubuntu1604
ubuntu-1804	(x) ubuntu1604	(x) ubuntu1604	(x) ubuntu1604	(x) ubuntu1604	ubuntu1804
debian-7	debian71	generic	generic	generic	generic
debian-8	debian81	debian81	debian81	debian81	debian81
debian-9	(x) debian81	(x) debian81	debian92	debian92	debian92
centos-6	rhel62	rhel62	rhel62	rhel62	rhel62
centos-7	rhel70	rhel70	rhel70	rhel70	rhel70
opensuse-leap	suse12	suse12	suse12	suse12	suse12
opensuse-tumbleweed	suse12	suse12	suse12	suse12	suse12
amazonlinux-1	generic	generic	generic	generic	generic
amazonlinux-2	generic	generic	generic	generic	generic

(x) doesn't work out of the box. Requires missing mongod dependencies (libcurl3, libssl.so.1.0.0, glibc, etc.)

Amazon Linux 1 lsb_release outputs AmazonAMI, but identifies itself properly in /etc/os-release Amazon Linux 2 lsb_release outputs n/a, but identifies itself properly in /etc/os-release

Without lsb_release:

Distro	3.2.20	3.4.16	3.6.6	4.0.0	4.0.1
ubuntu-1204	debian71	generic	generic	generic	generic
ubuntu-1404	ubuntu1404	ubuntu1404	ubuntu1404	ubuntu1404	ubuntu1404
ubuntu-1604	ubuntu1604	ubuntu1604	ubuntu1604	(x) ubuntu1604	(x) ubuntu1604
ubuntu-1804	(x) ubuntu1604	(x) ubuntu1604	(x) ubuntu1604	(x) ubuntu1604	ubuntu1804
debian-7	debian71	generic	generic	generic	generic
debian-8	debian81	debian81	debian81	debian81	debian81
debian-9	(x) debian81	(x) debian81	debian92	debian92	debian92
centos-6	generic	generic	generic	generic	generic
centos-7	rhel70	rhel70	rhel70	rhel70	rhel70
opensuse-leap	generic	generic	generic	generic	generic
opensuse-tumbleweed	generic	generic	generic	generic	generic
amazonlinux-1	amazon	amazon	amazon	(x) amazon2	(x) amazon2
amazonlinux-2	amazon	amazon	amazon	amazon2	amazon2

(x) doesn't work out of the box. Requires missing mongod dependencies (libcurl3, libssl.so.1.0.0, glibc, etc.)

Missing m requirements from official Docker images:

Distro	Missing requirements
ubuntu-1204	curl
ubuntu-1404	curl
ubuntu-1604	curl
ubuntu-1804	curl
debian-7	curl
debian-8	curl
debian-9	curl
centos-6	curl which
centos-7	curl which
opensuse-leap	curl which tar (libopenssl1_0_0)
opensuse-tumbleweed	curl which tar (libopenssl1_0_0)
amazonlinux-1	which tar gzip
amazonlinux-2	which tar gzip

@devkev @stennie please review the results above. Is there anything else that we can do to make the experience better? Some docker images provides a very bare bone installation that lacks basic tools such as which, tar, or gzip, but there's not much we can do about them.

[devkev]

That Amazon Linux lsb_release behaviour is really annoying.

[devkev]

I reviewed the commit on your branch, and left 2 comments.

[kevinadi]

Addressed the comments in my branch. Rerun the tests just in case. Verified that all is well and the test results are unchanged from the earlier table I posted.

[devkev]

Yep this looks good now, thanks! We can move forward to pushing the extra commits onto @p-mongo's branch, or just creating another PR, and then merging - assuming that this is ok with @stennie?

[p-mongo]

No love for debian 10?

[stennie]

Thanks for everyone's help testing the Linux distro pantheon! @kevinadi included all the changes from @p-mongo and @devkev in PR #40. I'm going to update the README to mention the new --legacy option and we'll ship this as a 1.5.0 release given the new distro magic.

No love for debian 10?

Debian 10 isn't GA yet, so there aren't any associated distro-specific MongoDB packages to test.

Regards, Stennie