using Libcrypto 3.x isntead of 1.x

[ario-afrashteh]

I have updated flapdoodle to 4.7.0 and it requieres libcrypto.so.1.1 which is old and (including 1.1.0, 1.0.2, 1.0.0 and 0.9.8) are now out of support and should not be used. Please check openssl.org.

[michaelmosmann]

@ario-afrashteh which mongodb version and platform are you using?

[ario-afrashteh]

@michaelmosmann I'm using Ubuntu 18.04 LTS. Libcrypto 3 is installed there, everything is fine on my local macos which I also have openssl 3! I have used below dependencies:

<dependency>
      <groupId>de.flapdoodle.embed</groupId>
      <artifactId>de.flapdoodle.embed.mongo</artifactId>
      <version>4.7.0</version>
      <scope>test</scope>
  </dependency>
  <dependency>
      <groupId>de.flapdoodle.embed</groupId>
      <artifactId>de.flapdoodle.embed.mongo.spring30x</artifactId>
      <version>4.7.0</version>
  </dependency>

And also this config in my test properties file (I also tried 6.0.6 and it has the same problem):

de:
  flapdoodle:
    mongodb:
      embedded:
        version: 5.0.18

[michaelmosmann]

@ario-afrashteh as there are binaries for ubuntu 18.04 (https://github.com/flapdoodle-oss/de.flapdoodle.embed.mongo.packageresolver/blob/b86a0fac90241375ba27c53481766e8eba805439/src/test/resources/de/flapdoodle/embed/mongo/packageresolver/explainedSnapshot.txt#L61) i guess that these versions are using libcrypto1.x .. you can try a version for a newer ubuntu release if you override the os detection (see https://github.com/flapdoodle-oss/de.flapdoodle.os#run) to see if this works.. but i am not sure against which version these binaries are linked.

[ario-afrashteh]

I added these configs as you said:

de:
  flapdoodle:
    mongodb:
      embedded:
        version: 6.0.6
    os:
      explain: true
      override: Linux|X86_64|Ubuntu|Ubuntu_18_04

I got this error: 2023-07-06T08:08:07.592Z ERROR 865 --- *** Thread-5*** d.f.e.m.s.a.EmbeddedMongo :mongod: error while loading shared libraries: libcrypto.so.1.1: cannot open shared object file: No such file or directory

[michaelmosmann]

@ario-afrashteh i am not sure if you can use it this way.. as i am not sure if this ends in an system property.. (see https://github.com/flapdoodle-oss/de.flapdoodle.os-api/blob/f01641c79dd6a561d15c2ae654f2a63a00d29535/src/main/java/de/flapdoodle/os/Platform.java#L91)

btw i think you should try a release for a newer ubuntu version bc you upgraded your libcrypto-stuff.. so it should be: 'Linux|X86_64|Ubuntu|Ubuntu_20_04' or even 'Linux|X86_64|Ubuntu|Ubuntu_22_04'

[XSpielinbox]

System properties only are set for a specific JVM. They are not read from the applications.properties in my case. Providing them via mvn -Dproperty=value also does not work for my Spring boot application, as apparently this all does not reach the right JVM. When wanting to add a property to my application run I have to use mvn spring-boot:run -Dspring-boot.run.jvmArguments="-Dde.flapdoodle.os.override=Linux|X86_64|CentOS|CentOS_9". (of course adapt to OS you are using)

For usage in tests I have to add the following to my pom.xml:

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>3.1.2</version>
                <configuration>
                    <systemPropertyVariables>
                        <de.flapdoodle.os.override>Linux|X86_64|CentOS|CentOS_9</de.flapdoodle.os.override>
                    </systemPropertyVariables>
                </configuration>
            </plugin>

According to the official MongoDB documentation, MongoDB 6.x supports Ubuntu 18.04. Ubuntu 18.04 ships with OpenSSL 1.1.1 and does not provide OpenSSL 3.x. MongoDB only added support for OpenSSL 3.x in 6.0.4. MongoDB therefore very likely does not support OpenSSL 3.x on Ubuntu 18.04.

On another note: support for Ubuntu 18.04 ended 1 month and 3 weeks ago (31 May 2023). I would therefore recommend you to switch to a newer release of Ubuntu.

de.flapdoodle.embed.mongo 4.7.1 works absolutely fine with MongoDB 6.0.6 and OpenSSL 3.x for me on Fedora 38 when setting above system properties.

[michaelmosmann]

note to myself: remove older mongodb versions from ubuntu >=22.xx

[sloppycoder]

I ran into the same problem on PopOS 22.04 (derived from Ubuntu 22.04) when using de.flapdoodle.embed.mongo.spring30x version 4.7.0.

Appnarelty the OS detection logic thinks it's running on Ubuntu 20.04 instead of 22.04. The workaround is to override

<systemPropertyVariables>
     <de.flapdoodle.os.override>Linux|X86_64|Ubuntu|Ubuntu_22_04</de.flapdoodle.os.override>
</systemPropertyVariables>

Download site https://fastdl.mongodb.org/linux have correct binaries for both Ubuntu 20.04 and 22.04. Once the OS detection recogznie Ubuntu 22 it should work fine.

# 20.04 binaries are linked with ssl 1.1 and ssl3
$ ldd mongodb-linux-x86_64-ubuntu2004-7.0.0-rc1/bin/mongod
    linux-vdso.so.1 (0x00007ffe653d4000)
         ...
    libcrypto.so.1.1 => not found
    libssl.so.1.1 => not found
       ...
        libssl.so.3 => /lib/x86_64-linux-gnu/libssl.so.3 (0x00007fb79a871000)

# 22.04 binary are only linked with ssl3
$ ldd mongodb-linux-x86_64-ubuntu2204-7.0.0-rc1/bin/mongod
    linux-vdso.so.1 (0x00007ffe653d4000)
        ...
        libssl.so.3 => /lib/x86_64-linux-gnu/libssl.so.3 (0x00007fb79a871000)

[michaelmosmann]

@sloppycoder can you provide me with the content of the /etc/os-release file?

[sloppycoder]

$ cat /etc/os-release NAME="Pop!_OS" VERSION="22.04 LTS" ID=pop ID_LIKE="ubuntu debian" PRETTY_NAME="Pop!_OS 22.04 LTS" VERSION_ID="22.04" HOME_URL="https://pop.system76.com" SUPPORT_URL="https://support.system76.com" BUG_REPORT_URL="https://github.com/pop-os/pop/issues" PRIVACY_POLICY_URL="https://system76.com/privacy" VERSION_CODENAME=jammy UBUNTU_CODENAME=jammy LOGO=distributor-logo-pop-os

[michaelmosmann]

@sloppycoder i guess that it works now is that there is a fallback to ubuntu if nothing works.. but i will add pop_os to the detection as i already had for Linux Mint - https://github.com/flapdoodle-oss/de.flapdoodle.os/blob/main/src/main/java/de/flapdoodle/os/linux/LinuxMintVersion.java

[sloppycoder]

I just tried to run the same build on Debian 12, got same problem. it tries to download binary for Debian 11

(devbox) lee@gf63 ~/Projects/git/vino9org/vinobank-java-base (main) $ cat /etc/os-release PRETTY_NAME="Debian GNU/Linux 12 (bookworm)" NAME="Debian GNU/Linux" VERSION_ID="12" VERSION="12 (bookworm)" VERSION_CODENAME=bookworm ID=debian HOME_URL="https://www.debian.org/" SUPPORT_URL="https://www.debian.org/support" BUG_REPORT_URL="https://bugs.debian.org/"

[XSpielinbox]

MongoDB only supports MongoDB 7 on Debian 12, but does not even offer downloads for Debian 12 yet. As always, MongoDB unfortunately is quite slow in adapting new OS versions, documenting these changes and updating the downloads...

[sloppycoder]

yeah. after I posted this I tried to set de.flapdoodle.os.explain to true and ran again. I found that detection works on Debian 12 but there's no binary to download.

2023-08-16T23:51:06.389+08:00 INFO 10517 --- [ main] de.flapdoodle.os.Platform : Platform.detect() -> Platform{operatingSystem=Linux, architecture=X86_64, distribution=Debian, version=DEBIAN_12}

On PopOS 22.04, the detection didn't work correctly.

2023-08-17T00:01:47.210+08:00 INFO 5181 --- [ main] de.flapdoodle.os.Platform : Platform.detect() -> Platform{operatingSystem=Linux, architecture=X86_64}

Looking forward to a fix soon.

There're quite a few Ubuntu variant out there, perhaps can tacle the whole bunch of tweaking of the Ubuntu detection instead of creating a new Enum of each variant?

[michaelmosmann]

@sloppycoder tweaking is slightly harder then just one more enum:)

[michaelmosmann]

@sloppycoder there is a new release (in maven central in about 12hrs i think) which popOs-Detection.. on debian12 the debian11 version should be used..

[sloppycoder]

Wow..that was fast. Thanks a lot for the quick turnaround.

On Debian 12, the Debian 11 binary actually didn't work for me . The Ubuntu 22 binary worked though...

MongoDB build system is rather messed up it seems....

[XSpielinbox]

MongoDB build system is rather messed up it seems....

Yes, but

On Debian 12, the Debian 11 binary actually didn't work for me . The Ubuntu 22 binary worked though...

this is to be expected, as Debian 12 and Ubuntu 22.04 only ship OpenSSL 3.0.x, whereas Debian 11 and Ubuntu 20.04 only ship OpenSSL 1.1.1x.

[michaelmosmann]

@sloppycoder @XSpielinbox oh.. ok. I will try to fake ubuntu22.x versions for debian12 .. hmm..

[michaelmosmann]

@XSpielinbox which mongodb version did you use?

[XSpielinbox]

When and for what do you mean?

On my Fedora machine I use the lastest MongoDB 6.0.x for RHEL 9 as stated last month, as there is no version for Fedora and Fedora 38 as Debian 12, Ubuntu 22.04 and RHEL 9 only ships OpenSSL 3.0.x (at least by default) and therefore the default fallback to Ubuntu 20.04 does not work, as that does not support OpenSSL 3.0.x, but whether one is using the Ubuntu 22.04 or RHEL 9.0 version should not really make a difference as both support OpenSSL 3.0.x .

@sloppycoder stated yesterday that the Ubuntu 22.04 version works on Debian 12.

[michaelmosmann]

@XSpielinbox so i guess you are using mongodb >= 6.0.4 because that's the first version for ubuntu 22.xx.. right?

[michaelmosmann]

@XSpielinbox Fedora 38 should now be mapped to the redhat 9 version...

[XSpielinbox]

so i guess you are using mongodb >= 6.0.4 because that's the first version for ubuntu 22.xx.. right?

Yes, of course. It's the same for Ubuntu 22.04 as for RHEL 9. Interestingly enough though, I just noticed, that MongoDB offers downloads for 6.0.3 too - though they should be non-functional then? (I haven't tried it and don't care to much, it's just another thing from MongoDB that makes no sense at all.)

Fedora 38 should now be mapped to the redhat 9 version...

Where can I find the code for that?

[michaelmosmann]

@XSpielinbox https://github.com/flapdoodle-oss/de.flapdoodle.embed.mongo.packageresolver/blob/c63f0ca29e3f7f05fe81617006571561434dacc3/src/main/java/de/flapdoodle/embed/mongo/packageresolver/linux/CentosRedhatPackageFinder.java#L74

[michaelmosmann]

@sloppycoder @XSpielinbox debian12 should now use the ubuntu 22.x version if there is any..

[XSpielinbox]

https://github.com/flapdoodle-oss/de.flapdoodle.embed.mongo.packageresolver/blob/c63f0ca29e3f7f05fe81617006571561434dacc3/src/main/java/de/flapdoodle/embed/mongo/packageresolver/linux/CentosRedhatPackageFinder.java#L74

Ah, so it is there already for two weeks! I thought, you just committed that change yesterday but could not find anything in yesterdays commits.

debian12 should now use the ubuntu 22.x version if there is any..

Sounds great : )

[michaelmosmann]

@XSpielinbox sorry for that.. to make this release i had to change things in multiple projects .. so i lost a little bit when i did things..

the downside of moving things out for code reusage.. :)

[erinyq712]

Edited: It is still possible to get this issue if you try to embed a version of MongoDB that is not supported by your OS. This is not a bug, but can be good to know. I tried to use 4.0.12 with Ubuntu 22 and then you get the version for Ubuntu 18 which will not work: 2024-03-14 07:36:35.154 [main] INFO d.f.e.m.s.a.EmbeddedMongo - download https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-ubuntu1804-4.0.12.tgz : finished 2024-03-14 07:36:36.600 [Thread-4] ERROR d.f.e.m.s.a.EmbeddedMongo - /root/.embedmongo/fileSets/de0c2b8f50ba7d3bc96e039e3d158b6c0e6f88d66dfc78533601ff522286a91b/mongod: error while loading shared libraries: libcrypto.so.1.1: cannot open shared object file: No such file or directory I fixed this issue by instead using version 6.3.2, but you can use newer versions if you like.

[michaelmosmann]

@erinyq712 you should use a newer version of mongodb .. i guess you can not run a mongodb 4.0.x on a newer ubuntu.. (see https://github.com/flapdoodle-oss/de.flapdoodle.embed.mongo.packageresolver/blob/main/src/test/resources/de/flapdoodle/embed/mongo/packageresolver/explainedSnapshot.txt)

[erinyq712]

Thanks Michael. That was the issue. I updated my comment to hide my initial lack of knowledge.

[michaelmosmann]

@erinyq712 .. i would not consider it as a lack of knowledge.. :) it is just more complicated than it should be..