Open danielpza opened 1 year ago
hasezoey
commented
1 year ago could you try some versions from the following link and report back which versions work? (like ubuntu / debian or rhel versions) https://www.mongodb.com/download-center/community/releases/archive
danielpza
commented
11 months ago Hey @hasezoey, sorry for the late reply, this one worked for me:
MONGOMS_DOWNLOAD_URL=https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-ubuntu2204-6.0.9.tgzHey @hasezoey, sorry for the late reply, this one worked for me:
thanks for the reply, i am a little unsure if ubuntu package compatability is with just this version, or multiple versions (across major mongodb versions), so for now i think i will only add at most a documentation entry noting that ubuntu packages may work (linking to this thread)
@danielpza are you still using the same nixos version? if not, could you post a updated detection output?
PS: for anyone coming across this issue in the future, since 8.15.0 a DISTRO config option was added, with this you can overwrite the distro detected
danielpza
commented
11 months ago These ones I've tested so far:
| Distro | 7.0.0 | 6.0.9 | 5.0.20 |
|---|---|---|---|
| debian10 | - | x | x |
| debian11 | x | x | x |
| ubuntu1804 | - | ✔ | x |
| ubuntu2004 | x | ✔ | x |
| ubuntu2204 | x | ✔ | x |
Instance failed to start because a library is missing or cannot be opened: "libcrypto.so.1.1"Instance closed unexpectedly with code "null" and signal "SIGSEGV"Instance closed unexpectedly with code "100" and signal "null"Instance failed to start because a library is missing or cannot be opened: "libcrypto.so.1.1"danielpza
commented
11 months ago @danielpza are you still using the same nixos version? if not, could you post a updated detection output?
Yeah, I'm stilling using the same version (23.11)
``` $ cat /etc/upstream-release/lsb-release '/etc/upstream-release/lsb-release': No such file or directory (os error 2) $ cat /etc/os-release BUG_REPORT_URL="https://github.com/NixOS/nixpkgs/issues" BUILD_ID="23.11pre527133.5ba549eafcf3" DOCUMENTATION_URL="https://nixos.org/learn.html" HOME_URL="https://nixos.org/" ID=nixos LOGO="nix-snowflake" NAME=NixOS PRETTY_NAME="NixOS 23.11 (Tapir)" SUPPORT_URL="https://nixos.org/community.html" VERSION="23.11 (Tapir)" VERSION_CODENAME=tapir VERSION_ID="23.11" $ cat /usr/lib/os-release '/usr/lib/os-release': No such file or directory (os error 2) $ cat /etc/lsb-release DISTRIB_CODENAME=tapir DISTRIB_DESCRIPTION="NixOS 23.11 (Tapir)" DISTRIB_ID=nixos DISTRIB_RELEASE="23.11" LSB_VERSION="23.11 (Tapir)" ```
hasezoey
commented
11 months ago These ones I've tested so far:
just to confirm - means "not tested", x means "tested, but does not work", and ✔ means "tested and works"?
and the distro you have set via the DISTRO option, or are you using that distro / somehow emulating it (or something else)?
for mongodb version 7.0.0 on ubuntu I got the error Instance closed unexpectedly with code "100" and signal "null"
if i remember correctly, exit code 100 means that it couldnt find the database directory (ie it is "working" but couldnt find a user-defined directory), would need more of the log to investigate
danielpza
commented
11 months ago just to confirm - means "not tested", x means "tested, but does not work", and ✔ means "tested and works"?
yup, exactly. I couldn't find the builds for debian10 and ubuntu 18.04 for mongodb 7, I think they are not supported
and the distro you have set via the DISTRO option, or are you using that distro / somehow emulating it (or something else)?
I'm using the MONGOMS_DOWNLOAD_URL env variable
danielpza
commented
11 months ago if i remember correctly, exit code 100 means that it couldnt find the database directory (ie it is "working" but couldnt find a user-defined directory), would need more of the log to investigate
More logs with MONGOMS_DEBUG=1 for ubuntu 20.04, mongodb 7.0.0:
...
MongoMS:MongoInstance Mongo[37743]: stderrHandler: ""/home/daniel/.cache/mongodb-binaries/mongod-x86_64-ubuntu-7.0.0: /run/current-system/sw/share/nix-ld/lib/libcurl.so.4: no version information available (required by /home/daniel/.cache/mongodb-binaries/mongod-x86_64-ubuntu-7.0.0)"" +0ms
...
MongoMS:MongoMemoryServer Mongo[unknown]: _startUpInstance threw a Error: StdoutInstanceError: Instance Failed to start with "DBException in initAndListen". Original Error:
Location18656: Cannot start server with an unknown storage engine: ephemeralForTest
...``` MONGOMS_DEBUG=1 MONGOMS_DOWNLOAD_URL=https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-ubuntu2004-7.0.0.tgz yarn test ``` ``` Determining test suites to run... MongoMS:ResolveConfig Debug Mode Enabled, through Environment Variable +0ms MongoMS:ResolveConfig findPackageJson: Found package.json at "/home/daniel/project/package.json" +1ms MongoMS:MongoMemoryServer Mongo[unknown]: start: Called .start() method +0ms MongoMS:MongoMemoryServer Mongo[unknown]: _startUpInstance: Called MongoMemoryServer._startUpInstance() method +0ms MongoMS:MongoMemoryServer Mongo[unknown]: getStartOptions: forceSamePort: false +1ms MongoMS:MongoMemoryServer Mongo[unknown]: _startUpInstance: Creating new MongoDB instance with options: { instance: { port: 37743, dbName: '', ip: '127.0.0.1', storageEngine: 'ephemeralForTest', replSet: undefined, dbPath: '/tmp/mongo-mem-EzuW1b', tmpDir: '/tmp/mongo-mem-EzuW1b', keyfileLocation: undefined, launchTimeout: undefined, args: undefined, auth: false }, binary: undefined, spawn: undefined } +2ms MongoMS:MongoInstance create: Called .create() method +0ms MongoMS:MongoInstance Mongo[37743]: start +1ms MongoMS:MongoBinary getPath +0ms MongoMS:DryMongoBinary generateOptions +0ms MongoMS:utils tryReleaseFile: "/etc/upstream-release/lsb-release" does not exist +0ms MongoMS:getos getLinuxInformation: Using etcOsRelease +0ms MongoMS:DryMongoBinary generateDownloadPath: Generating Download Path, preferGlobal: "true" +1ms MongoMS:DryMongoBinary generatePaths { version: '5.0.18', downloadDir: '', os: { os: 'linux', dist: 'nixos', codename: 'tapir', release: '23.11', id_like: undefined }, platform: 'linux', arch: 'x64', systemBinary: '' } +1ms MongoMS:DryMongoBinary getBinaryName +0ms MongoMS:DryMongoBinary combineBinaryName +0ms MongoMS:DryMongoBinary combineBinaryName +0ms MongoMS:DryMongoBinary combineBinaryName +0ms MongoMS:DryMongoBinary generateDownloadPath: Paths: { legacyHomeCache: '/home/daniel/.cache/mongodb-binaries/mongod-x64-nixos-5.0.18', modulesCache: '/home/daniel/project/node_modules/.cache/mongodb-memory-server/mongod-x64-nixos-5.0.18', relative: '/home/daniel/project/mongodb-binaries/mongod-x64-nixos-5.0.18', resolveConfig: '' } +0ms MongoMS:DryMongoBinary generateDownloadPath: no existing binary for version "5.0.18" was found +1ms MongoMS:DryMongoBinary generateDownloadPath: using global (preferGlobal) "/home/daniel/.cache/mongodb-binaries/mongod-x64-nixos-5.0.18" +0ms MongoMS:DryMongoBinary generateOptions: ARCHIVE_NAME or DOWNLOAD_URL defined, generating options based on that (input: "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-ubuntu2004-7.0.0.tgz") +0ms MongoMS:DryMongoBinary parseArchiveNameRegex (input: "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-ubuntu2004-7.0.0.tgz") +0ms MongoMS:MongoBinary getPath: MongoBinary options: { "version": "7.0.0", "downloadDir": "/home/daniel/.cache/mongodb-binaries", "os": { "os": "linux", "dist": "ubuntu", "release": "" }, "platform": "linux", "arch": "x86_64", "systemBinary": "", "checkMD5": false } +3ms MongoMS:DryMongoBinary locateBinary: Trying to locate Binary for version "7.0.0" +0ms MongoMS:DryMongoBinary generateOptions +0ms MongoMS:DryMongoBinary generateDownloadPath: Generating Download Path, preferGlobal: "true" +1ms MongoMS:DryMongoBinary generatePaths { version: '7.0.0', downloadDir: '/home/daniel/.cache/mongodb-binaries', os: { os: 'linux', dist: 'ubuntu', release: '' }, platform: 'linux', arch: 'x86_64', systemBinary: '' } +0ms MongoMS:DryMongoBinary getBinaryName +0ms MongoMS:DryMongoBinary combineBinaryName +0ms MongoMS:DryMongoBinary combineBinaryName +0ms MongoMS:DryMongoBinary generatePaths: resolveConfigValue is not empty +0ms MongoMS:DryMongoBinary combineBinaryName +0ms MongoMS:DryMongoBinary combineBinaryName +0ms MongoMS:DryMongoBinary generateDownloadPath: Paths: { legacyHomeCache: '/home/daniel/.cache/mongodb-binaries/mongod-x86_64-ubuntu-7.0.0', modulesCache: '/home/daniel/project/node_modules/.cache/mongodb-memory-server/mongod-x86_64-ubuntu-7.0.0', relative: '/home/daniel/project/mongodb-binaries/mongod-x86_64-ubuntu-7.0.0', resolveConfig: '/home/daniel/.cache/mongodb-binaries/mongod-x86_64-ubuntu-7.0.0' } +0ms MongoMS:DryMongoBinary generateDownloadPath: Found binary in resolveConfig (DOWNLOAD_DIR): "/home/daniel/.cache/mongodb-binaries/mongod-x86_64-ubuntu-7.0.0" +0ms MongoMS:DryMongoBinary generateOptions: ARCHIVE_NAME or DOWNLOAD_URL defined, generating options based on that (input: "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-ubuntu2004-7.0.0.tgz") +0ms MongoMS:DryMongoBinary parseArchiveNameRegex (input: "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-ubuntu2004-7.0.0.tgz") +0ms MongoMS:DryMongoBinary locateBinary: running generateDownloadPath +0ms MongoMS:DryMongoBinary generateDownloadPath: Generating Download Path, preferGlobal: "true" +0ms MongoMS:DryMongoBinary generatePaths { version: '7.0.0', downloadDir: '/home/daniel/.cache/mongodb-binaries', os: { os: 'linux', dist: 'ubuntu', release: '' }, platform: 'linux', arch: 'x86_64', systemBinary: '' } +0ms MongoMS:DryMongoBinary getBinaryName +0ms MongoMS:DryMongoBinary combineBinaryName +0ms MongoMS:DryMongoBinary combineBinaryName +0ms MongoMS:DryMongoBinary generatePaths: resolveConfigValue is not empty +0ms MongoMS:DryMongoBinary combineBinaryName +1ms MongoMS:DryMongoBinary combineBinaryName +0ms MongoMS:DryMongoBinary generateDownloadPath: Paths: { legacyHomeCache: '/home/daniel/.cache/mongodb-binaries/mongod-x86_64-ubuntu-7.0.0', modulesCache: '/home/daniel/project/node_modules/.cache/mongodb-memory-server/mongod-x86_64-ubuntu-7.0.0', relative: '/home/daniel/project/mongodb-binaries/mongod-x86_64-ubuntu-7.0.0', resolveConfig: '/home/daniel/.cache/mongodb-binaries/mongod-x86_64-ubuntu-7.0.0' } +0ms MongoMS:DryMongoBinary generateDownloadPath: Found binary in resolveConfig (DOWNLOAD_DIR): "/home/daniel/.cache/mongodb-binaries/mongod-x86_64-ubuntu-7.0.0" +0ms MongoMS:DryMongoBinary locateBinary: found binary at "/home/daniel/.cache/mongodb-binaries/mongod-x86_64-ubuntu-7.0.0" +0ms MongoMS:MongoBinary getPath: Mongod binary path: "/home/daniel/.cache/mongodb-binaries/mongod-x86_64-ubuntu-7.0.0" +2ms MongoMS:MongoInstance Mongo[37743]: start: Starting Processes +5ms MongoMS:MongoInstance Mongo[37743]: _launchMongod: Launching Mongod Process +0ms MongoMS:MongoInstance Mongo[37743]: prepareCommandArgs +0ms MongoMS:MongoInstance Mongo[37743]: prepareCommandArgs: final argument array:["--port","37743","--dbpath","/tmp/mongo-mem-EzuW1b","--storageEngine","ephemeralForTest","--bind_ip","127.0.0.1","--noauth"] +0ms MongoMS:MongoInstance Mongo[37743]: _launchKiller: Launching Killer Process (parent: 419051, child: 419071) +18ms MongoMS:MongoInstance Mongo[37743]: stdoutHandler: ""{"t":{"$date":"2023-09-25T16:32:35.961-04:00"},"s":"I", "c":"NETWORK", "id":4915701, "ctx":"main","msg":"Initialized wire specification","attr":{"spec":{"incomingExternalClient":{"minWireVersion":0,"maxWireVersion":21},"incomingInternalClient":{"minWireVersion":0,"maxWireVersion":21},"outgoing":{"minWireVersion":6,"maxWireVersion":21},"isInternalClient":true}}} MongoMS:MongoInstance {"t":{"$date":"2023-09-25T16:32:35.962-04:00"},"s":"I", "c":"CONTROL", "id":23285, "ctx":"main","msg":"Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'"}"" +18ms MongoMS:MongoInstance Mongo[37743]: stderrHandler: ""/home/daniel/.cache/mongodb-binaries/mongod-x86_64-ubuntu-7.0.0: /run/current-system/sw/share/nix-ld/lib/libcurl.so.4: no version information available (required by /home/daniel/.cache/mongodb-binaries/mongod-x86_64-ubuntu-7.0.0)"" +0ms MongoMS:MongoInstance Mongo[37743]: stdoutHandler: ""{"t":{"$date":"2023-09-25T16:32:35.963-04:00"},"s":"I", "c":"NETWORK", "id":4648601, "ctx":"main","msg":"Implicit TCP FastOpen unavailable. If TCP FastOpen is required, set tcpFastOpenServer, tcpFastOpenClient, and tcpFastOpenQueueSize."} MongoMS:MongoInstance {"t":{"$date":"2023-09-25T16:32:35.963-04:00"},"s":"I", "c":"REPL", "id":5123008, "ctx":"main","msg":"Successfully registered PrimaryOnlyService","attr":{"service":"TenantMigrationDonorService","namespace":"config.tenantMigrationDonors"}} MongoMS:MongoInstance {"t":{"$date":"2023-09-25T16:32:35.963-04:00"},"s":"I", "c":"REPL", "id":5123008, "ctx":"main","msg":"Successfully registered PrimaryOnlyService","attr":{"service":"TenantMigrationRecipientService","namespace":"config.tenantMigrationRecipients"}}"" +0ms MongoMS:MongoInstance Mongo[37743]: stdoutHandler: ""{"t":{"$date":"2023-09-25T16:32:35.963-04:00"},"s":"I", "c":"CONTROL", "id":5945603, "ctx":"main","msg":"Multi threading initialized"} MongoMS:MongoInstance {"t":{"$date":"2023-09-25T16:32:35.964-04:00"},"s":"I", "c":"TENANT_M", "id":7091600, "ctx":"main","msg":"Starting TenantMigrationAccessBlockerRegistry"}"" +1ms MongoMS:MongoInstance Mongo[37743]: stdoutHandler: ""{"t":{"$date":"2023-09-25T16:32:35.964-04:00"},"s":"I", "c":"CONTROL", "id":4615611, "ctx":"initandlisten","msg":"MongoDB starting","attr":{"pid":419071,"port":37743,"dbPath":"/tmp/mongo-mem-EzuW1b","architecture":"64-bit","host":"nixos"}}"" +0ms MongoMS:MongoInstance Mongo[37743]: stdoutHandler: ""{"t":{"$date":"2023-09-25T16:32:35.964-04:00"},"s":"I", "c":"CONTROL", "id":23403, "ctx":"initandlisten","msg":"Build Info","attr":{"buildInfo":{"version":"7.0.0","gitVersion":"37d84072b5c5b9fd723db5fa133fb202ad2317f1","openSSLVersion":"OpenSSL 3.0.10 1 Aug 2023","modules":[],"allocator":"tcmalloc","environment":{"distmod":"ubuntu2204","distarch":"x86_64","target_arch":"x86_64"}}}} MongoMS:MongoInstance {"t":{"$date":"2023-09-25T16:32:35.964-04:00"},"s":"I", "c":"CONTROL", "id":51765, "ctx":"initandlisten","msg":"Operating System","attr":{"os":{"name":"nixos","version":"\"23.11\""}}} MongoMS:MongoInstance {"t":{"$date":"2023-09-25T16:32:35.964-04:00"},"s":"I", "c":"CONTROL", "id":21951, "ctx":"initandlisten","msg":"Options set by command line","attr":{"options":{"net":{"bindIp":"127.0.0.1","port":37743},"security":{"authorization":"disabled"},"storage":{"dbPath":"/tmp/mongo-mem-EzuW1b","engine":"ephemeralForTest"}}}}"" +0ms MongoMS:MongoInstance Mongo[37743]: stdoutHandler: ""{"t":{"$date":"2023-09-25T16:32:35.964-04:00"},"s":"E", "c":"CONTROL", "id":20557, "ctx":"initandlisten","msg":"DBException in initAndListen, terminating","attr":{"error":"Location18656: Cannot start server with an unknown storage engine: ephemeralForTest"}}"" +0ms MongoMS:MongoInstance Mongo[37743]: constructor: Instance has thrown an Error: Error: Instance Failed to start with "DBException in initAndListen". Original Error: MongoMS:MongoInstance Location18656: Cannot start server with an unknown storage engine: ephemeralForTest +0ms MongoMS:MongoInstance Mongo[37743]: stop +1ms MongoMS:utils Mongo[37743] killProcess: mongodProcess: sending "SIGINT" +43ms MongoMS:MongoMemoryServer Mongo[unknown]: _startUpInstance threw a Error: StdoutInstanceError: Instance Failed to start with "DBException in initAndListen". Original Error: Location18656: Cannot start server with an unknown storage engine: ephemeralForTest at MongoInstance.checkErrorInLine (/home/daniel/project/node_modules/mongodb-memory-server-core/src/util/MongoInstance.ts:640:11) at MongoInstance.stdoutHandler (/home/daniel/project/node_modules/mongodb-memory-server-core/src/util/MongoInstance.ts:585:10) at Socket.emit (node:events:513:28) at Socket.emit (node:domain:489:12) at addChunk (node:internal/streams/readable:324:12) at readableAddChunk (node:internal/streams/readable:297:9) at Socket.Readable.push (node:internal/streams/readable:234:10) at Pipe.onStreamRead (node:internal/stream_base_commons:190:23) +44ms MongoMS:MongoMemoryServer Mongo[unknown]: stop: Called .stop() method +4ms MongoMS:MongoMemoryServer Mongo[unknown]: stop: "instanceInfo" is not defined (never ran?) +0ms Error: Instance Failed to start with "DBException in initAndListen". Original Error: Location18656: Cannot start server with an unknown storage engine: ephemeralForTest at MongoInstance.checkErrorInLine (/home/daniel/project/node_modules/mongodb-memory-server-core/src/util/MongoInstance.ts:640:11) at MongoInstance.stdoutHandler (/home/daniel/project/node_modules/mongodb-memory-server-core/src/util/MongoInstance.ts:585:10) at Socket.emit (node:events:513:28) at Socket.emit (node:domain:489:12) at addChunk (node:internal/streams/readable:324:12) at readableAddChunk (node:internal/streams/readable:297:9) at Socket.Readable.push (node:internal/streams/readable:234:10) at Pipe.onStreamRead (node:internal/stream_base_commons:190:23) ```
hasezoey
commented
11 months ago More logs with MONGOMS_DEBUG=1 for ubuntu 20.04, mongodb 7.0.0:
MongoMS:MongoInstance Mongo[37743]: stderrHandler: ""/home/daniel/.cache/mongodb-binaries/mongod-x86_64-ubuntu-7.0.0: /run/current-system/sw/share/nix-ld/lib/libcurl.so.4: no version information available (required by /home/daniel/.cache/mongodb-binaries/mongod-x86_64-ubuntu-7.0.0)"" +0ms
well, i have not run this is message yet, though it seems like it is non-fatal?
Location18656: Cannot start server with an unknown storage engine: ephemeralForTest
your issue is that by default mms (8.x and below) is using storage engine ephermeralForTest, but it got removed in mongodb 7.0, MMS 9.0 fixes this issue (currently released is 9.0.0-beta.2)
danielpza
commented
11 months ago @hasezoey after manually installing libcrypto.so.1.1 I got it to work in both debian and ubuntu
danielpza
commented
11 months ago For reference, this is an extract of my home manager config:
programs.bash.bashrcExtra = ''
export PATH="$VOLTA_HOME/bin:$PATH"
export LD_LIBRARY_PATH="${
lib.makeLibraryPath (with pkgs; [
stdenv.cc.cc
openssl
openssl_1_1 # https://discourse.nixos.org/t/how-to-fix-library-is-missing-or-cannot-be-opened-libcrypto-so-1-1/30730, https://github.com/nodkz/mongodb-memory-server/issues/782
lzlib # related https://github.com/NixOS/nix/issues/1550
libGL
libuuid
curlFull
])
}:$LD_LIBRARY_PATH"
export MONGOMS_DISTRO="ubuntu2204"
'';
nixpkgs.config.permittedInsecurePackages = [ "openssl-1.1.1w" ];@danielpza could you maybe re-test mongodb 7.0.0 with either storageEngine: "wiredTiger" or use mms version 9.0.0-beta.2?
danielpza
commented
8 months ago @hasezoey I haven't been able to try it again with your suggestion. Just passing by to leave a configuration that works for me:
{
inputs = {
nixpkgs = { url = "github:NixOS/nixpkgs/nixpkgs-unstable"; };
flake-utils = { url = "github:numtide/flake-utils"; };
};
outputs = { self, nixpkgs, flake-utils }:
flake-utils.lib.eachDefaultSystem (system:
let
pkgs = import nixpkgs {
inherit system;
config.permittedInsecurePackages = [ "openssl-1.1.1w" ];
};
lib = nixpkgs.lib;
in {
devShell = pkgs.mkShell {
MONGOMS_DISTRO = "ubuntu-22.04";
NIX_LD_LIBRARY_PATH = lib.makeLibraryPath
(with pkgs; [ stdenv.cc.cc openssl_1_1 curlFull ]);
NIX_LD =
builtins.readFile "${pkgs.stdenv.cc}/nix-support/dynamic-linker";
};
});
}
Versions
Detection
Current Error
Extra