Closed bryanasdev000 closed 2 years ago
Hi,
master seems to be building just fine on macOS. Started a build, which hasn't completed yet. But I've got past the check phase for 5_0
.
Could you check a different revision?:
NIXPKGS_ALLOW_UNFREE=1 nix build --impure github:NixOS/nixpkgs/a7cf9372e97725eaa6da1e72698af9d23a3ea083#mongodb-5_0
This points to master shortly after the PR was merged.
Hi, master seems to be building just fine on macOS.
macOS is locked to an older version: https://github.com/NixOS/nixpkgs/blob/master/pkgs/servers/nosql/mongodb/5.0.nix#L14
linux-x86_64
errors as:
Hi, master seems to be building just fine on macOS.
macOS is locked to an older version: https://github.com/NixOS/nixpkgs/blob/master/pkgs/servers/nosql/mongodb/5.0.nix#L14
linux-x86_64
errors as:
Yes, I know. :) could you check the revision I mentioned.
Yes, I know. :) could you check the revision I mentioned.
Nixpkgs at commit: a7cf9372e97725eaa6da1e72698af9d23a3ea083
mongodb-5_0
builds fine for linux-x86_64.
Yes, I know. :) could you check the revision I mentioned.
Nixpkgs at commit: a7cf937
mongodb-5_0
builds fine for linux-x86_64.
Ok. So it should be possible to bisect this. I can try to look into this later. But you are happy to take a stab at finding out wich commit introduced the regression.
Ok. So it should be possible to bisect this. I can try to look into this later. But you are happy to take a stab at finding out wich commit introduced the regression.
My bisect attempt pointed first failure to: https://github.com/NixOS/nixpkgs/commit/999a16ca632ae8ea042b29869af540abc3372500 (PR #168737)
Previous commit (f3803e3d17d067e565ac1f9ee6512f908aefd353) builds fine.
But. I had to cheat. Because 1be4226669a58a64db6118c5d257ceaee02589cd
was first top-level commit for package mongodb-5_0
. (And it builds fine.) No commit before that would find mongodb-5_0
package. So I informed every commit that nix build
failed to find package as good.
I think the problem is related to the GCC11 update. Gentoo has a patch for it until this if fixed upsteam. Maybe we should apply this one too.
I think the problem is related to the GCC11 update. Gentoo has a patch for it until this if fixed upsteam. Maybe we should apply this one too.
Nice. Yes, this should fix the issue with mongodb-4_4
. Not sure about mongodb-5_0
though.
mongodb-4_0
and mongodb-4_2
are also broken.
mongodb-4_0
errormongodb-4_2
errorHi, master seems to be building just fine on macOS. Started a build, which hasn't completed yet. But I've got past the check phase for
5_0
. Could you check a different revision?:NIXPKGS_ALLOW_UNFREE=1 nix build --impure github:NixOS/nixpkgs/a7cf9372e97725eaa6da1e72698af9d23a3ea083#mongodb-5_0
This points to master shortly after the PR was merged.
Yes I can build (or atleast start the build) with 92402c908ff53ee426f112a5fc13928105668f9d (mongodb: 5.0.5 -> 5.0.7).
Also, I had forgotten how heavy it was to compile mongo :P
mongodb-4_0
andmongodb-4_2
are also broken.
mongodb-4_0
errorIn file included from /nix/store/bd9k3zwm0b3p8k3qw0nlk71vzfa5hn8n-glibc-2.34-115-dev/include/signal.h:328, from /nix/store/1wbkdhc44n09f6mhdw7y0d8dlkck29m9-gcc-11.2.0/include/c++/11.2.0/csignal:42, from src/mongo/stdx/thread.h:34, from src/mongo/db/client.h:46, from src/mongo/db/commands.h:43, from src/mongo/db/stats/operation_latency_histogram.h:34, from src/mongo/db/stats/operation_latency_histogram.cpp:33: src/mongo/stdx/thread.h:109:56: error: call to non-'constexpr' function 'long int sysconf(int)' 109 | std::max(kMongoMinSignalStackSize, std::size_t{MINSIGSTKSZ}); | ^~~~~~~~~~~ In file included from include/boost/config/stdlib/libstdcpp3.hpp:78, from include/boost/config.hpp:48, from include/boost/core/addressof.hpp:17, from include/boost/optional/optional.hpp:28, from include/boost/optional.hpp:15, from src/mongo/db/commands.h:33, from src/mongo/db/stats/operation_latency_histogram.h:34, from src/mongo/db/stats/operation_latency_histogram.cpp:33: /nix/store/bd9k3zwm0b3p8k3qw0nlk71vzfa5hn8n-glibc-2.34-115-dev/include/unistd.h:640:17: note: 'long int sysconf(int)' declared here 640 | extern long int sysconf (int __name) __THROW; | ^~~~~~~
mongodb-4_2
errorIn file included from src/mongo/db/exec/plan_stage.h:35, from src/mongo/db/query/plan_ranker.h:37, from src/mongo/db/query/plan_ranker.cpp:39: src/mongo/db/exec/plan_stats.h:214:10: error: ‘optional’ in namespace ‘std’ does not name a template type 214 | std::optional<std::string> replanReason; | ^~~~~~~~ src/mongo/db/exec/plan_stats.h:41:1: note: ‘std::optional’ is defined in header ‘<optional>’; did you forget to ‘#include <optional>’? 40 | #include "mongo/util/container_size_helper.h" +++ |+#include <optional> 41 | #include "mongo/util/time_support.h"
Hmm both seems to relate with @bachp comment (https://github.com/NixOS/nixpkgs/issues/171928#issuecomment-1120261831).
Trying the patch...
mongodb-4_0
andmongodb-4_2
are also broken.
mongodb-4_0
errorIn file included from /nix/store/bd9k3zwm0b3p8k3qw0nlk71vzfa5hn8n-glibc-2.34-115-dev/include/signal.h:328, from /nix/store/1wbkdhc44n09f6mhdw7y0d8dlkck29m9-gcc-11.2.0/include/c++/11.2.0/csignal:42, from src/mongo/stdx/thread.h:34, from src/mongo/db/client.h:46, from src/mongo/db/commands.h:43, from src/mongo/db/stats/operation_latency_histogram.h:34, from src/mongo/db/stats/operation_latency_histogram.cpp:33: src/mongo/stdx/thread.h:109:56: error: call to non-'constexpr' function 'long int sysconf(int)' 109 | std::max(kMongoMinSignalStackSize, std::size_t{MINSIGSTKSZ}); | ^~~~~~~~~~~ In file included from include/boost/config/stdlib/libstdcpp3.hpp:78, from include/boost/config.hpp:48, from include/boost/core/addressof.hpp:17, from include/boost/optional/optional.hpp:28, from include/boost/optional.hpp:15, from src/mongo/db/commands.h:33, from src/mongo/db/stats/operation_latency_histogram.h:34, from src/mongo/db/stats/operation_latency_histogram.cpp:33: /nix/store/bd9k3zwm0b3p8k3qw0nlk71vzfa5hn8n-glibc-2.34-115-dev/include/unistd.h:640:17: note: 'long int sysconf(int)' declared here 640 | extern long int sysconf (int __name) __THROW; | ^~~~~~~
mongodb-4_2
errorIn file included from src/mongo/db/exec/plan_stage.h:35, from src/mongo/db/query/plan_ranker.h:37, from src/mongo/db/query/plan_ranker.cpp:39: src/mongo/db/exec/plan_stats.h:214:10: error: ‘optional’ in namespace ‘std’ does not name a template type 214 | std::optional<std::string> replanReason; | ^~~~~~~~ src/mongo/db/exec/plan_stats.h:41:1: note: ‘std::optional’ is defined in header ‘<optional>’; did you forget to ‘#include <optional>’? 40 | #include "mongo/util/container_size_helper.h" +++ |+#include <optional> 41 | #include "mongo/util/time_support.h"
Also for MongoDB v4.0 this patch may be needed:
https://gitweb.gentoo.org/repo/gentoo.git/tree/dev-db/mongodb/files/mongodb-5.0.2-glibc-2.34.patch
Arround one hour in MongoDB 4.4 compilation, I think that's safe to assume that the patch worked.
I will cancel it and try 4.2 and 4.0.
EDIT did not needed to cancel, it worked!
scons: done reading SConscript files.
scons: Building targets ...
scons: `install-core' is up to date.
scons: done building targets.
installPhase completed in 1 minutes 15 seconds
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/izgc0a6r3f7g7nw0nadciw0jwqia4qjw-mongodb-4.4.13
shrinking /nix/store/izgc0a6r3f7g7nw0nadciw0jwqia4qjw-mongodb-4.4.13/bin/mongos
shrinking /nix/store/izgc0a6r3f7g7nw0nadciw0jwqia4qjw-mongodb-4.4.13/bin/mongo
shrinking /nix/store/izgc0a6r3f7g7nw0nadciw0jwqia4qjw-mongodb-4.4.13/bin/mongod
strip is /nix/store/k0np19xv7wjc1grhrb2y6r85qiv1cikd-gcc-wrapper-11.2.0/bin/strip
stripping (with command strip and flags -S) in /nix/store/izgc0a6r3f7g7nw0nadciw0jwqia4qjw-mongodb-4.4.13/lib /nix/store/izgc0a6r3f7g7nw0nadciw0jwqia4qjw-mongodb-4.4.13/bin
patching script interpreter paths in /nix/store/izgc0a6r3f7g7nw0nadciw0jwqia4qjw-mongodb-4.4.13
checking for references to /build/ in /nix/store/izgc0a6r3f7g7nw0nadciw0jwqia4qjw-mongodb-4.4.13...
fixupPhase completed in 41 seconds
running install tests
MongoDB shell version v4.4.13
Build Info: {
"version": "4.4.13",
"gitVersion": "df25c71b8674a78e17468f48bcda5285decb9246",
"openSSLVersion": "OpenSSL 1.1.1n 15 Mar 2022",
"modules": [],
"allocator": "tcmalloc",
"environment": {
"distarch": "x86_64",
"target_arch": "x86_64"
}
}
/nix/store/izgc0a6r3f7g7nw0nadciw0jwqia4qjw-mongodb-4.4.13
4.2 worked, 4.0 did not, testing with the other patch...
Ok with some adjusts this patch worked, so 4.0, 4.2 and 4.4 is building now (sending a PR in a few minutes), still need to sort de OpenSSL problem with 5.0...
I am hitting this https://github.com/mongodb/mongo/blob/master/SConstruct#L4076 do not know exactly why yet.
I am hitting this https://github.com/mongodb/mongo/blob/master/SConstruct#L4076 do not know exactly why yet.
Yep, scons seems to not be picking up the environment set by Nix. The interesting thing is that this used to work and broke at some point. See https://github.com/NixOS/nixpkgs/issues/171928#issuecomment-1120206629
Interesting note on openssl
library:
https://github.com/mongodb/mongo/blob/master/SConstruct#L4043
As a test, try to lock stdenv = gcc10Stdenv;
.
At pkgs/top-level/all-packages.nix
:
mongodb-5_0 = callPackage ../servers/nosql/mongodb/5.0.nix {
sasl = cyrus_sasl;
boost = boost17x.override { enableShared = false; };
inherit (darwin) cctools;
inherit (darwin.apple_sdk.frameworks) CoreFoundation Security;
###
stdenv = gcc10Stdenv;
###
};
At pkgs/servers/nosql/mongodb/5.0.nix
:
buildMongoDB = callPackage ./mongodb.nix {
inherit sasl boost Security CoreFoundation cctools stdenv;
};
Add stdenv
there. ^
If only added to top level, wouldn't work.
My host is under powered to build mongodb. So I don't know if this works. Leave feedback if you can build it. (Also I'm curious to know if it works or not.)
As a test, try to lock
stdenv = gcc10Stdenv;
.
I've suspected something similar. Diffing the two commits that you bisected shows that a notable difference is that the one that works uses gcc10 and the one that doesn't uses gcc11. I've been building gcc10 for over an hour now. Let's see whether the installation of 5.0 works after that.
Interesting note on
openssl
library: https://github.com/mongodb/mongo/blob/master/SConstruct#L4043As a test, try to lock
stdenv = gcc10Stdenv;
.At
pkgs/top-level/all-packages.nix
:mongodb-5_0 = callPackage ../servers/nosql/mongodb/5.0.nix { sasl = cyrus_sasl; boost = boost17x.override { enableShared = false; }; inherit (darwin) cctools; inherit (darwin.apple_sdk.frameworks) CoreFoundation Security; ### stdenv = gcc10Stdenv; ### };
At
pkgs/servers/nosql/mongodb/5.0.nix
:buildMongoDB = callPackage ./mongodb.nix { inherit sasl boost Security CoreFoundation cctools stdenv; };
Add
stdenv
there. ^ If only added to top level, wouldn't work.My host is under powered to build mongodb. So I don't know if this works. Leave feedback if you can build it. (Also I'm curious to know if it works or not.)
Thanks for the tip! It at least pass the OpenSSL check and starts building, will report once it completes.
Yet, its still obscure to me why GCC version will implicate a problem with the OpenSSL check.
EDIT wait I was adding openssl.dev to the buildInputs, will remove and restart.
EDIT 2 build passed the OpenSSL check, the glibc patch from Gentoo probably won't be needed since we are now using GCC10 dropping it and restarting the build.
As a test, try to lock
stdenv = gcc10Stdenv;
.I've suspected something similar. Diffing the two commits that you bisected shows that a notable difference is that the one that works uses gcc10 and the one that doesn't uses gcc11. I've been building gcc10 for over an hour now. Let's see whether the installation of 5.0 works after that.
Sorry for the late reply. I can confirm this works.
As a test, try to lock
stdenv = gcc10Stdenv;
.I've suspected something similar. Diffing the two commits that you bisected shows that a notable difference is that the one that works uses gcc10 and the one that doesn't uses gcc11. I've been building gcc10 for over an hour now. Let's see whether the installation of 5.0 works after that.
Sorry for the late reply. I can confirm this works.
Thanks! Pushing this commit now!
I still need to update the docs, after that I think we are good to go :D
Also editing OP to thanks @superherointj !
About when can we expect this issue to be fixed?
About when can we expect this issue to be fixed?
I have extracted the fix for 4_0 and 4_2, in case that helps you. Feel free to review at: https://github.com/NixOS/nixpkgs/pull/178881
4_4 and 5 still errors. (There was some other newer change that broke it (again). I haven't bisected it.)
About when can we expect this issue to be fixed?
When my PC doesn't heat up the whole room trying to build this :P
Jokes aside, just like @superherointj said, I got it to build, then it has broken again and times limited what I can invest right now to fix it.
About when can we expect this issue to be fixed?
I have extracted the fix for 4_0 and 4_2, in case that helps you. Feel free to review at: #178881
4_4 and 5 still errors. (There was some other newer change that broke it (again). I haven't bisected it.)
Will do it, thanks. At least, for now, we should have a 4.0 and 4.2 working.
As soon as I have some time I will try to dig in https://github.com/NixOS/nixpkgs/pull/172009 and rebase with https://github.com/NixOS/nixpkgs/pull/178881.
Sorry I used the work fix but it won't fix completely so I reworded but you get 2 GitHub link.
I managed to build mongod v5 (in nix-shell) with install-mongod
as target as I have not 600Go (doc say 13 for minimal and 600 with test and al.) for nixos. (might be used for create a minimal variant)
I took a commit of master that doesn't need to drop gcc version.
For unkown reason I can't build with nixos-rebuild. So I relaunched with -K and wait for another 8h \:) (I hope it isn't a disk-space issue 😬). So not easy to debug+I am a new user.
The reason was it copy and donc move it.
Do you think hydra can add exception to MongoDB once all free versions are dropped ?
Sorry I used the work fix but it won't fix completely so I reworded but you get 2 GitHub link.
I managed to build mongod v5 (in nix-shell) with
install-mongod
as target as I have not 600Go (doc say 13 for minimal and 600 with test and al.) for nixos. (might be used for create a minimal variant)I took a commit of master that doesn't need to drop gcc version.
For unkown reason I can't build with nixos-rebuild. So I relaunched with -K and wait for another 8h :) (I hope it isn't a disk-space issue grimacing). So not easy to debug+I am a new user.
The reason was it copy and donc move it.
Do you think hydra can add exception to MongoDB once all free versions are dropped ?
Thanks for the input, but sorry I did not fully understood your message.
600Go = GCC?
About Hydra and MongoDB I think not, see here https://github.com/NixOS/nixpkgs/issues/83433 and here https://github.com/NixOS/nixpkgs/issues/83884 for context.
The problem between nixos-rebuild and nix-shell maybe the appointing to nixpkgs, if you use flakes you should change your input to your repo and if not set NIX_PATH or something along those lines.
I highly suggest using nix-build
and invoking resulting binaries under ./result/bin/MONGO_BINARIES
.
Sadly, with MongoDB license, I am just fixing it because I want it fixed, building it here takes a bit of time and since the client itself was my main interest this gives little practical usage for me, of course, in a server it will be probably quick enough and since there's no need to update it often, it might get way with it.
About 13 GB of free disk space for the core binaries (mongod, mongos, and mongo) and about 600 GB for the install-all target.
the default target is: https://github.com/mongodb/mongo/blob/6f7dae919422dcd7f4892c10ff20cdc721ad00e6/SConstruct#L4925 In the docs they don't say what build install-default I supposed = install-all
The issue was my /nix/store is mounted on disk that didn't have enough space to copy binary from /tmp I just resized and relaunched. I see the logs and nixos-rebuild took the correct nixpkgs and managed to build install-mongod
target.
Can this issue be closed ?
Can this issue be closed ?
Great catch!
Based on the latest comments of https://github.com/NixOS/nixpkgs/pull/172009#issuecomment-1243085896 and no new issues, I think we can.
Just to verify, this PR did not fix mongodb 5.0 on OSX and left it marked broken right? I tried building it anyway with a recent nixpkgs to verify it it actually broken with the scons error:
error: builder for '/nix/store/l50w9mkw96n75ca53gpfgv2dsw5swhhf-mongodb-5.0.3.drv' failed with exit code 2;
last 10 log lines:
> Compiling build/nixos/third_party/libstemmer_c/src_c/stem_UTF_8_portuguese.o
> Compiling build/nixos/third_party/libstemmer_c/runtime/api.o
> Compiling build/nixos/third_party/libstemmer_c/src_c/stem_UTF_8_finnish.o
> Compiling build/nixos/third_party/libstemmer_c/src_c/stem_UTF_8_russian.o
> Compiling build/nixos/third_party/libstemmer_c/src_c/stem_UTF_8_danish.o
> Compiling build/nixos/third_party/libstemmer_c/libstemmer/libstemmer_utf8.o
> Compiling build/nixos/mongo/db/auth/auth_options_gen.o
> scons: *** [build/nixos/third_party/mozjs-60/extract/js/src/gc/StoreBuffer.o] AttributeError : 'NoneType' object has no attribute 'disambiguate'
> scons: building terminated because of errors.
> build/nixos/third_party/mozjs-60/extract/js/src/gc/StoreBuffer.o failed: AttributeError : 'NoneType' object has no attribute 'disambiguate'
For full logs, run 'nix log /nix/store/l50w9mkw96n75ca53gpfgv2dsw5swhhf-mongodb-5.0.3.drv'.
error: 1 dependencies of derivation '/nix/store/flkjngmc7w6lkm3agdxr2lx6xnqrakcr-devenv-profile.drv' failed to build
error:
error: 1 dependencies of derivation '/nix/store/jf5xqs4d6frz8lz2n1d56gxbdj44b970-devenv-shell-env.drv' failed to build
In October it builded. Maybe another regression (since it is non free hydra doesn't check it) just open a new issue (if none is already open)
Mongodb-4_4 (required by Unifi) doesn't compile with the latest unstable:
scons: building terminated because of errors.
build/nixos/mongo/db/repl/replication_coordinator_impl.o failed: Error 1
error: builder for '/nix/store/6zm9vh7277y5yfs62vd80nsjnlip2da4-mongodb-4.4.23.drv' failed with exit code 2
@Et7f3 @alexvorobiev mongodb_4-4 does not build. Maybe for a different reason, I don't know. New issue here: https://github.com/NixOS/nixpkgs/issues/258489
Describe the bug
MongoDB 4.4 and 5.0 is currently broken in master.
Steps To Reproduce
mongodb-5_0
ormongodb-4.4
.Expected behavior
Screenshots
N/A
Additional context
My old PR: https://github.com/NixOS/nixpkgs/pull/146324#issuecomment-1120089746
For 5_0:
For 4.4:
Notify maintainers
@bluescreen303 @offline @cstrahan @kfiz @otavio @bachp
Metadata
Please run
nix-shell -p nix-info --run "nix-info -m"
and paste the result.