Open antifuchs opened 1 year ago
Update to this, making a wrapper around git that exports NIX_SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt
seems to do the trick; it looks like go (or hugo) resets the environment somehow and removes that variable.
Found it. There is a regex-based allowlist of environment variables that hugo will let through to subprocesses it executes (including the go mod
processes that it uses for module operations). The NIX_SSL_CERT_FILE variable seems to be load-bearing on macOS, but not on Linux; I think we can patch that out.
I haven't fiddled with the cert envs much and don't have the best sense of why this only manifests on macOS--but I did look a little, so I'll leave some context.
The behavior is implemented with these patches, which do indeed differ for darwin: https://github.com/NixOS/nixpkgs/blob/7250b5ac791b9ae992db9f82dfe6ef400c838b21/pkgs/development/libraries/openssl/default.nix#L247-L260
relevant patches themselves:
Even after seeing how the patches differ between darwin and !darwin, it's unclear to me why this is working on Linux and not macOS. The only real thought I have is to see if this path is missing on macOS?
There was a change in Nix to provide a config alternative to this env a few months ago (https://github.com/NixOS/nix/pull/8062).
There's no indication of whether there's long-term intent to deprecate the env, but the fact of the change (and its later necessity for very new macOS versions) does make me think nixpkgs is conspicuously coupled to a Nix implementation detail here.
Even after seeing how the patches differ between darwin and !darwin, it's unclear to me why this is working on Linux and not macOS. The only real thought I have is to see if this path is missing on macOS?
Good call:
ls: cannot access '/nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt': No such file or directory
OK, digging into the rationale for the darwin patch, it sounds like this was added in https://github.com/NixOS/nixpkgs/pull/24203 in 2017, where the intent was to pull the ca bundle from the default profile; I'm not sure what the "default" profile is supposed to be, but my machines do not have that (they have a "system" profile, and per-user / per-container ones, but none named "default").
My suspicion is that this is a stale customization and can be removed, especially as nix-darwin now manages the "normal" ssl CA bundle file under /etc/ssl/certs/ca-certificates.crt
just like NixOS does.
What do you think, @abathur, should we remove that darwin patch from openssl & use the other patch?
I'm not really sure what's ~right, here, but depending on a nix-darwin implementation detail doesn't feel like a step in the right direction.
I think a stock Nix install will still come with a default profile that includes the cert bundle.
macOS stores the root certificates in the keychain, and while you can export them to a certificate bundle file, most software on macOS will read and write the certificates in the keychain, and not expect a certificate bundle file. nixpkgs packages expect a certificate bundle file either in a hardcoded location or pointed at by NIX_SSL_CERT_FILE
or similar environment variables. The cacert
will set up hooks to set that environment variable.
I think whether this works or not / how it works makes sense or not depends on how you're using nix on darwin:
Nix / nixpkgs on darwin has a couple of these situations.
Personally I think the certs should be read from the keychain: certs should be provided by the system, that's why they're read from the running system instead of treated as a build input, but because of how we're reading them in nixpkgs it's very easy for users using nix as an environment manager to provide them in nix shells via cacert and pin old versions of them.
I would like to exclude "rework openssl's certificate bundle logic to work fundamentally differently on macOS" or "use Security.framework from cli tools on macOS" from the scope of this issue: Reducing the bug to be the smallest it can be, it would be that currently openssl looks in a default location that doesn't exist for several people (suspect it's those of us who delete or never set up the profile named "default").
Given that there seems to be some (reasonable!) reluctance to making openssl reference the cacert package directly for its hard-coded certificate bundle path, and that nix-darwin sets up a decent-enough /etc/ssl/certs/ca-certificates.crt
(which can be managed via system config), I continue to think it's the best path forward to using the same path on both systems.
Yes, agreed. Given that a lot of sources discourage the use of nix-env, I believe we should let users either configure the cacerts manually by setting the environment variable / adding cacert
to their environment, or letting nix-darwin configure it for them. Leaving the default on darwin as is is likely to lead to users unknowingly using very old cacert bundles.
If it's not too complex of a change, what if we check the existence of both paths? Alternatively, I've noticed there's a /etc/ssl/cert.pem
on Macs, which we could use as a default path and change nix-darwin to overwrite that file.
Hello! Thanks for doing some great digging here.
I have recently run into this error, and it was baffling me a bit, so I think it would be helpful to document the major pieces of evidence if it helps solve the problem.
system
profiledefault
and per-user/$(whoami)
profilenix-darwin
(I'm new, I'll get there)/nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt
$NIX_SSL_CERT_FILE
is set to /nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt
/etc/ssl/certs/ca-certificates.crt
node2nix
to generate a derivation for a private projectgit
could download from https://registry.npmjs.org
git
failed with a similar issue when faced with https://github.com
sourcepkgs.cacert
to the derivation that ran node2nix
seems to resolve the issueI'm not sure why registry.npmjs.org
worked when github.com
did not, but anyone else runs into this issue, try adding cacert
to the nativeBuildInputs
in the derivation.
Issue description
On macOS, using the git packaged in nixpkgs, hugo from nixpkgs can not download modules with git: It shows the following error:
This error does not occur on linux, and only happens when using the
git
package packaged in nixpkgs. See https://github.com/antifuchs/hugo-mod-report for a fully-functional repro repository.Steps to reproduce
git clone the repro repository in https://github.com/antifuchs/hugo-mod-report, activate the flake's devshell there with
nix develop
, then runhugo mod tidy
. It'll work in linux, but fail on darwin.Technical details
Please run
nix-shell -p nix-info --run "nix-info -m"
and paste the result."aarch64-darwin"
Darwin 22.5.0, macOS 13.4.1
yes
no
nix-env (Nix) 2.13.3
"nixpkgs"
"darwin"
/nix/store/274dy8km1x7zrnn4wffzwqd1h1nggx71-source