ssl cert file behavior document

[mohnishkodnani]

Hi, I am back with the ssl_cert_file issue. I had originally reported this issue and it was fixed in the previous installer. But I recently wiped out the laptop and reinstalled the with the latest version. This time I set the environment variable export NIX_INSTALLER_SSL_CERT_FILE=/Library/Application Support/Netskope/STAgent/download/nscacert_combined.pem

Installer succeeded. I see in the receipt.json the following lines.

  95   │         "action": "provision_nix",
  96   │         "fetch_nix": {
  97   │           "action": {
  98   │             "url_or_path": {
  99   │               "Url": "https://releases.nixos.org/nix/nix-2.21.2/nix-2.21.2-aarch64-darwin.tar.xz"
 100   │             },
 101   │             "dest": "/nix/temp-install-dir",
 102   │             "proxy": null,
 103   │             "ssl_cert_file": "/Library/Application Support/Netskope/STAgent/download/nscacert_combined.pem
       │ "
 104   │           },
 105   │           "state": "Completed"
 106   │         },

All good so far. Now, I have a project level nix flake flake.nix file and I use direnv. My .envrc file is as follows

   1   │ nix_direnv_manual_reload
   2   │ use flake . --show-trace --print-build-logs --verbose

Now inside the flake I have the following lines

      nativeBuildInputs = with pkgs; [
        cmake
        ninja
        pkg-config
        doxygen
        prometheus-cpp
        gbenchmark
        catch2_3
        (opentelemetry-cpp.overrideAttrs(oldAttrs: rec {
#          CURL_CA_BUNDLE="/Library/Application Support/Netskope/STAgent/download/nscacert_combined.pem";
#          REQUESTS_CA_BUNDLE="/Library/Application Support/Netskope/STAgent/download/nscacert_combined.pem";
#          NIX_SSL_CERT_FILE="/Library/Application Support/Netskope/STAgent/download/nscacert_combined.pem";
          version = "1.15.0";
          src = fetchFromGitHub {
            owner = "open-telemetry";
            repo = "opentelemetry-cpp";
            rev = "v${version}";
            sha256 = "";
          };
          outputs = ["out"];
          cmakeFlags = oldAttrs.cmakeFlags ++ [
            "-DWITH_ABSEIL=OFF"
            "-DBUILD_SHARED_LIBS=OFF"
            "-DBUILD_TESTING=OFF"
            "-DCMAKE_CXX_STANDARD=20"
            "-DCMAKE_POSITION_INDEPENDENT_CODE=ON" 
            "-DWITH_STL=ON"
            "-DWITH_PROMETHEUS=ON"
            "-DWITH_BENCHMARK=OFF"
            "-DWITH_OTLP_GRPC=OFF"
            "-DWITH_OTLP_HTTP=OFF"
            "-DOPENTELEMETRY_INSTALL=ON"
            "-DWITH_EXAMPLES=OFF"
            "-DCMAKE_VERBOSE_MAKEFILE:BOOL=ON"
            "-DWITH_ELASTICSEARCH=OFF"
            "-DWITH_ZIPKIN=OFF"
            "-DWITH_HTTP_CLIENT_CURL=OFF"
            "-DWITH_ETW=OFF"
          ];
        }))
        fmt
      ];

the important part here is the opentelemetry-cpp overrideAttrs section. I wanted to use the latest version available from github rather than the nixpkgs one. When I save this file nix tries to download the archive tar.gz file through curl but fails.

source> error: cannot download source from any mirror
error: builder for '/nix/store/ms5g9il49nlb0mp4n6cgmr70chvkq3wy-source.drv' failed with exit code 1;
       last 19 log lines:
       > error checking the existence of https://tarballs.nixos.org/sha256/:
       > curl: (60) SSL certificate problem: self-signed certificate in certificate chain
       > More details here: https://curl.se/docs/sslcerts.html
       >
       > curl failed to verify the legitimacy of the server and therefore could not
       > establish a secure connection to it. To learn more about this situation and
       > how to fix it, please visit the web page mentioned above.
       >
       > trying https://github.com/open-telemetry/opentelemetry-cpp/archive/v1.15.0.tar.gz
       >   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
       >                                  Dload  Upload   Total   Spent    Left  Speed
       >   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
       > curl: (60) SSL certificate problem: self-signed certificate in certificate chain
       > More details here: https://curl.se/docs/sslcerts.html
       >
       > curl failed to verify the legitimacy of the server and therefore could not
       > establish a secure connection to it. To learn more about this situation and
       > how to fix it, please visit the web page mentioned above.
       > error: cannot download source from any mirror
       For full logs, run 'nix log /nix/store/ms5g9il49nlb0mp4n6cgmr70chvkq3wy-source.drv'.
error: 1 dependencies of derivation '/nix/store/z891gdjgr144f2z7iniw0dli7pm8426k-opentelemetry-cpp-1.15.0.drv'

Last time, I remember that I had to modify the launch daemon, so i checked and found that

1. The nix-daemon environment section does not have the NIX_SSL_CERT_FILE set to the file I passed.
2. cat /etc/nix/nix.conf has the attribute ssl-cert-file = /Library/Application Support/Netskope/STAgent/download/nscacert_combined.pem

I tried to change the nix-daemon.plist to add the environment variable as well and relaunch the daemon.

  <dict>
   5   │     <key>EnvironmentVariables</key>
   6   │     <dict>
   7   │       <key>OBJC_DISABLE_INITIALIZE_FORK_SAFETY</key>
   8   │       <string>YES</string>
   9   │       <key>NIX_SSL_CERT_FILE</key>
  10   │       <string>/Library/Application Support/Netskope/STAgent/download/nscacert_combined.pem</string>
  11   │     </dict>

I confirmed with the launchctl procinfo command that it indeed does have this variable set.

sudo launchctl procinfo 4481

nvironment vector = {
    OBJC_DISABLE_INITIALIZE_FORK_SAFETY => YES
    PATH => /usr/bin:/bin:/usr/sbin:/sbin
    PWD => /
    XPC_FLAGS => 0x0
    NIX_SSL_CERT_FILE => /Library/Application Support/Netskope/STAgent/download/nscacert_combined.pem
    XPC_SERVICE_NAME => org.nixos.nix-daemon
    SHLVL => 0
}

However, I still get the same error on the project level flake. I then tried to override the file ( which is the 2nd thing I used to do before ) /nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt with my concatenated version of the file. I use the original append mine and put it there. These 2 things in the past seemed to have worked, but not anymore.

Other observation.

1. After installation I did not have a the directory cd /nix/var/nix/profiles/per-user/<My user> created. So, when I ran nix build .#laptop it failed . I had to create this directory and put the correct chown permission.
2. The directory ~/.nix-profile which gets symlinked does not have the /etc/ssl/certs folder which means the per-user directory does not have this folder, but the default profile directory does have this. I don't know the repercusions of this.

I also found that the documentation around ssl-cert-file is not clear and hidden. It would be nice to document what are the changes that happen to the system when someone passes in ssl-cert-file during installation. For example, is the nix-daemon plist modified or not, if not why not.

Also, where should the per project flakes which use fetchFromGithub get the correct settings so that the internal usage of curl can get the proper certificates to be able to download.

If within that project directory ( where direnv and nix develop failed with the curl ssl error ) i manually do the curl on that above URL it can download the file, which means that during the develop it's not getting the same settings.

I picked one output derivation that gave error.

 File: /nix/store/a7b54jala1cljbj9cf94ychnggrb9hb8-source.drv
───────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────
   1   │ Derive([("out","/nix/store/ig6k3jn4sbl8dg1ni4pagr3d7vlcdc8w-source","r:sha256","69b0fef380658e15be9d817bfc
       │ b32e3f5de96da652bcdce77b4e750ed8beddee")],[("/nix/store/7vg2lrybxqayp599anksgdza7d63xqia-stdenv-darwin.drv
       │ ",["out"]),("/nix/store/n45m4wavay258b00rnlbj3zmijkbk7nr-mirrors-list.drv",["out"]),("/nix/store/spczjwghx
       │ 93fi5zk2946q58zgcb34gkx-bash-5.2p26.drv",["out"]),("/nix/store/wk8rcav76q4q2yziyndj23vx7df5bisx-curl-8.6.0
       │ .drv",["dev"])],["/nix/store/g0gn91m56b267ncx05w93kihyqia39cm-builder.sh"],"aarch64-darwin","/nix/store/bs
       │ a1v1mr7c42a0yd90ncnchcs18ylm4b-bash-5.2p26/bin/bash",["-e","/nix/store/g0gn91m56b267ncx05w93kihyqia39cm-bu
       │ ilder.sh"],[("SSL_CERT_FILE","/no-cert-file.crt"

You can see that the SSL_CERT_FILE is set to /no-cert-file.crt instead of the one that I provided during install.

I build my system using nix flake and am also noticing that ~/.nix-profile/etc points to etc -> /nix/store/v1v6sq5w6p6jbqbf1l7myms27mfi71w2-home-manager-path/etc now which does not have ssl certs

[mohnishkodnani]

I tried to create a symlink in ~/.nix-profile/etc to the /nix/var/nix/profiles/default/etc/ssl such that ~/.nix-profile/ points to /nix/var/nix/profiles/per-user/USER/profile ~/.nix-profile/bin/ -> points to /nix/store/v1v6sq5w6p6jbqbf1l7myms27mfi71w2-home-manager-path/bin <- home manager ~/.nix-profile/etc` points to -> /nix/store/v1v6sq5w6p6jbqbf1l7myms27mfi71w2-home-manager-path/etc <- home-manager tc.

However, inside ~/.nix-profile/etc/ssl points to /nix/var/nix/profiles/default/etc/ssl and now my flake works fine. So, I am not sure how all of this lines up but I find that everytime I find a new way to do things.

[mohnishkodnani]

The /nix/var/nix/profiles/default/etc has ssl directory but /nix/var/nix/profiles/per-user/<myuser>/profile/etc does not have ssl directory. The later is what is linked to ~/.nix-profile and hence, I believe the SSL certificates are not factored when running nix flakes.