Add missing libblst to GHC options of wrapped-cabal in static mode.

[angerman]

@yvan-sraka can you test this? @KtorZ mentioned this did not work 😰

I have this on my list to test tomorrow, but if you can before me that would be great!

[yvan-sraka]

I got this issue (is this similar to what you got @KtorZ?):

% nix develop ".#ghc810-static-minimal"
error:
       … while calling the 'derivationStrict' builtin

         at /builtin/derivation.nix:9:12: (source not available)

       … while evaluating derivation 'nix-shell'
         whose name attribute is located at /nix/store/p7iz0r8gs6ppkhj83zjmwyd21k8b7v3y-source/pkgs/stdenv/generic/make-derivation.nix:300:7

       … while evaluating attribute 'nativeBuildInputs' of derivation 'nix-shell'

         at /nix/store/p7iz0r8gs6ppkhj83zjmwyd21k8b7v3y-source/pkgs/stdenv/generic/make-derivation.nix:344:7:

          343|       depsBuildBuild              = lib.elemAt (lib.elemAt dependencies 0) 0;
          344|       nativeBuildInputs           = lib.elemAt (lib.elemAt dependencies 0) 1;
             |       ^
          345|       depsBuildTarget             = lib.elemAt (lib.elemAt dependencies 0) 2;

       (stack trace truncated; use '--show-trace' to show the full trace)

       error: function 'anonymous lambda' called with unexpected argument 'enableShared'

       at /nix/store/fimq5djjixscylfrppd2fh483vz3bgg3-source/overlays/crypto/libblst.nix:1:1:

            1| { stdenv, lib, autoreconfHook, inputs }:
             | ^
            2|

I "fixed" it with this patch:

diff --git a/flake.nix b/flake.nix
index 95a8722..993df4e 100644
--- a/flake.nix
+++ b/flake.nix
@@ -23,7 +23,7 @@
           static-openssl = (final.openssl.override { static = true; });
           static-zlib = final.zlib.override { shared = false; };
           static-pcre = final.pcre.override { shared = false; };
-          static-libblst = (final.libblst.override { enableShared = false; }).overrideDerivation (old: {
+          static-libblst = (final.libblst.override { }).overrideDerivation (old: {
             postFixup = "";
           });
         });

Maybe we should rather update the blst definition in nixpkgs?

[KtorZ]

I got this issue (is this similar to what you got @KtorZ?):

% nix develop ".#ghc810-static-minimal"
error:
       … while calling the 'derivationStrict' builtin

         at /builtin/derivation.nix:9:12: (source not available)

       … while evaluating derivation 'nix-shell'
         whose name attribute is located at /nix/store/p7iz0r8gs6ppkhj83zjmwyd21k8b7v3y-source/pkgs/stdenv/generic/make-derivation.nix:300:7

       … while evaluating attribute 'nativeBuildInputs' of derivation 'nix-shell'

         at /nix/store/p7iz0r8gs6ppkhj83zjmwyd21k8b7v3y-source/pkgs/stdenv/generic/make-derivation.nix:344:7:

          343|       depsBuildBuild              = lib.elemAt (lib.elemAt dependencies 0) 0;
          344|       nativeBuildInputs           = lib.elemAt (lib.elemAt dependencies 0) 1;
             |       ^
          345|       depsBuildTarget             = lib.elemAt (lib.elemAt dependencies 0) 2;

       (stack trace truncated; use '--show-trace' to show the full trace)

       error: function 'anonymous lambda' called with unexpected argument 'enableShared'

       at /nix/store/fimq5djjixscylfrppd2fh483vz3bgg3-source/overlays/crypto/libblst.nix:1:1:

            1| { stdenv, lib, autoreconfHook, inputs }:
             | ^
            2|

I "fixed" it with this patch:

diff --git a/flake.nix b/flake.nix
index 95a8722..993df4e 100644
--- a/flake.nix
+++ b/flake.nix
@@ -23,7 +23,7 @@
           static-openssl = (final.openssl.override { static = true; });
           static-zlib = final.zlib.override { shared = false; };
           static-pcre = final.pcre.override { shared = false; };
-          static-libblst = (final.libblst.override { enableShared = false; }).overrideDerivation (old: {
+          static-libblst = (final.libblst.override { }).overrideDerivation (old: {
             postFixup = "";
           });
         });

Maybe we should rather update the blst definition in nixpkgs?

That's exactly what I ran into.

[yvan-sraka]

Then, I believe that what you want to do is something more like:

diff --git a/flake.nix b/flake.nix
index 95a8722..438a181 100644
--- a/flake.nix
+++ b/flake.nix
@@ -23,7 +23,8 @@
           static-openssl = (final.openssl.override { static = true; });
           static-zlib = final.zlib.override { shared = false; };
           static-pcre = final.pcre.override { shared = false; };
-          static-libblst = (final.libblst.override { enableShared = false; }).overrideDerivation (old: {
+          static-libblst = final.libblst.overrideDerivation (old: {
+            configureFlags = old.configureFlags ++ [ "--enable-static" "--disable-shared" ];
             postFixup = "";
           });
         });

Do you have a minimal example project, so I can test if it builds as expected?

[angerman]

Ahh, so maybe the issue is that we don't pull the blst and friends from IOHK-nix? But rely on nixpkgs upstream? I know that for IOG projects the cryptography folks advised against using nixpkgs releases and default to our vendor release ones.

[KtorZ]

Do you have a minimal example project, so I can test if it builds as expected?

Not really minimal but I am experimenting this in kupo. This latest patch seems slightly better but it'll still expect the blst library to be shipped alongside the executable:

otool -L /path/to/kupo:
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.60.1)
        /usr/lib/libz.dylib (compatibility version 1.0.0, current version 1.3.0)
        libblst.dylib (compatibility version 0.0.0, current version 0.0.0)
        /usr/lib/libiconv.dylib (compatibility version 7.0.0, current version 7.0.0)
        /usr/lib/libffi.dylib (compatibility version 10.0.0, current version 10.2.0)

[KtorZ]

Fun note 1 (for some definition of fun): if I compile Kupo on my local dev environment, outside of the devx nix-shell, libblst is nowhere to be seen in the linked dependencies. Which is curious (I double-check and I am using the same compiler version on both, and I'd expect the constraints on the cabal.project to yield a same cardano-ledger and plutus versions that either require blst or not).

Fun note 2: when compiling through the devx shell, the linker warns me about "was built for newer macOS version (12.0) than being linked (11.0)"

[angerman]

@yvan-sraka can you please try to figure out with @KtorZ why his local dev env supposedly doesn't have/need libblst? That seems odd to me.

plutus-core -> cardano-crypto-class -> libblst

I would expect that dependency to be there?

Side note: Regarding the linker warning, that's not very problematic, it's just that nix passes some deploymen target. I'm fairly sure that warning can be ignored. But would be nice to clean that up as well.

[KtorZ]

I would expect that dependency to be there?

I believe it depends on what version of the ledger/plutus gets bundled in. cabal freeze tells me:

             any.plutus-core ==1.15.0.1,
             any.cardano-crypto-class ==2.1.4.0,
             cardano-crypto-class +secp256k1-support,

Which points to: https://github.com/IntersectMBO/cardano-base/blob/a4b01d1b990fab63505f2c20b79ccbbed033f3fc/cardano-crypto-class/cardano-crypto-class.cabal#L115-L125. Could it be that it's an old enough plutus-core package with only foundational work on BLS but unused; thus stripped by GHC somehow? Is the compiler smart-enough to do this kind of thing 😅 ?

[angerman]

For dynamic libraries on macOS, we do indeed deadstripping of dynamic libraries.

[angerman]

We use the same code paths in nix as well though. Hence I wouldn't expect that to result in any difference.

[KtorZ]

I checked and the dependencies selected (cabal freeze) are the same on my local dev env and within the nix shell. The only difference is that I am using:

package cardano-crypto-praos
  flags: -external-libsodium-vrf

on my local setup. Though I don't see why this would impact the need for libblst 🤔. Anyway, I'll be looking into this tomorrow morning with @yvan-sraka

[angerman]

Are you certain that flag is only used locally? Or should always be used. Relying on the vrf hack is a bad idea. It should at best be used in development.

[yvan-sraka]

@KtorZ could you merge https://github.com/CardanoSolutions/devx/pull/1, and then we could fix the tests of this PR :)

[angerman]

@yvan-sraka I guess this one is good to go? Can you have a last look and then merge it?