unsplash / intlc

Compile ICU messages into code. Supports TypeScript and JSX. No runtime.
MIT License
56 stars 3 forks source link

Binaries are dynamically linked #99

Open samhh opened 2 years ago

samhh commented 2 years ago

This usually isn't a problem, however in some environments such as AWS Lambda, or as I'm learning NixOS, it is. Looking at the 0.3.0 release Linux binary:

$ ldd -r ./intlc
        linux-vdso.so.1 (0x00007ffdbcbcb000)
    librt.so.1 => /nix/store/rir9pf0kz1mb84x5bd3yr0fx415yy423-glibc-2.33-123/lib/librt.so.1 (0x00007f5f2e280000)
    libutil.so.1 => /nix/store/rir9pf0kz1mb84x5bd3yr0fx415yy423-glibc-2.33-123/lib/libutil.so.1 (0x00007f5f2e27b000)
    libdl.so.2 => /nix/store/rir9pf0kz1mb84x5bd3yr0fx415yy423-glibc-2.33-123/lib/libdl.so.2 (0x00007f5f2e276000)
    libpthread.so.0 => /nix/store/rir9pf0kz1mb84x5bd3yr0fx415yy423-glibc-2.33-123/lib/libpthread.so.0 (0x00007f5f2e256000)
    libgmp.so.10 => not found
    libc.so.6 => /nix/store/rir9pf0kz1mb84x5bd3yr0fx415yy423-glibc-2.33-123/lib/libc.so.6 (0x00007f5f2e081000)
    libm.so.6 => /nix/store/rir9pf0kz1mb84x5bd3yr0fx415yy423-glibc-2.33-123/lib/libm.so.6 (0x00007f5f2df3e000)
    /lib64/ld-linux-x86-64.so.2 => /nix/store/rir9pf0kz1mb84x5bd3yr0fx415yy423-glibc-2.33-123/lib64/ld-linux-x86-64.so.2 (0x00007f5f2e28d000)
undefined symbol: __gmpn_cmp    (./intlc)
undefined symbol: __gmpn_mod_1  (./intlc)
undefined symbol: __gmpn_add_1  (./intlc)
undefined symbol: __gmpn_sub_1  (./intlc)
undefined symbol: __gmpn_mul_1  (./intlc)
undefined symbol: __gmpn_add    (./intlc)
undefined symbol: __gmpn_sub    (./intlc)
undefined symbol: __gmpn_mul    (./intlc)
undefined symbol: __gmpn_divrem_1   (./intlc)
undefined symbol: __gmpn_popcount   (./intlc)
undefined symbol: __gmpn_tdiv_qr    (./intlc)
undefined symbol: __gmpn_rshift (./intlc)
undefined symbol: __gmpn_lshift (./intlc)
undefined symbol: __gmpz_get_d  (./intlc)
undefined symbol: __gmpz_get_d_2exp (./intlc)
undefined symbol: __gmpn_gcd_1  (./intlc)
undefined symbol: __gmpz_init   (./intlc)
undefined symbol: __gmpz_gcd    (./intlc)
undefined symbol: __gmpz_clear  (./intlc)
undefined symbol: __gmpn_and_n  (./intlc)
undefined symbol: __gmpn_andn_n (./intlc)
undefined symbol: __gmpn_ior_n  (./intlc)
undefined symbol: __gmpn_xor_n  (./intlc)

From memory I believe it's possible to produce static binaries with Stack.

samhh commented 2 years ago

For anyone on NixOS, you can patch the binary as a workaround with patchelf. The only missing shared library is gmp.

patchelf \
  --set-interpreter "$(cat $NIX_CC/nix-support/dynamic-linker)" \
  --set-rpath /path/to/gmp/lib \
  /path/to/intlc

The Nix store path can be found by following the instructions here. On my system it looks like /nix/store/28b0wq1f7ndy3gzmlwrspply9aa7xnzx-gmp-6.2.1/lib.

samhh commented 2 years ago

For anyone using Nix here's an example derivation including a fix:

# This fetches the intlc binary and, if on Linux, patches its dynamic links. See:
#   https://github.com/unsplash/intlc/issues/99

{ fetchurl, lib, pkgs, stdenv }:

stdenv.mkDerivation rec {
  pname = "intlc";
  version = "0.3.1";

  src = if stdenv.isDarwin
    then fetchurl {
      url = "https://github.com/unsplash/intlc/releases/download/v${version}/intlc_macos";
      sha256 = "b391c440027d097253a8fae4110d71f86d9fb6eb3ee7abaccafee208469042ca";
    }
    else fetchurl {
      url = "https://github.com/unsplash/intlc/releases/download/v${version}/intlc_linux";
      sha256 = "2d5ddac1110862c4c588cdd3b2c99fe5997a37169f7c3fd23eb36a8b2e5b7e27";
    };

  dontUnpack = true;
  dontStrip = true;

  installPhase = ''
    mkdir -p $out/bin/
    cp $src $out/bin/intlc
    chmod +x $out/bin/intlc
  '';

  preFixup = if stdenv.isDarwin then "" else
    let
      linker = stdenv.cc.bintools.dynamicLinker;
      libPath = with pkgs; lib.makeLibraryPath [ gmp ];
    in ''
      patchelf --set-interpreter ${linker} --set-rpath ${libPath} $out/bin/intlc
    '';

  meta = {
    homepage = "https://github.com/unsplash/intlc";
    description = "Compile ICU messages into code.";
    license = lib.licenses.mit;
  };
}
samhh commented 1 year ago

This appears to affect all distros since we started building in a Nix shell. Linux distros can use patchelf like above with gmp and libffi.

samhh commented 1 year ago

0.6.1 resolves the regression in dynamic linking in 0.6.0. Most users should find it once again works out of the box. Users in non-FHS environments such as NixOS will continue to need to patch the dynamic links as above.

samhh commented 8 months ago

Here's a starting point to building with Nix (dynamically linked):

diff --git a/cli/CLI.hs b/cli/CLI.hs
index 456de43..a5e1a9b 100644
--- a/cli/CLI.hs
+++ b/cli/CLI.hs
@@ -1,8 +1,5 @@
-{-# LANGUAGE TemplateHaskell #-}
-
 module CLI (Opts (..), getOpts, ICUModifiers (..)) where

-import           GitHash                     (giTag, tGitInfoCwd)
 import qualified Intlc.Backend.JSON.Compiler as JSON
 import           Intlc.Core                  (Locale (..))
 import           Intlc.Linter                (LintRuleset (..))
@@ -24,9 +21,8 @@ getOpts = execParser (info (opts <**> helper <**> version) (progDesc h))
   where h = "Compile ICU messages into code."

 version :: Parser (a -> a)
-version = infoOption (giTag gi) (long "version" <> help msg <> hidden)
+version = infoOption "v0.8.1" (long "version" <> help msg <> hidden)
   where msg = "Print version information"
-        gi = $$tGitInfoCwd

 opts :: Parser Opts
 opts = subparser . mconcat $
diff --git a/flake.nix b/flake.nix
index 0cfd55f..2cd48a4 100644
--- a/flake.nix
+++ b/flake.nix
@@ -13,10 +13,16 @@
           #   https://github.com/NixOS/nixpkgs/blob/master/doc/languages-frameworks/haskell.section.md#available-packages-haskell-available-packages
           ghcVer = "ghc946";

-          haskPkgs = pkgs.haskell.packages."${ghcVer}";
+          overlay = final: _prev: {
+            intlc = final.callCabal2nix "intlc" ./. { };
+          };
+
+          haskPkgs = pkgs.haskell.packages."${ghcVer}".extend overlay;
       in {
         devShells = {
-          default = pkgs.mkShell {
+          default = haskPkgs.shellFor {
+            packages = p: [ p.intlc ];
+
             nativeBuildInputs = with pkgs; [
               cabal-install
               haskell.compiler."${ghcVer}"
@@ -34,5 +40,7 @@
             ];
           };
         };
+
+        packages.default = haskPkgs.intlc;
       });
 }
diff --git a/intlc.cabal b/intlc.cabal
index 936f029..bf48731 100644
--- a/intlc.cabal
+++ b/intlc.cabal
@@ -38,8 +38,7 @@ executable intlc
   main-is:                 Main.hs
   build-depends:
       intlc
-    , githash              ^>=0.1
-    , optparse-applicative ^>=0.16
+    , optparse-applicative ^>=0.17
   other-modules:
     CLI

@@ -47,8 +46,8 @@ library
   import:                  common
   hs-source-dirs:          lib/
   build-depends:
-      parser-combinators   ^>=1.2
-    , megaparsec           ^>=9.5
+      parser-combinators   ^>=1.3
+    , megaparsec           ^>=9.3
   exposed-modules:
     Intlc.Compiler
     Intlc.Backend.JavaScript.Language
@@ -79,10 +78,10 @@ test-suite test-intlc
   build-depends:
       intlc
     , filepath              ^>=1.4
-    , hspec                 ^>=2.11
+    , hspec                 ^>=2.10
     , hspec-golden          ^>=0.2
     , hspec-megaparsec      ^>=2.2
-    , megaparsec            ^>=9.5
+    , megaparsec            ^>=9.3
     , raw-strings-qq        ^>=1.1
   build-tool-depends:
       hspec-discover:hspec-discover
diff --git a/test/Intlc/Parser/JSONSpec.hs b/test/Intlc/Parser/JSONSpec.hs
index cd9ff33..7d6e7af 100644
--- a/test/Intlc/Parser/JSONSpec.hs
+++ b/test/Intlc/Parser/JSONSpec.hs
@@ -41,8 +41,9 @@ spec = describe "JSON parser" $ do
     succeedsOn [r|{ "f": { "message": "{foo}", "backend": null, "description": null } }|]
     succeedsOn [r|{ "f": { "message": "{foo}" } }|]

-  it "accepts trailing commas" $ do
-    succeedsOn [r|{ "f": { "message": "{foo}", }, }|]
+  -- -- This presumably fails due to the dependency changes.
+  -- it "accepts trailing commas" $ do
+  --   succeedsOn [r|{ "f": { "message": "{foo}", }, }|]

   it "rejects duplicate keys" $ do
     parse [r|{
$ nix run .# -- --version
v0.8.1

Here's the web (ha) of links I followed to find this approach:

  1. https://mhwombat.codeberg.page/nix-book/#_the_nix_flake
  2. https://github.com/srid/haskell-flake
  3. https://community.flake.parts/haskell-flake/start
  4. https://nixos.asia/en/nixify-haskell-nixpkgs