statically linked GHC libraries on darwin still depend on haskell libraries

domenkozar commented 7 years ago

Linux:

$ nix-store -qR /nix/store/2f55fiad4ybm18clkwz2ar628vp2x83s-stack-1.3.0
/nix/store/m7a3k9cgci9mz114c8497f89w28c5dzn-glibc-2.24
/nix/store/3hz3n8x1lsdm3xz4xy0j1bzrrlka02gk-libyaml-0.1.7
/nix/store/9jwwgvd5cc2g3cyq3bwdcjb71znw9x05-zlib-1.2.8
/nix/store/3rgj4hgy23l3vfp1sfn9bwn3fn05vgzh-gcc-5.4.0-lib
/nix/store/w88qskb3jcvpfc7y2vzb87hx5680brwg-gmp-6.1.1
/nix/store/2f55fiad4ybm18clkwz2ar628vp2x83s-stack-1.3.0

Darwin:

$ nix-store -qR /nix/store/y9l7dc6bd0r1wf21qbfwzkdsbc0l0c2v-stack-1.3.0
/nix/store/9m5h3sigwabyr9a821vnaw9zwv5ls7gl-Libsystem-osx-10.11.6
/nix/store/ddjhfiqb2zqafsznw4ipbq9lz02yf964-bash-4.3-p46
/nix/store/1148bsdx708s78xpl5grkra05cigzqwq-libffi-3.2.1
/nix/store/4prh5805mv437z9y1jnjrx00qshgpkix-libc++abi-3.7.1
/nix/store/85i2j9ya564xm4qnzg58lyccsmn8d899-zlib-1.2.8
/nix/store/a0nin3wv4fm3jc9rprq1k569m3r5rxbl-ncurses-5.9
/nix/store/s8gw9wbmwvy5rbsz8plwkh99yik9pwmd-libc++-3.7.1
/nix/store/szkqjvcx3zvs1f7ssj2xlpbsmxsk30mf-zlib-1.2.8-dev
/nix/store/arvfhdmyvy9s7skpsacx85cq2jigpzmm-ncurses-5.9-man
/nix/store/ziqd39zgnsjblfzg6mcazjm1v7hx68a2-ncurses-5.9-dev
/nix/store/12ickjbl8rkfp6z2amkxvp9fld9fkcsq-llvm-3.7.1
/nix/store/fhjhmlcap6m2jflljrd9wdpcipb9d7ry-clang-3.7.1
/nix/store/5i5nky923zsngnvz2xvsgf13628m4zk6-pcre-8.39
/nix/store/sbklms5hlwqyjn2wr33qz0vcx671zhs5-libiconv-osx-10.11.6
/nix/store/nrnk6g621bhl1qr12w1d4adqky5v7x9c-gnugrep-2.25
/nix/store/3zf09m8ykc41pya4yw41q75vh5k26g73-binutils-2.27
/nix/store/fg2b6g0miy0sxn3nwnqbqxc37gixp6c8-binutils-2.27-dev
/nix/store/p0v80jz6saiyq5d1srrj2pac4hn88zk7-cctools-port-886
/nix/store/x4gxf1lh9gfd7qlih3izja2w11bhd0wy-cctools-binutils-darwin
/nix/store/xyblsjninr97nq7fq128x5msbgx3sljk-gmp-6.1.1
/nix/store/y6s43yxzrlgdwgj985dgkl0qmgdr7lcj-coreutils-8.25
/nix/store/82mpl5dz8zflwa022yaidqpx38yyak2v-clang-wrapper-3.7.1
/nix/store/b4xw568x40r2bpmb7cx8fs08ky1gf67w-perl-5.22.2
/nix/store/qw3mv1wfzg8pl4z8xhrh4cgqxig8gr4k-gmp-6.1.1-dev
/nix/store/xmhxj56bnmi35lpb7x0ykfn709hs3gfl-ghc-8.0.1-doc
/nix/store/7296lbnzj1sqzalg2kl5j6q7anyx6i2d-ghc-8.0.1
/nix/store/08qmf9hlxpdkibcx1a79hp9j3hpxvbib-unbounded-delays-0.1.0.9
/nix/store/2gkcnrnsj5299fvg9p1sgapcsxqy130b-text-1.2.2.1
/nix/store/0fcb00qza8h3ra21zizhamy4xrap2jq2-ansi-terminal-0.6.2.3
/nix/store/3168x24mmr3c3b1rxy2a12kiwabsma3d-xml-1.3.14
/nix/store/48hqk35wdwd3yjdbvawnwgcjdvc1jpkn-hostname-1.0
/nix/store/5cc35aqx849sczhr3ascnzfa4y09k5sy-mtl-2.2.1
/nix/store/fiwkn1gqykyczxs6kixcwn6fn63lbmds-ansi-wl-pprint-0.6.7.3
/nix/store/mn7gz7a5sqi09wdgyd3hz4pxsh8gk84h-regex-base-0.93.2
/nix/store/gnqcn9hybrlsfannygy1s80vb5q49xp7-regex-posix-0.95.2
/nix/store/l9l3ysf7d9r9bg6va75zb1q1hlz36bkk-random-1.1
/nix/store/rkil86rk9ry5mxhnib2i6z2ik2smgr2h-old-locale-1.0.0.7
/nix/store/5hdar32vyf3wxxbwrffprxk76kw5ijsh-test-framework-0.8.1.1
/nix/store/ywdm6pslnq2d8g19ymy7cfjim3dbdrp3-primitive-0.6.1.0
/nix/store/7h59wjm6lm3m01bm4s12838isivk27i0-tf-random-0.5
/nix/store/5clsycck5ah171s6qgr3kcnhk2ax5sdg-QuickCheck-2.8.2
/nix/store/ly34f3q59gx2l8dg6w9j3p40wyyvimqf-extensible-exceptions-0.1.1.4
/nix/store/yvsf20m35pqckl8jcj28yiai2zy7ym2d-test-framework-quickcheck2-0.3.0.3
/nix/store/a0x8q1swhzrkm3yg6vp4n78r9y49ssqx-vector-0.11.0.0
/nix/store/xwvwh9i5hdfn9mygzf5qwmjqsfjln6pz-th-lift-0.7.6
/nix/store/0blw3wr5abryj1km1fd1dpp13bd3xm2g-th-lift-instances-0.1.11
/nix/store/3ph129j5i0fc5arnka90c5286i5iy967-icu4c-57.1
/nix/store/n26kkbd83mccf1xs8qgv0pxvlhcjiby3-CF-osx-10.9.5
/nix/store/2iwhv7s9ss7c0pjr29q118rr8rrxpmdq-CF-osx-10.9.5-private
/nix/store/q3zchbqpcsppja0w0smdfq4sgnprmvlp-apple-framework-IOKit
/nix/store/650md7pg6czknia6rr61j024vd4wjqgy-apple-framework-Security
/nix/store/29hrp9yg9nqbl1kvkw8ccw087jz0rfbr-apple-framework-SystemConfiguration
/nix/store/inskk4q8s4vmim6x88jwg1fddpriscwl-MacOS_SDK-10.9
/nix/store/y4h178iwdv1iy6zgjlk74y72q2canl43-apple-lib-xpc
/nix/store/gzpkg6l2h4w83j0cx1xw593m0wyrf151-apple-framework-IOSurface
/nix/store/g95rdvr8bvmkbl0b26campg1n33hi69g-apple-framework-SecurityFoundation
/nix/store/3n7ymnbxa5aq9w6ak8sv08fs092gbkpz-apple-framework-CoreWLAN
/nix/store/7nqzil5ablarqiykmvlf269q85zsh365-apple-framework-IOBluetooth
/nix/store/z759dls8bzrxw6946qlgnryjwy2242ni-apple-framework-Accelerate
/nix/store/fhagyv797dkgv6hv80c8sya1l8pc9v8k-apple-framework-CoreGraphics
/nix/store/0ddsrrgcylv9nh6s9ppznr6qh94rmg1p-apple-framework-CoreText
/nix/store/jclxipp4mkpzwymnxmvp3897jm1l08w0-HUnit-1.3.1.2
/nix/store/8nx1fk1hnff7fcii1dlb3i3wj7aacwg1-hspec-expectations-0.7.2
/nix/store/chz4066g69fj7y6py2y7w38qkk9yhdda-setenv-0.1.1.3
/nix/store/ivq68apjv1ay391acq0wxq5s0nb1y1i7-stm-2.4.4.1
/nix/store/mfzm2f2w7a8617y4banigni65i90wsj6-test-framework-hunit-0.3.0.2
/nix/store/q9g6v634mc6yim0mkkbhc6zl6z2b8vmj-async-2.1.1
/nix/store/r54lndbjsrm2hd6x8ymgmwvlci2qj68a-quickcheck-io-0.1.4
/nix/store/svfy35limf74czsc094hzxpffd25wiyz-hspec-meta-2.2.1
/nix/store/0hwmln0wdcmqxql3bd1v0gfr4kd81z3g-hspec-discover-2.2.4
/nix/store/15lbkcsdrv2bxqz0qkj8vgkcv6frw0qf-quickcheck-unicode-1.0.0.1
/nix/store/194kalanyh3c8y6kf1x98yxxx11yrvii-libsecurity_pkcs12-osx-10.7.5
/nix/store/6sy5kqzrvyavjls80yam2icj8mqcdjf3-clock-0.7.2
/nix/store/k5f153979mrnn00f06zpymlpdbcsibvp-transformers-compat-0.5.1.4
/nix/store/78prj7g5rmciaj6cp6h1lmdq9gplckgm-optparse-applicative-0.13.0.0
/nix/store/9nqzm0wf1j7np8r2p0d6j7jg41dx23mn-tagged-0.8.5
/nix/store/s6h0zi18kkfy2s7092l3v1567cv6pz8y-parsec-3.1.11
/nix/store/rnnl0vmknhg4vk4g1av3fvdkw42r8i5x-regex-tdfa-1.2.2
/nix/store/6j5nlbdvr7xd1giq38cz83zzsn65rq5q-tasty-0.11.0.4
/nix/store/jgwdfcahrb3773dgj4n46v6xh1940904-tasty-hunit-0.9.2
/nix/store/ryb2y5vzqwbg0scb40fzcnizff5ndz38-pcre-8.39-bin
/nix/store/sn9wjcw7lw2i27ir9f71amf59f6l47q9-pcre-8.39-dev
/nix/store/misynd6xdmggc9b209hjp9qm1dvz159z-pcre-light-0.4.0.4
/nix/store/262jfpslywh1dhr8m9wrgzy8c4l5h6s0-tasty-quickcheck-0.8.4
/nix/store/y54hjrfwmn6f2zc4fkvqvb8gbspalq1x-OSXPrivateSDK-f4d52b60e86b496abfaffa119a7d299562d99783-src
/nix/store/ycsb7qzwh90skfr3rmkihpf3fiacx5kx-libsecurity_utilities-osx-10.7.5
/nix/store/2k7r3cads96rmpmq3ki8jdw4bjj29wxy-libsecurity_cdsa_utilities-osx-10.7.5
/nix/store/kxi327vyydvgb85ixkfg7bp0mr50cwba-libsecurity_cdsa_client-osx-10.7.5
/nix/store/rxb3x56qc1csm9fhmazlxac2mf14gl5b-libsecurity_cdsa_plugin-osx-10.7.5
/nix/store/28r5jrsqsyca446qn4wk9nqsfi4111qp-libsecurity_apple_cspdl-osx-10.7.5
/nix/store/4hw18f4c1ncmfr6074hnsnc5r2igih7q-exceptions-0.8.3
/nix/store/73ndxa5h7ksr0iag7ngk4x8id0r6xx2h-nanospec-0.2.1
/nix/store/95llvxk66m086rckr6ml85spgslc11iz-temporary-1.2.0.4
/nix/store/zal6kkpzb6r7visnj5gbnfs0x30s57wa-silently-1.2.5
/nix/store/gxwrcglmkmhcz9a9k1j7li4q4kvsqfp6-hspec-core-2.2.4
/nix/store/j4ga5xxlm04di3ps15wb089k65ysncvr-stringbuilder-0.5.0
/nix/store/pjrlymzqf1qk9ygrds86731q8fg2xv1n-hspec-2.2.4
/nix/store/nzkxbvy13py5wasvg32b12v92hmywfaa-generic-deriving-1.11.1
/nix/store/2mdjamqn444zafbqkjq4mvdrl3x4nn13-tasty-ant-xml-1.0.2
/nix/store/2pipdadp6k5jfdh3hycrd83hj116x76z-libsecurityd-osx-10.7.5
/nix/store/gpmaiz2zb8cxavdd73f82ap9zd1da8jg-hashable-1.2.4.0
/nix/store/nrilfncr0zm9a2figsz899678miyvwrr-logict-0.6.0.2
/nix/store/c5kjd80iwqy5jqmcxi6avkil91fbgjir-smallcheck-1.1.1
/nix/store/y61ld7m2mjq3jfw36z2f5pz5w18si19d-tasty-smallcheck-0.8.1
/nix/store/si3in6bbdx3p5b36nbsbb629iapddrmq-scientific-0.3.4.9
/nix/store/347ry9hy5k3jbc7p4z045gyadrjyc99i-attoparsec-0.13.1.0
/nix/store/3d67dl97bq2yawy61dkj7q26xzx4dzn9-apple-framework-GSS
/nix/store/44zdlpxp0lrx0rbjvsjwy6hjy9f3ix4a-libsecurity_apple_file_dl-osx-10.7.5
/nix/store/5b2yj7v5wb1s8hvps0ipgdyqw1z9g9dl-apple-framework-CFNetwork
/nix/store/5l6iiv9qj48xhiysx9714yvv7mnh7jqn-base-compat-0.9.1
/nix/store/65i14fp7bd985s7c7zpwqdpn3c68c8ms-apple-framework-Kerberos
/nix/store/igcc9aaq1sln7fzl1c4ym4gxmrz8plmm-polyparse-1.12
/nix/store/xrd1hk37b5id2hyjr8xn8clw4146psza-old-time-1.1.0.3
/nix/store/6d1h6f3va8653a8pqh0ybg43x9isda7v-cpphs-1.20.2
/nix/store/79kvjrc6aiwzr88iysg3a0lzjgn3rvyx-libsecurity_asn1-osx-10.7.5
/nix/store/hhf2xk6frmkj09vn7a82i0lpgff3apfl-apple-framework-CoreData
/nix/store/8bjc7l525k05nx8dhay09k7gk2p4mjww-apple-framework-PCSC
/nix/store/a7vm87z42mf4s22j6y40bylmvssz28xb-libsecurity_filedb-osx-10.7.5
/nix/store/bq9nya6ilz2wrpvbk8iw94r9kwj65n32-libsecurity_apple_x509_tp-osx-10.7.5
/nix/store/cxqm38g4chp0648h24gxla1y0pj15vp2-libsecurity_cdsa_utils-osx-10.7.5
/nix/store/f2vippjpzz4ff324kizz2ijp4n2rln9x-libsecurity_apple_csp-osx-10.7.5
/nix/store/hjwkzw60mncpjvp7iq4m8016yrnklpir-libsecurity_keychain-osx-10.7.5
/nix/store/l3y569lrqhibprr4mxzywzc76a7y6fjj-libsecurity_sd_cspdl-osx-10.7.5
/nix/store/mlip1b6ad49gnb4aix30856gwr0vkvyq-libsecurity_apple_x509_cl-osx-10.7.5
/nix/store/v22xwp6xcznlswnk6g3w2m9mk9hphdcf-libsecurity_cssm-osx-10.7.5
/nix/store/xwxvj035va9hc34xjm35akfqicv1yb1q-libsecurity_mds-osx-10.7.5
/nix/store/krnmg41ii4pdzz1ddgq6dfchs8ylhkxl-apple-framework-NetFS
/nix/store/nmkd976l43f3vmikbgaxn0s1zx6kca9n-apple-framework-CoreAudio
/nix/store/qm2p2rjyiqaxpl981d13ff15w8insnmc-apple-framework-OpenDirectory
/nix/store/r1n8brr4s24731khp95rrjg3dbl2r0lq-apple-framework-ServiceManagement
/nix/store/zzwcfdwz4111z84ypajbi3pfyvjs6lrx-apple-framework-DiskArbitration
/nix/store/gji2p6ah1cb9im5ska6ay585l3d5xlvl-apple-framework-CoreServices
/nix/store/r5wr20virxh1apnjwr61s83hzz5z4mkx-apple-framework-ImageIO
/nix/store/88rdiyg8y9sn2kz231gy6qc4qnlrsb6m-apple-framework-ApplicationServices
/nix/store/hqpjhpmslnc34hpjkpgdcm4szhdpbjb2-objc4-osx-10.11.6
/nix/store/zhi418sxl4sslymc8j7ihabiipbbnnil-apple-framework-Foundation
/nix/store/7b5qd8r1x52pbcpcqlvbq6z2pim0k37x-SecurityTool-55115
/nix/store/7j0m7yljvnpzl7bmj9faw5r122zs7p2c-transformers-base-0.4.4
/nix/store/s7fi75ggaj9imqlp40ydy3w1y9ba10dr-dlist-0.8.0.2
/nix/store/7lhs016npraf8zw1kj3b10f8wc6hbnpj-Glob-0.7.13
/nix/store/824xzd90aca7k16xs9dnqy06qdrwdv0g-syb-0.6
/nix/store/j1xzzv71haz1ns4gvkz85ypzk0dvz3x2-safe-0.3.10
/nix/store/vs6yki3srwk3cbvb4myam5mvblakr726-th-expand-syns-0.4.1.0
/nix/store/9f5vni0jrwmaiy1jzhxk4x9adf54k16b-th-reify-many-0.1.6
/nix/store/qq6s1pc6grjrchds6a1aywv8n5gw1j3s-monad-control-1.0.1.0
/nix/store/bzqrhgb9a9xqz6f4ygq5glm0avhszmk2-lifted-base-0.2.3.8
/nix/store/rhzgjl3440lkav77kaaqzabnxj8sj67l-mmorph-1.0.9
/nix/store/cainhi890qzfqzk3df0xqqphlywp9qy6-resourcet-1.1.8.1
/nix/store/fr4cmyyrlcqqzxqq7hwdipn9mvirg0v2-semigroups-0.18.2
/nix/store/hj4janaphadp78z1ns2q5mkkixk5rbr9-th-orphans-0.13.3
/nix/store/sd8l7j9jsgw1yhwbdiglh6zgff0hs7bd-haskell-src-exts-1.17.1
/nix/store/ix91phbz4sij385vax75nxdlchri4src-haskell-src-meta-0.6.0.14
/nix/store/jl9f35zwgbnil9b1kdzlnnbq4m027bsv-enclosed-exceptions-1.0.2
/nix/store/qbs62v9d8883blhl0kmf0hc4qd92ghip-ChasingBottoms-1.3.1.2
/nix/store/ljcjfv8r48w2cz3b7hbkggr7dnh639rk-unordered-containers-0.2.7.1
/nix/store/h997262xrj10pzpjm3s2377myzhvmrsd-quickcheck-instances-0.3.12
/nix/store/hll4kpc74xkpjjqnkhvd1h3j0wfcvl6s-hashable-time-0.2
/nix/store/p2ssikdidcs1g2k7r7qkqyhblymdrwrv-time-locale-compat-0.1.1.3
/nix/store/qgk0mwcn0vy2ccrvjnmf8qyzaaznzmic-base-orphans-0.5.4
/nix/store/sp27dz4x43xy3jqnb19jjik1rvxxsqfc-base16-bytestring-0.1.1.6
/nix/store/n1w2gdcdi8ivj7f13d2yxvdgcfb77jw6-aeson-1.0.2.1
/nix/store/q6mb9kl5m0x8y2xxp5ps7ygdf0ax3sv4-conduit-1.2.8
/nix/store/rnxwgzg6c9cqhz2v2lg1zs2wpnravpsq-libyaml-0.1.7
/nix/store/yi41pqqgvfp862z6kpni3kk2jr7xpc85-logging-facade-0.1.1
/nix/store/w62kzcj65p6bkz6npz8mgvjf462r3v8a-mockery-0.3.4
/nix/store/x8l3l11y9llzjqn87rb2n1by9kmgbkd3-aeson-qq-0.8.1
/nix/store/8sbjhq3k9ajijxidns92dq3x04pkasha-yaml-0.8.21.1
/nix/store/dfrfn1w53ml27314f3ky9i2scdrx2c51-hpack-0.15.0
/nix/store/y9l7dc6bd0r1wf21qbfwzkdsbc0l0c2v-stack-1.3.0

Can something be done about it? I didn't see any notes around stack and executable sharing on darwin. It's 2.7G that needs to be fetched for CI to build projects with stack.

cc @peti @copumpkin

mitchty commented 7 years ago

As a note, this causes ghc bug 12479 where the OS X sierra linker no longer accepts mach o binaries with greater than 32k of linked libraries to run:

$ stack new whatever
dyld: malformed mach-o: load commands size (40960) > 32768

If you run otool on the binary and add up the path lengths for each library, it adds up to the 40960 length. Its likely this will also affect any other haskell binary on sierra that uses a lot of libraries.

$ otool -l $(which stack) | grep cmdsize | awk '{print $2}' | paste -s -d+ - | bc
40960

domenkozar commented 7 years ago

Use ghc802 and it should work :)

peti commented 7 years ago

This issue might be caused by the fact that the generic Haskell builder cannot use --enable-split-objs on Darwin. Is there something like ldd on Darwin? I.e. could someone please check whether the stack binary on Darwin actually uses all those shared libraries? This is what the output on Linux looks like:

$ nix-shell -p stack --run 'ldd $(type -p stack)'
    linux-vdso.so.1 (0x00007ffcc39f4000)
    libz.so.1 => /nix/store/nlvp2f6p41k0bf3ymqkmsg2llqm19xwp-zlib-1.2.8/lib/libz.so.1 (0x00007f80c3d72000)
    libpthread.so.0 => /nix/store/8lbpq1vmajrbnc96xhv84r87fa4wvfds-glibc-2.24/lib/libpthread.so.0 (0x00007f80c3b55000)
    libyaml-0.so.2 => /nix/store/fajxyfbvqqlwq9wbkzn7qg9fkcf3mcbp-libyaml-0.1.7/lib/libyaml-0.so.2 (0x00007f80c3936000)
    librt.so.1 => /nix/store/8lbpq1vmajrbnc96xhv84r87fa4wvfds-glibc-2.24/lib/librt.so.1 (0x00007f80c372e000)
    libutil.so.1 => /nix/store/8lbpq1vmajrbnc96xhv84r87fa4wvfds-glibc-2.24/lib/libutil.so.1 (0x00007f80c3529000)
    libdl.so.2 => /nix/store/8lbpq1vmajrbnc96xhv84r87fa4wvfds-glibc-2.24/lib/libdl.so.2 (0x00007f80c3325000)
    libgmp.so.10 => /nix/store/z658anjp1piihwl3z4q5pkvvkpdmc28r-gmp-6.1.1/lib/libgmp.so.10 (0x00007f80c3092000)
    libm.so.6 => /nix/store/8lbpq1vmajrbnc96xhv84r87fa4wvfds-glibc-2.24/lib/libm.so.6 (0x00007f80c2d8d000)
    libc.so.6 => /nix/store/8lbpq1vmajrbnc96xhv84r87fa4wvfds-glibc-2.24/lib/libc.so.6 (0x00007f80c29ef000)
    /nix/store/8lbpq1vmajrbnc96xhv84r87fa4wvfds-glibc-2.24/lib/ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2 (0x00005627621a7000)

domenkozar commented 7 years ago

@peti good call. GHC 8.0.2 does fix split-obj for darwin again, so I'll give that a go.

$ otool -L $(which stack)
/nix/store/zv8724aimblrr3r9sdp5q3979sh1xbpj-stack-1.2.0/bin/stack:
        /System/Library/Frameworks/Cocoa.framework/Versions/A/Cocoa (compatibility version 1.0.0, current version 22.0.0)
        /nix/store/85i2j9ya564xm4qnzg58lyccsmn8d899-zlib-1.2.8/lib/libz.dylib (compatibility version 1.0.0, current version 1.2.8)
        /nix/store/9m5h3sigwabyr9a821vnaw9zwv5ls7gl-Libsystem-osx-10.11.6/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)
        /nix/store/rnxwgzg6c9cqhz2v2lg1zs2wpnravpsq-libyaml-0.1.7/lib/libyaml-0.2.dylib (compatibility version 3.0.0, current version 3.5.0)
        /nix/store/sbklms5hlwqyjn2wr33qz0vcx671zhs5-libiconv-osx-10.11.6/lib/libiconv.dylib (compatibility version 7.0.0, current version 7.0.0)
        /nix/store/xyblsjninr97nq7fq128x5msbgx3sljk-gmp-6.1.1/lib/libgmp.10.dylib (compatibility version 14.0.0, current version 14.1.0)
        /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1255.1.0)
        /System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices (compatibility version 1.0.0, current version 728.4.0)

peti commented 7 years ago

Just for reference: the relevant bit in the generic builder is at https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/haskell-modules/generic-builder.nix#L23.

domenkozar commented 7 years ago

Compiling the GHCs chain to test this. Will report once it's done.

mitchty commented 7 years ago

I doubt split objs has anything to do with the rpaths/library search paths in the stack binary to be honest. The 8.0.1 compiled stack on sierra yields me:

$ otool -L $(which stack)                                                                                                  
/Users/mitch/.nix-profile/bin/stack:
    /System/Library/Frameworks/Cocoa.framework/Versions/A/Cocoa (compatibility version 1.0.0, current version 22.0.0)
    @rpath/libHSstack-1.3.0-uDodXDTdCbLoUnaaN9Fjs-ghc8.0.1.dylib (compatibility version 0.0.0, current version 0.0.0)
    @rpath/libHSzip-archive-0.3.0.5-CbemXvsU6SOJkcGcsRzKMK-ghc8.0.1.dylib (compatibility version 0.0.0, current version 0.0.0)
    @rpath/libHSdigest-0.0.1.2-Jkg2DJlP1uNCnEMQswznXb-ghc8.0.1.dylib (compatibility version 0.0.0, current version 0.0.0)
    @rpath/libHSvector-binary-instances-0.2.3.3-ITLfSosnCjgDHIm70KxMIu-ghc8.0.1.dylib (compatibility version 0.0.0, current version 0.0.0)
******** tons of ghc libraries omitted
    @rpath/libHSinteger-gmp-1.0.0.1-ghc8.0.1.dylib (compatibility version 0.0.0, current version 0.0.0)
    @rpath/libHSghc-prim-0.5.0.0-ghc8.0.1.dylib (compatibility version 0.0.0, current version 0.0.0)
    @rpath/libHSrts_thr-ghc8.0.1.dylib (compatibility version 0.0.0, current version 0.0.0)
    @rpath/libffi.dylib (compatibility version 7.0.0, current version 7.4.0)
    /nix/store/9i841xhnnhm4kyrbqdgcf0rk2yjj587w-zlib-1.2.8/lib/libz.dylib (compatibility version 1.0.0, current version 1.2.8)
    /nix/store/0bvc64agjngcsbnzpqvsl7v2pnh3xnz8-Libsystem-osx-10.11.6/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)
    /nix/store/72jppdc4d3sirwskys8znzm36zq6w65h-libyaml-0.1.7/lib/libyaml-0.2.dylib (compatibility version 3.0.0, current version 3.5.0)
    /nix/store/8ph66agckz5xhl04bfkrf9w8r0w88sv8-libiconv-osx-10.11.6/lib/libiconv.dylib (compatibility version 7.0.0, current version 7.0.0)
    /nix/store/7m21kxr5i7qfcnvgr8380dbviixlz1ai-gmp-6.1.1/lib/libgmp.10.dylib (compatibility version 14.0.0, current version 14.1.0)
    /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1259.0.0)

I'm about 99.9% sure its due to enableSharedExecutables evaluating to true. I hard set it to false and got this:

[nix-shell:~/Downloads/stack-1.3.2-osx-x86_64]$ otool -L $(which ghc-mod)
/nix/store/b6kv8i68wkx0a79lmvzfqdn5c7zzw6v0-ghc-mod-5.6.0.0/bin/ghc-mod:
    /nix/store/8ph66agckz5xhl04bfkrf9w8r0w88sv8-libiconv-osx-10.11.6/lib/libiconv.dylib (compatibility version 7.0.0, current version 7.0.0)
    /nix/store/7m21kxr5i7qfcnvgr8380dbviixlz1ai-gmp-6.1.1/lib/libgmp.10.dylib (compatibility version 14.0.0, current version 14.1.0)
    /nix/store/0bvc64agjngcsbnzpqvsl7v2pnh3xnz8-Libsystem-osx-10.11.6/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)
    /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1348.28.0)

Vs

$ otool -L $(which ghc-mod)                                                                                                
/Users/mitch/.nix-profile/bin/ghc-mod:
    @rpath/libHSghc-mod-5.6.0.0-2IkcKvuWEXD2E5X5gwZnGy-ghc8.0.1.dylib (compatibility version 0.0.0, current version 0.0.0)
    @rpath/libHStemporary-1.2.0.4-HrDpr7Xe8eIBlscuzEzXU8-ghc8.0.1.dylib (compatibility version 0.0.0, current version 0.0.0)
    @rpath/libHSsplit-0.2.3.1-IJKK4pOCYGKud0jZImZke-ghc8.0.1.dylib (compatibility version 0.0.0, current version 0.0.0)

So I would presume either cabal is getting -dynamic or -optl=-shared passed in by default here: https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/haskell-modules/generic-builder.nix#L21

peti commented 7 years ago

I doubt split objs has anything to do with the rpaths/library search paths in the stack binary to be honest.

No, split-objs has nothing to do with rpaths compiled into the stack binary. It it does have an effect on the store paths that Nix might consider a run-time dependency (as opposed to a build-time dependency).

mitchty commented 7 years ago

Looks to be !isCross evaluating to true from this commit, reverting the enableSharedExecutables line to before restores static haskell libraries being linked into executables. https://github.com/NixOS/nixpkgs/commit/64ec4dd87bf7b211773541fa350ef2f56b9c658f

domenkozar commented 7 years ago

@mitchty I'm not following, which line exactly?

domenkozar commented 7 years ago

http://hydra.nixos.org/job/nixpkgs/trunk/stack.x86_64-darwin#tabs-charts says it always had 2.5TB of closure size.

mitchty commented 7 years ago

@domenkozar https://github.com/NixOS/nixpkgs/commit/64ec4dd87bf7b211773541fa350ef2f56b9c658f#diff-360bed088faac7580cee137eeffb39e0L20

Presumably all the other lines with isCross are also evaluating not as expected in this case, but that one alone gets me binaries with ghc libraries statically linked. Stack fails to build with it fixed, but other things, ghc-mod as my prior example, build as expected by forcing it to false or removing !isCross.

The stack build error with it set to false (only thing that seems to fail to build I have hit yet):

[  1 of 121] Compiling Text.PrettyPrint.Leijen.Extended ( src/Text/PrettyPrint/Leijen/Extended.hs, dist/build/Text/PrettyPrint/Leijen/Extended.o )
[  2 of 121] Compiling Stack.Ghci.Script ( src/Stack/Ghci/Script.hs, dist/build/Stack/Ghci/Script.o )
[  3 of 121] Compiling Stack.FileWatch  ( src/Stack/FileWatch.hs, dist/build/Stack/FileWatch.o )
[  4 of 121] Compiling System.Process.PagerEditor ( src/System/Process/PagerEditor.hs, dist/build/System/Process/PagerEditor.o )
[  5 of 121] Compiling System.Process.Log ( src/System/Process/Log.hs, dist/build/System/Process/Log.o )
[  6 of 121] Compiling Paths_stack      ( dist/build/autogen/Paths_stack.hs, dist/build/Paths_stack.o )
[  7 of 121] Compiling Path.Find        ( src/Path/Find.hs, dist/build/Path/Find.o )
[  8 of 121] Compiling Path.Extra       ( src/Path/Extra.hs, dist/build/Path/Extra.o )
[  9 of 121] Compiling System.Process.Read ( src/System/Process/Read.hs, dist/build/System/Process/Read.o )
ghc: panic! (the 'impossible' happened)
  (GHC version 8.0.1 for x86_64-apple-darwin):
    Loading temp shared object failed: dlopen(/private/var/folders/_r/lyjgk4qd501fhnw2n416kygw0000gn/T/nix-build-stack-1.3.0.drv-0/ghc12706_0/libghc_73.dylib, 5): no suitable image found.  Did find:
    /private/var/folders/_r/lyjgk4qd501fhnw2n416kygw0000gn/T/nix-build-stack-1.3.0.drv-0/ghc12706_0/libghc_73.dylib: malformed mach-o: load commands size (40216) > 32768

Please report this as a GHC bug:  http://www.haskell.org/ghc/reportabug

builder for ‘/nix/store/j9amrhc69z4hn1rbk6x4b3csf58f9b9y-stack-1.3.0.drv’ failed with exit code 1
error: build of ‘/nix/store/j9amrhc69z4hn1rbk6x4b3csf58f9b9y-stack-1.3.0.drv’ failed

The libghc dylib name looks suspicious for 8.0.1, so have to negate its not my setup somehow.

domenkozar commented 7 years ago

@mitchty that's a bug in GHC with Sierra, you have to use 8.0.2-rc2:

diff --git a/pkgs/top-level/all-packages.nix b/pkgs/top-level/all-packages.nix
index 73b2510122..fcc813971a 100644
--- a/pkgs/top-level/all-packages.nix
+++ b/pkgs/top-level/all-packages.nix
@@ -4959,7 +4959,7 @@ in

   cabal-install = haskell.lib.disableSharedExecutables haskellPackages.cabal-install;

-  stack = haskell.lib.overrideCabal haskellPackages.stack (drv: {
+  stack = haskell.lib.overrideCabal haskell.packages.ghc802.stack (drv: {
     enableSharedExecutables = false;
     isLibrary = false;
     doHaddock = false;

The issue with 8.0.2 is that packages won't compile yet :)

domenkozar commented 7 years ago

I have the following patch

diff --git a/pkgs/development/haskell-modules/configuration-ghc-8.0.x.nix b/pkgs/development/haskell-modules/configuration-ghc-8.0.x.nix
index 2760489770..84814a2e3c 100644
--- a/pkgs/development/haskell-modules/configuration-ghc-8.0.x.nix
+++ b/pkgs/development/haskell-modules/configuration-ghc-8.0.x.nix
@@ -66,4 +66,9 @@ self: super: {
   # https://github.com/Deewiant/glob/issues/8
   Glob = doJailbreak super.Glob;

+  vector-algorithms = dontCheck super.vector-algorithms;
+  path-io = doJailbreak super.path-io;
+
+  stack = doJailbreak super.stack;
+
 }
diff --git a/pkgs/development/haskell-modules/generic-builder.nix b/pkgs/development/haskell-modules/generic-builder.nix
index dfe9f68ec6..3e65b6193c 100644
--- a/pkgs/development/haskell-modules/generic-builder.nix
+++ b/pkgs/development/haskell-modules/generic-builder.nix
@@ -18,9 +18,9 @@
 , enableLibraryProfiling ? false
 , enableExecutableProfiling ? false
 # TODO enable shared libs for cross-compiling
-, enableSharedExecutables ? !isCross && (((ghc.isGhcjs or false) || stdenv.lib.versionOlder "7.7" ghc.version))
-, enableSharedLibraries ? !isCross && (((ghc.isGhcjs or false) || stdenv.lib.versionOlder "7.7" ghc.version))
-, enableSplitObjs ? !stdenv.isDarwin # http://hackage.haskell.org/trac/ghc/ticket/4013
+, enableSharedExecutables ? ((ghc.isGhcjs or false) || stdenv.lib.versionOlder "7.7" ghc.version)
+, enableSharedLibraries ? ((ghc.isGhcjs or false) || stdenv.lib.versionOlder "7.7" ghc.version)
+, enableSplitObjs ? !(stdenv.isDarwin && (stdenv.lib.versionAtLeast "8.0.1" ghc.version)) # http://hackage.haskell.org/trac/ghc/ticket/4013
 , enableStaticLibraries ? true
 , extraLibraries ? [], librarySystemDepends ? [], executableSystemDepends ? []
 , homepage ? "http://hackage.haskell.org/package/${pname}"
diff --git a/pkgs/top-level/all-packages.nix b/pkgs/top-level/all-packages.nix
index 465a872aa2..9643a77267 100644
--- a/pkgs/top-level/all-packages.nix
+++ b/pkgs/top-level/all-packages.nix
@@ -4959,7 +4959,7 @@ in

   cabal-install = haskell.lib.disableSharedExecutables haskellPackages.cabal-install;

-  stack = haskell.lib.overrideCabal haskellPackages.stack (drv: {
+  stack = haskell.lib.overrideCabal haskell.packages.ghc802.stack (drv: {
     enableSharedExecutables = false;
     isLibrary = false;
     doHaddock = false;

but it fails with

dist/build/Language/Haskell/HsColour_o_split:
getDirectoryContents:openDirStream: does not exist (No such file or directory)
builder for _/nix/store/9girdlhfv307ld4163irbv4wl5kxjv8d-hscolour-1.24.1.drv_ failed with exit code 1

domenkozar commented 7 years ago

$ ls /private/var/folders/v0/ydgx93x96375ws14_16p93gr0000gq/T/nix-build-hscolour-1.24.1.drv-0/hscolour-1.24.1/dist/build/Language/Haskell/
HsColour        HsColour.dyn_hi HsColour.dyn_o  HsColour.hi     HsColour.o

mitchty commented 7 years ago

Yeah I can't actually get stack to build/run on sierra at all. I've given up and just installed it from their release page to get stuff done.

domenkozar commented 7 years ago

This might be the clue:

Preprocessing library hscolour-1.24.1...

on the commandline: warning: ignoring -fsplit-objs

More: https://ghc.haskell.org/trac/ghc/ticket/8405

domenkozar commented 7 years ago

Did some research and clang has different ways of stripping dead code:

diff --git a/pkgs/top-level/all-packages.nix b/pkgs/top-level/all-packages.nix
index ac48186d58..faf7227df9 100644
--- a/pkgs/top-level/all-packages.nix
+++ b/pkgs/top-level/all-packages.nix
@@ -4965,7 +4965,8 @@ in

   cabal-install = haskell.lib.disableSharedExecutables haskellPackages.cabal-install;

-  stack = haskell.lib.overrideCabal haskellPackages.stack (drv: {
+  stack = haskell.lib.overrideCabal haskell.packages.ghc802.stack (drv: {
+    configureFlags = (drv.configureFlags or []) ++ ["--ghc-option=-optl=-dead_strip"];
     enableSharedExecutables = false;
     isLibrary = false;
     doHaddock = false;

results into:

$ du -shc $(nix-store -qR /nix/store/djcqgdxcgdgnmavfl6nj4ajf71qciqr1-stack-1.3.2)
9.1M    /nix/store/fv1bdv2jxvzhd2k49hlvdnl7cfy1pcxf-Libsystem-osx-10.11.6
720K    /nix/store/86gcwgh050vyw09qllcf0zj26wy99h54-bash-4.3-p46
149M    /nix/store/5hhh1jfrmgl4rn2jngz6r7fi8qxxa1l5-MacOS_SDK-10.9
4.0K    /nix/store/6zw35r9i711rfwrgb4njcgkgp2i4fi38-apple-lib-xpc
728K    /nix/store/zajqc7n5zab4p45vi28nim29aa93piwm-libc++abi-3.7.1
4.7M    /nix/store/1ngsnx8kljrd7hv8n7hhd0jhpvf372r9-libc++-3.7.1
30M     /nix/store/rvl82bvj906av3x7hc4qf512v1yxgiqv-icu4c-57.1
2.5M    /nix/store/z88y3w9vfy90zk5qdbqrlkf91sr4fkcd-CF-osx-10.9.5
2.5M    /nix/store/mjhhp17a0r41y047k0baxh1vh1rgdl7j-apple-framework-IOKit
40K     /nix/store/f1ca8wp9c3rfrcpjjlwll3qcz8im9vyh-apple-framework-IOSurface
704K    /nix/store/rhvycdmy6dpx35ykdg8c9s2h33x3c4x8-apple-framework-IOBluetooth
16K     /nix/store/2r5ka9hfjn3vrp1mjncvhpsiihiw7r3w-apple-framework-SecurityFoundation
92K     /nix/store/vvivbb6wsgw17bdp0jrwqgi9axh04xnx-apple-framework-CoreWLAN
1.5M    /nix/store/mqjn93jawm82y10mr0jry68vavsvl2zc-apple-framework-Accelerate
1.3M    /nix/store/2bpi30nzglvnanf3yxshp2jhvnfrfqqi-apple-framework-Security
380K    /nix/store/qa3x8lzvc32day837wj4nqxk76wrhlyb-apple-framework-SystemConfiguration
416K    /nix/store/j1j7yv1pfaqdx1qpym5y3dmmvgx2xpb2-apple-framework-CoreGraphics
124K    /nix/store/1gaiwkkfsqdy449ka7svz41haa4lsv13-apple-framework-ImageIO
196K    /nix/store/3ldkk82pnakw3v58fpmvxnk6xbm2sdgw-libsecurity_pkcs12-osx-10.7.5
204K    /nix/store/539gbh93kbwryj9cqlwq75qh28if7gmz-apple-framework-CoreData
80K     /nix/store/3qj4db757f9sbrgrsh5c6z236fglpdq0-apple-framework-PCSC
256K    /nix/store/7bl50dxd6wa0sm85d38digcldg5nq8ah-libsecurity_apple_x509_cl-osx-10.7.5
490M    /nix/store/y54hjrfwmn6f2zc4fkvqvb8gbspalq1x-OSXPrivateSDK-f4d52b60e86b496abfaffa119a7d299562d99783-src
860K    /nix/store/a1y0092sxs2rp2shxn400bby5f91wni9-libsecurity_utilities-osx-10.7.5
612K    /nix/store/kj1rnxww2sl5pj96h9w448lf0nhaxa1p-libsecurity_cdsa_utilities-osx-10.7.5
464K    /nix/store/84x0g1dc4faqk9dp0dk72948rgngfwxp-libsecurityd-osx-10.7.5
248K    /nix/store/8b94p16kn07h1pwy9fk08qjrqq0jjpkx-libsecurity_mds-osx-10.7.5
100K    /nix/store/8dphhdcnf1gg5qnjhk5c7qz1cnfa2vpd-libsecurity_cdsa_utils-osx-10.7.5
324K    /nix/store/a8mddg7csz8xjr2ixlv7lv8zda1nyh9v-apple-framework-Kerberos
500K    /nix/store/fmfa3g3c64iak7w780i669602h6y0k32-libsecurity_cdsa_client-osx-10.7.5
404K    /nix/store/m73rxkn8kxf8xzcsgh50qca717lhiwzc-libsecurity_cdsa_plugin-osx-10.7.5
268K    /nix/store/ckvp8mbxmiskqhmqsqg93dfyf9fszsdq-libsecurity_apple_cspdl-osx-10.7.5
528K    /nix/store/f159slgh2f6x4z7m1wzkrxz32sb1z0jh-libsecurity_apple_csp-osx-10.7.5
176K    /nix/store/9pp2pz51kd2mrrkv4f6k3f4j0lsc9g2c-objc4-osx-10.11.6
60K     /nix/store/5z6454hs2yhvx8sqg4fq3yfinhsdw6b4-apple-framework-DiskArbitration
480K    /nix/store/bnxy413gki5jp85h4xdqmpjihgbykqgs-apple-framework-CoreAudio
28K     /nix/store/djzja14djy32x0q9szdrdr9i3dc11aai-apple-framework-ServiceManagement
40K     /nix/store/i26mxkx48inxhy379lj0bclcvbd27d5d-apple-framework-NetFS
240K    /nix/store/km7vyi3j26wjwn93jwaln2lss647lw0j-apple-framework-CFNetwork
292K    /nix/store/m0xw096862n5alflp3h46qgkgvva1acw-apple-framework-OpenDirectory
3.6M    /nix/store/53j0y0nv9kaw3wrr9gp8agvnx2rl7f7z-apple-framework-CoreServices
368K    /nix/store/j5608s54vhqznhgdirbmw7anbg9ypi00-CF-osx-10.9.5-private
372K    /nix/store/6hbimrhrsp019b710kqxc6blw18qnqyj-apple-framework-CoreText
2.1M    /nix/store/smncyigx1ml4g713lmkijw6j2mxqrr7w-apple-framework-ApplicationServices
1.3M    /nix/store/fyvwq8qpxkw8bngq1b78zd1i4dkfq26f-apple-framework-Foundation
304K    /nix/store/ibsgflm5xl35bmv89lj8ywj1q0bfhy4h-libsecurity_filedb-osx-10.7.5
36K     /nix/store/iij69b2qq2im9mvg4nfvl3c8n1r08jz1-libsecurity_apple_file_dl-osx-10.7.5
172K    /nix/store/p37mdz9g2f7jl2pnialr7fd5h6gg2267-libsecurity_sd_cspdl-osx-10.7.5
100K    /nix/store/p81z4nvbfvgawffz6cfv1fa8p2snkagw-apple-framework-GSS
852K    /nix/store/v1srcs3lgzr7hmn5s46p77q4ry5lhnm4-libsecurity_cssm-osx-10.7.5
2.0M    /nix/store/vzqlq4v9wz53g1naa8w8f3fd4rlr4hj6-libsecurity_keychain-osx-10.7.5
292K    /nix/store/x596kpg8awg7azcmxzkdp81m9sgksrn1-libsecurity_apple_x509_tp-osx-10.7.5
464K    /nix/store/xgvkcmhc0z6gqxz1c7i35rlx6s7d8h1g-libsecurity_asn1-osx-10.7.5
196K    /nix/store/1qa2x0l65l9byijva6s28zmar4qwalrq-SecurityTool-55115
644K    /nix/store/cjvzla6gqw5rjrnh1pl18kax78864ic1-gmp-6.1.1
196K    /nix/store/jjqb4ql67gfdp0k404cwii1l7r4wz2yf-libyaml-0.1.7
100K    /nix/store/l5yvr296hdx8bv4ny38rb7w677nc4mjv-zlib-1.2.8
1.1M    /nix/store/lxb2z1zdz9rd5bs8p255xaipyfrzqlw6-libiconv-osx-10.11.6
63M     /nix/store/djcqgdxcgdgnmavfl6nj4ajf71qciqr1-stack-1.3.2
775M    total

@copumpkin do we really need following at runtime?

490M    /nix/store/y54hjrfwmn6f2zc4fkvqvb8gbspalq1x-OSXPrivateSDK-f4d52b60e86b496abfaffa119a7d299562d99783-src

domenkozar commented 7 years ago

[nix-shell:~/nixpkgs-stack]$ grep -R /nix/store/y54hjrfwmn6f2zc4fkvqvb8gbspalq1x-OSXPrivateSDK-f4d52b60e86b496abfaffa119a7d299562d99783-src /nix/store/a1y0092sxs2rp2shxn400bby5f91wni9-libsecurity_utilities-osx-10.7.5
/nix/store/a1y0092sxs2rp2shxn400bby5f91wni9-libsecurity_utilities-osx-10.7.5/include/security_utilities/powerwatch.h:#include "/nix/store/y54hjrfwmn6f2zc4fkvqvb8gbspalq1x-OSXPrivateSDK-f4d52b60e86b496abfaffa119a7d299562d99783-src/PrivateSDK10.9.sparse.sdk/System/Library/Frameworks/IOKit.framework/Versions/A/PrivateHeaders/pwr_mgt/IOPMLibPrivate.h"

[nix-shell:~/nixpkgs-stack]$ grep -R /nix/store/y54hjrfwmn6f2zc4fkvqvb8gbspalq1x-OSXPrivateSDK-f4d52b60e86b496abfaffa119a7d299562d99783-src /nix/store/f159slgh2f6x4z7m1wzkrxz32sb1z0jh-libsecurity_apple_csp-osx-10.7.5
/nix/store/f159slgh2f6x4z7m1wzkrxz32sb1z0jh-libsecurity_apple_csp-osx-10.7.5/include/security_apple_csp/castContext.h:#include "/nix/store/y54hjrfwmn6f2zc4fkvqvb8gbspalq1x-OSXPrivateSDK-f4d52b60e86b496abfaffa119a7d299562d99783-src/PrivateSDK10.9.sparse.sdk/usr/include/CommonCrypto/CommonCryptorSPI.h"
/nix/store/f159slgh2f6x4z7m1wzkrxz32sb1z0jh-libsecurity_apple_csp-osx-10.7.5/include/security_apple_csp/desContext.h:#include "/nix/store/y54hjrfwmn6f2zc4fkvqvb8gbspalq1x-OSXPrivateSDK-f4d52b60e86b496abfaffa119a7d299562d99783-src/PrivateSDK10.9.sparse.sdk/usr/include/CommonCrypto/CommonCryptorSPI.h"
/nix/store/f159slgh2f6x4z7m1wzkrxz32sb1z0jh-libsecurity_apple_csp-osx-10.7.5/include/security_apple_csp/gladmanContext.h:#include "/nix/store/y54hjrfwmn6f2zc4fkvqvb8gbspalq1x-OSXPrivateSDK-f4d52b60e86b496abfaffa119a7d299562d99783-src/PrivateSDK10.9.sparse.sdk/usr/include/CommonCrypto/CommonCryptorSPI.h"
/nix/store/f159slgh2f6x4z7m1wzkrxz32sb1z0jh-libsecurity_apple_csp-osx-10.7.5/include/security_apple_csp/rc4Context.h:#include "/nix/store/y54hjrfwmn6f2zc4fkvqvb8gbspalq1x-OSXPrivateSDK-f4d52b60e86b496abfaffa119a7d299562d99783-src/PrivateSDK10.9.sparse.sdk/usr/include/CommonCrypto/CommonCryptorSPI.h"

domenkozar commented 7 years ago

Seems like we should just depend on:

pkgs/os-specific/darwin/apple-source-releases/CommonCrypto/default.nix
pkgs/os-specific/darwin/apple-source-releases/IOKit/default.nix

domenkozar commented 7 years ago

I'll add my fixes to https://github.com/NixOS/nixpkgs/pull/21589 which brings closure down to ~700MB, but if we want stack closure to be down to ~150MB on darwin we'll have to wait for https://github.com/NixOS/nixpkgs/issues/15718#issuecomment-270119262

copumpkin commented 7 years ago

@copumpkin do we really need following at runtime?

No, I'm really sad that OSXPrivateSDK is used anywhere and I don't know how it got introduced. A huge chunk of my stdenv work was to avoid bringing that in. Also, no clue why it would be a runtime dependency, but that makes even less sense than using it in the first place 😄

More broadly, how does this tie to the SDK being too old? Not saying it doesn't, but I don't follow the argument that it does.

domenkozar commented 7 years ago

Dunno, I was just asking if upgrading would make it better :)

copumpkin commented 7 years ago

Oh, I doubt it. I'd just use that new utility function to strip all references to OSXPrivateSDK for now and file an issue that it's being used. Is that the last big part of the closure? I'll need to track down where and how it appeared.

domenkozar commented 7 years ago

Back to the issue, this is clearly a bug somewhere. Same problem with purescript, which should be built with static haskell libraries.

The whole closure is 3GB, including haskell dependencies.

$ otool -L /nix/store/djlilwh82n6qachy184406c18qag7bdq-purescript-0.10.7/bin/psci

/nix/store/djlilwh82n6qachy184406c18qag7bdq-purescript-0.10.7/bin/psci:
        /nix/store/ncc1wn41rawn8k8v1pa4gcyj4n3mlazn-zlib-1.2.11/lib/libz.dylib (compatibility version 1.0.0, current version 1.2.11)
        /nix/store/il826qrrls32gjnmcqjlmhxxhrcy5lgz-Libsystem-osx-10.11.6/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)
        /nix/store/nwxk93kcnlby666wjngr6wfmcxn71pk8-ncurses-5.9/lib/libncursesw.5.dylib (compatibility version 5.0.0, current version 5.0.0)
        /nix/store/g734ixf6p1d0s5afsrfg9h1x8b274y46-libiconv-osx-10.11.6/lib/libiconv.dylib (compatibility version 7.0.0, current version 7.0.0)
        /nix/store/ysf9dgxj6qak2wl67b31jn0bbijxqlqg-gmp-6.1.1/lib/libgmp.10.dylib (compatibility version 14.0.0, current version 14.1.0)
        /System/Library/Frameworks/Cocoa.framework/Versions/A/Cocoa (compatibility version 1.0.0, current version 22.0.0)
        /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1255.1.0)

It's linked correctly, as expected, just system libs.

Let's inspect Nix runtime graph:

$ nix-store -q --graph /nix/store/djlilwh82n6qachy184406c18qag7bdq-purescript-0.10.7 | grep purescript
"/nix/store/djlilwh82n6qachy184406c18qag7bdq-purescript-0.10.7" [label = "purescript-0.10.7", shape = box, style = filled, fillcolor = "#ff0000"];
"/nix/store/g734ixf6p1d0s5afsrfg9h1x8b274y46-libiconv-osx-10.11.6" -> "/nix/store/djlilwh82n6qachy184406c18qag7bdq-purescript-0.10.7" [color = "black"];
"/nix/store/il826qrrls32gjnmcqjlmhxxhrcy5lgz-Libsystem-osx-10.11.6" -> "/nix/store/djlilwh82n6qachy184406c18qag7bdq-purescript-0.10.7" [color = "red"];
"/nix/store/ncc1wn41rawn8k8v1pa4gcyj4n3mlazn-zlib-1.2.11" -> "/nix/store/djlilwh82n6qachy184406c18qag7bdq-purescript-0.10.7" [color = "green"];
"/nix/store/nwxk93kcnlby666wjngr6wfmcxn71pk8-ncurses-5.9" -> "/nix/store/djlilwh82n6qachy184406c18qag7bdq-purescript-0.10.7" [color = "blue"];
"/nix/store/wgb1mi4dwmpalsbrln3qjx6klx3y0d64-warp-3.2.11.1" -> "/nix/store/djlilwh82n6qachy184406c18qag7bdq-purescript-0.10.7" [color = "magenta"];
"/nix/store/wjcrp83pyv9d8rcx6afxwrkqbbddjhvv-SecurityTool-55115" -> "/nix/store/djlilwh82n6qachy184406c18qag7bdq-purescript-0.10.7" [color = "burlywood"];
"/nix/store/ysf9dgxj6qak2wl67b31jn0bbijxqlqg-gmp-6.1.1" -> "/nix/store/djlilwh82n6qachy184406c18qag7bdq-purescript-0.10.7" [color = "black"];

Aha, somehow it still depends on warp.

$ grep -R /nix/store/wgb1mi4dwmpalsbrln3qjx6klx3y0d64-warp-3.2.11.1 /nix/store/djlilwh82n6qachy184406c18qag7bdq-purescript-0.10.7
Binary file /nix/store/djlilwh82n6qachy184406c18qag7bdq-purescript-0.10.7/bin/psci matches

Binary references the warp package, this doesn't happen on Linux, but just macOS.

I tried using clang dead code elimination, but it fails to work:

1 of 2] Compiling Paths_purescript ( dist/build/autogen/Paths_purescript.hs, dist/build/psci/psci-t
mp/Paths_purescript.o )
[2 of 2] Compiling Main             ( psci/Main.hs, dist/build/psci/psci-tmp/Main.o )

<no location info>: error:
    <command line>: can't load .so/.DLL for: /private/var/folders/wd/pxddfzvn77s7kzmwryg8lx080000gq/
T/nix-build-purescript-0.10.7.drv-0/purescript-0.10.7/dist/build/libHSpurescript-0.10.7-K6ha0Q2gF7P4
tlJGxzjqAd-ghc8.0.2.dylib (dlopen(/private/var/folders/wd/pxddfzvn77s7kzmwryg8lx080000gq/T/nix-build
-purescript-0.10.7.drv-0/purescript-0.10.7/dist/build/libHSpurescript-0.10.7-K6ha0Q2gF7P4tlJGxzjqAd-
ghc8.0.2.dylib, 5): no suitable image found.  Did find:
        /private/var/folders/wd/pxddfzvn77s7kzmwryg8lx080000gq/T/nix-build-purescript-0.10.7.drv-0/p
urescript-0.10.7/dist/build/libHSpurescript-0.10.7-K6ha0Q2gF7P4tlJGxzjqAd-ghc8.0.2.dylib: malformed
mach-o: load commandssize (33216) > 32768)
builder for _/nix/store/mg26g3z9siq97k17hjhiib39f1y1pmll-purescript-0.10.7.drv_ failed with exit cod
e 1
error: build of _/nix/store/mg26g3z9siq97k17hjhiib39f1y1pmll-purescript-0.10.7.drv_ failed

copumpkin commented 7 years ago

I mentioned this on IRC but the warp references are all inside the Paths module generated by Cabal. Unclear (to me at least) if Cabal behaves on Darwin or if some of our Nix Haskell wrapper stuff clears out extraneous references differently on Darwin and Linux.

domenkozar commented 7 years ago

@Ericson2314 is this somewhat familiar with you? :)

Ericson2314 commented 7 years ago

I don't know the details, but static linking + data dirs causing problems is not at all surprising to me.

domenkozar commented 7 years ago

Still happening with latest purescript:

$ cat -v /nix/store/wdl9l3a7a80dwhjl6ymnwvx9x6wfnfv7-purescript-0.11.4/bin/purs | grep /nix/store/qs0l00pl22wnjwfiydxbh34v79ahbn3j-warp-3.2.11.2              
^@^@^@^@^@^@ ^@^@^@^@^@^@^@File not found^@^@text/plain; charset=utf-8^@^@^@^@^@^@^@keep-alive^@^@^@^@^@^@close^@^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Response.RspNoBody^@^@^@^@^@^@^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Response.RspFile^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Response.RspFile^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Response.RspBuilder^@^@^@^@^@^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Response.RspBuilder^@^@^@^@^@^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Response.RspStream^@^@^@^@^@^@^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Response.RspStream^@^@^@^@^@^@^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Response.RspRaw^@^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Response.RspRaw^@^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT^@^@^@^@Network.Wai.Handler.Warp.ResponseHeader^@HTTP/1.0 ^@^@^@^@^@^@^@HTTP/1.1 ^@^@^@^@^@^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT^@^@^@^@Network.Wai.Handler.Warp.Run^@^@^@^@error^@^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT^@^@^@^@Network.Wai.Handler.Warp.Run^@^@^@^@./Network/Wai/Handler/Warp/Run.hs^@^@^@^@^@^@^@keepAliveRef not filled^@UNKNOWN^@TCP6^@^@^@^@TCP4^@^@^@^@PROXY^@^@^@PRI ^@^@^@^@PROXY ^@^@ww Port^@ww HostPreference^@^@^@^@^@^@^@ww Maybe Manager^@^@^@^@^@^@^@^@ww Int^@^@ww IO ()^@^@^@^@^@^@^@^@ww IO () -> IO ()^@^@^@^@^@^@^@ww Maybe Int^@^@^@^@PORT^@^@^@^@Invalid value in $PORT: ^@^@^@^@^@^@^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT^@^@^@^@Network.Wai.Handler.Warp.SendFile^@^@^@^@^@^@^@positionRead^@^@^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT^@^@^@^@Network.Wai.Handler.Warp.Settings^@^@^@^@^@^@^@'ProxyProtocolOptional^@^@'ProxyProtocolRequired^@^@'ProxyProtocolNone^@^@^@^@^@^@ProxyProtocol^@^@^@'Settings^@^@^@^@^@^@^@Settings^@^@^@^@^@^@^@^@text/plain; charset=utf-8^@^@^@^@^@^@^@Exception: ^@^@^@^@^@Something went wrong^@^@^@^@Bad Request^@^@^@^@^@*4^@^@^@^@^@^@Warp/^@^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Settings.ProxyProtocolNone^@^@^@^@^@^@^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Settings.ProxyProtocolRequired^@^@^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Settings.ProxyProtocolOptional^@^@^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Settings.Settings^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Settings.Settings^@Tom ^@^@^@^@One ^@^@^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT^@^@^@^@Network.Wai.Handler.Warp.Some^@^@^@'Tom^@^@^@^@'One^@^@^@^@Some^@^@^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Some.One^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Some.One^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Some.Tom^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Some.Tom^@^@Thread killed by Warp's timeout reaper^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT^@^@^@^@Network.Wai.Handler.Warp.Timeout^@^@^@^@^@^@^@^@TimeoutThread^@^@^@'Canceled^@^@^@^@^@^@^@'Paused^@'Inactive^@^@^@^@^@^@^@'Active^@State^@^@^@'Handle^@Handle^@^@'TimeoutThread^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Timeout.TimeoutThread^@^@^@^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Timeout.Active^@^@^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Timeout.Inactive^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Timeout.Paused^@^@^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Timeout.Canceled^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Timeout.Handle^@^@^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Timeout.Handle^@^@^@^@Warp: Request headers too large, possible memory attack detected. Closing connection.^@^@^@Warp: Client closed connection prematurely^@^@^@^@^@^@Warp: Request headers did not finish transmission^@^@^@^@^@^@^@Warp: Request line specified a non-HTTP request^@Warp: Incomplete request headers, received: ^@^@^@^@Warp: Invalid first line of request: ^@^@^@Warp: Invalid PROXY protocol header: ^@^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT^@^@^@^@Network.Wai.Handler.Warp.Types^@^@InvalidRequest^@^@tlsChiperID^@^@^@^@^@tlsNegotiatedProtocol^@^@^@tlsMinorVersion^@tlsMajorVersion^@'BadProxyHeader^@'OverLargeHeader^@^@^@^@^@^@^@^@'ConnectionClosedByPeer^@'IncompleteHeaders^@^@^@^@^@^@'NonHttp^@^@^@^@^@^@^@^@'BadFirstLine^@^@^@'NotEnoughLines^@'FileId^@FileId^@^@'Connection^@^@^@^@^@Connection^@^@^@^@^@^@'InternalInfo0^@^@InternalInfo0^@^@^@'InternalInfo1^@^@InternalInfo1^@^@^@'InternalInfo^@^@^@InternalInfo^@^@^@^@'Source^@Source^@^@'TLS^@^@^@^@'TCP^@^@^@^@Transport^@^@^@^@^@^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Types.TCP^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Types.TLS^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Types.TLS^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Types.Source^@^@^@^@^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Types.Source^@^@^@^@^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Types.InternalInfo^@^@^@^@^@^@^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Types.InternalInfo^@^@^@^@^@^@^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Types.InternalInfo1^@^@^@^@^@^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Types.InternalInfo1^@^@^@^@^@^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Types.InternalInfo0^@^@^@^@^@^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Types.InternalInfo0^@^@^@^@^@^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Types.Connection^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Types.Connection^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Types.FileId^@^@^@^@^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Types.FileId^@^@^@^@^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Types.NotEnoughLines^@^@^@^@^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Types.NotEnoughLines^@^@^@^@^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Types.BadFirstLine^@^@^@^@^@^@^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Types.BadFirstLine^@^@^@^@^@^@^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Types.NonHttp^@^@^@^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Types.IncompleteHeaders^@^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Types.ConnectionClosedByPeer^@^@^@^@^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Types.OverLargeHeader^@^@^@^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Types.BadProxyHeader^@^@^@^@^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT:Network.Wai.Handler.Warp.Types.BadProxyHeader^@^@^@^@^@^@/nix/store/qs0l00pl22wnjwfiydxbh34v79ahbn3j-warp-3.2.11.2/bin^@^@^@/nix/store/qs0l00pl22wnjwfiydxbh34v79ahbn3j-warp-3.2.11.2/lib/ghc-8.0.2/warp-3.2.11.2^@^@^@/nix/store/qs0l00pl22wnjwfiydxbh34v79ahbn3j-warp-3.2.11.2/lib/ghc-8.0.2/x86_64-osx-ghc-8.0.2^@^@^@^@/nix/store/qs0l00pl22wnjwfiydxbh34v79ahbn3j-warp-3.2.11.2/share/x86_64-osx-ghc-8.0.2/warp-3.2.11.2^@^@^@^@^@^@/nix/store/qs0l00pl22wnjwfiydxbh34v79ahbn3j-warp-3.2.11.2/libexec^@^@^@^@^@^@^@/nix/store/qs0l00pl22wnjwfiydxbh34v79ahbn3j-warp-3.2.11.2/etc^@^@^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT^@^@^@^@Paths_warp^@^@^@^@^@^@warp_bindir^@^@^@^@^@warp_libdir^@^@^@^@^@warp_dynlibdir^@^@warp_datadir^@^@^@^@/^@^@^@^@^@^@^@warp_libexecdir^@warp_sysconfdir^@warp-3.2.11.2-EhC54e12K323P2W7GoXWIT^@^@^@^@Network.Wai.Handler.Warp.MultiMap^@^@^@^@^@^@^@rangeLength^@^@^@^@^@rangeOffset^@^@^@^@^@simple-sendfile-0.2.25-JJ9zm1wZxYIARxlVa3oQgs^@^@^@Network.Sendfile.Types^@^@'PartOfFile^@^@^@^@^@'EntireFile^@^@^@^@^@FileRange^@^@^@^@^@^@^@simple-sendfile-0.2.25-JJ9zm1wZxYIARxlVa3oQgs:Network.Sendfile.Types.EntireFile^@simple-sendfile-0.2.25-JJ9zm1wZxYIARxlVa3oQgs:Network.Sendfile.Types.PartOfFile^@simple-sendfile-0.2.25-JJ9zm1wZxYIARxlVa3oQgs:Network.Sendfile.Types.PartOfFile^@simple-sendfile-0.2.25-JJ9zm1wZxYIARxlVa3oQgs^@^@^@Network.Sendfile.BSD^@^@^@^@Network.SendFile.MacOS.sendloop^@Network.SendFile.MacOS.sendloopHeader^@^@^@SfHdtr^@^@'SfHdtr^@IOVec^@^@^@'IOVec^@^@Network.Sendfile.IOVec^@^@simple-sendfile-0.2.25-JJ9zm1wZxYIARxlVa3oQgs^@^@^@simple-sendfile-0.2.25-JJ9zm1wZxYIARxlVa3oQgs:Network.Sendfile.IOVec.SfHdtr^@^@^@^@^@simple-sendfile-0.2.25-JJ9zm1wZxYIARxlVa3oQgs:Network.Sendfile.IOVec.SfHdtr^@^@^@^@^@simple-sendfile-0.2.25-JJ9zm1wZxYIARxlVa3oQgs:Network.Sendfile.IOVec.IOVec^@^@^@^@^@^@simple-sendfile-0.2.25-JJ9zm1wZxYIARxlVa3oQgs:Network.Sendfile.IOVec.IOVec^@^@^@^@^@^@error^@^@^@iproute-1.7.1-4HULutk0xITKNjI1pwYrD4^@^@^@^@Data.IP.Addr^@^@^@^@./Data/IP/Addr.hs^@^@^@^@^@^@^@enumFromThenTo: Incompatible IP families^@^@^@^@^@^@^@^@enumFromTo: Incompatible IP families^@^@^@^@enumFromThen: Incompatible IP families^@^@IPv6^@^@^@^@'IPv4^@^@^@'IPv6^@^@^@'IP6^@^@^@^@'IP4^@^@^@^@ipv4^@^@^@^@ipv6^@^@^@^@ip4ToIp6^@^@^@^@^@^@^@^@::^@^@^@^@^@^@toWord32^@^@^@^@^@^@^@^@Data/IP/Addr.hs:280:5-44|[x1, x2, x3, x4]^@^@^@^@^@^@^@::ffff:^@foldr1^@^@Data/IP/Addr.hs:296:5-44|[x1, x2, x3, x4]^@^@^@^@^@^@^@IP^@^@^@^@^@^@IPv6^@^@^@^@IPv4^@^@^@^@Data.IP.Addr^@^@^@^@iproute-1.7.1-4HULutk0xITKNjI1pwYrD4^@^@^@^@IP^@^@^@^@^@^@ipv6^@^@^@^@ipv4^@^@^@^@IPv4^@^@^@^@IP6^@^@^@^@^@IP4^@^@^@^@^@minimum^@iproute-1.7.1-4HULutk0xITKNjI1pwYrD4:Data.IP.Addr.IPv4^@^@iproute-1.7.1-4HULutk0xITKNjI1pwYrD4:Data.IP.Addr.IPv4^@^@iproute-1.7.1-4HULutk0xITKNjI1pwYrD4:Data.IP.Addr.IPv6^@^@iproute-1.7.1-4HULutk0xITKNjI1pwYrD4:Data.IP.Addr.IPv6^@^@BigEndian^@^@^@^@^@^@^@LittleEndian^@^@^@^@Mixed^@^@^@Mixed ^@^@LittleEndian^@^@^@^@BigEndian^@^@^@^@^@^@^@byteorder-1.0.4-9NlNHkBVdo7F1prFVKrkcH^@^@System.ByteOrder^@^@^@^@^@^@^@^@'Mixed^@^@'LittleEndian^@^@^@'BigEndian^@^@^@^@^@^@ByteOrder^@^@^@^@^@^@^@Pattern match failure in do expression at System/ByteOrder.hs:51:10-22^@^@byteorder-1.0.4-9NlNHkBVdo7F1prFVKrkcH:System.ByteOrder.BigEndian^@^@^@^@^@^@^@byteorder-1.0.4-9NlNHkBVdo7F1prFVKrkcH:System.ByteOrder.LittleEndian^@^@^@^@byteorder-1.0.4-9NlNHkBVdo7F1prFVKrkcH:System.ByteOrder.Mixed^@^@^@byteorder-1.0.4-9NlNHkBVdo7F1prFVKrkcH:System.ByteOrder.Mixed^@^@^@appar-0.1.4-DGWHHt2WRn9FI49prcwO9^@^@^@^@^@^@^@Text.Appar.Input^@^@^@^@^@^@^@^@'C:Input^@^@^@^@^@^@^@^@Input^@^@^@appar-0.1.4-DGWHHt2WRn9FI49prcwO9:Text.Appar.Input.C:Input^@^@^@^@^@^@appar-0.1.4-DGWHHt2WRn9FI49prcwO9:Text.Appar.Input.C:Input^@^@^@^@^@^@appar-0.1.4-DGWHHt2WRn9FI49prcwO9^@^@^@^@^@^@^@Text.Appar.Parser^@^@^@^@^@^@^@'P^@^@^@^@^@^@MkParser^@^@^@^@^@^@^@^@appar-0.1.4-DGWHHt2WRn9FI49prcwO9:Text.Appar.Parser.P^@^@^@appar-0.1.4-DGWHHt2WRn9FI49prcwO9:Text.Appar.Parser.P^@^@^@http2-1.6.3-Bp7nrXDLFN4GAtMqa4uiIs^@^@^@^@^@^@Network.HPACK^@^@^@}^@^@^@^@^@^@^@tokenKey = ^@^@^@^@^@, ^@^@^@^@^@^@isPseudo = ^@^@^@^@^@shouldBeIndexed = ^@^@^@^@^@^@ix = ^@^@^@Token {^@http2-1.6.3-Bp7nrXDLFN4GAtMqa4uiIs^@^@^@^@^@^@Network.HPACK.Token^@^@^@^@^@'Token^@^@Token^@^@^@for other tokens^@^@^@^@^@^@^@^@TE^@^@^@^@^@^@Connection^@^@^@^@^@^@Www-Authenticate^@^@^@^@^@^@^@^@Via^@^@^@^@^@Vary^@^@^@^@User-Agent^@^@^@^@^@^@Transfer-Encoding^@^@^@^@^@^@^@Strict-Transport-Security^@^@^@^@^@^@^@Set-Cookie^@^@^@^@^@^@Server^@^@Retry-After^@^@^@^@^@Refresh^@Referer^@Range^@^@^@Proxy-Authorization^@^@^@^@^@Proxy-Authenticate^@^@^@^@^@^@Max-Forwards^@^@^@^@Location^@^@^@^@^@^@^@^@Link^@^@^@^@Last-Modified^@^@^@If-Unmodified-Since^@^@^@^@^@If-Range^@^@^@^@^@^@^@^@If-None-Match^@^@^@If-Modified-Since^@^@^@^@^@^@^@If-Match^@^@^@^@^@^@^@^@Host^@^@^@^@From^@^@^@^@Expires^@Expect^@^@Etag^@^@^@^@Date^@^@^@^@Cookie^@^@Content-Type^@^@^@^@Content-Range^@^@^@Content-Location^@^@^@^@^@^@^@^@Content-Length^@^@Content-Language^@^@^@^@^@^@^@^@Content-Encoding^@^@^@^@^@^@^@^@Content-Disposition^@^@^@^@^@Cache-Control^@^@^@Authorization^@^@^@Allow^@^@^@Age^@^@^@^@^@Access-Control-Allow-Origin^@^@^@^@^@Accept^@^@Accept-Ranges^@^@^@Accept-Language^@Accept-Encoding^@Accept-Charset^@^@:status^@:scheme^@:path^@^@^@:method^@:authority^@^@^@^@^@^@te^@^@^@^@^@^@age^@^@^@^@^@via^@^@^@^@^@vary^@^@^@^@host^@^@^@^@from^@^@^@^@link^@^@^@^@etag^@^@^@^@date^@^@^@^@allow^@^@^@:path^@^@^@range^@^@^@accept^@^@expect^@^@server^@^@cookie^@^@:status^@expires^@referer^@refresh^@:scheme^@:method^@location^@^@^@^@^@^@^@^@if-match^@^@^@^@^@^@^@^@if-range^@^@^@^@^@^@^@^@:authority^@^@^@^@^@^@user-agent^@^@^@^@^@^@connection^@^@^@^@^@^@set-cookie^@^@^@^@^@^@retry-after^@^@^@^@^@max-forwards^@^@^@^@content-type^@^@^@^@accept-ranges^@^@^@authorization^@^@^@cache-control^@^@^@if-none-match^@^@^@content-range^@^@^@last-modified^@^@^@accept-charset^@^@content-length^@^@accept-encoding^@accept-language^@content-location^@^@^@^@^@^@^@^@content-encoding^@^@^@^@^@^@^@^@www-authenticate^@^@^@^@^@^@^@^@content-language^@^@^@^@^@^@^@^@transfer-encoding^@^@^@^@^@^@^@if-modified-since^@^@^@^@^@^@^@proxy-authenticate^@^@^@^@^@^@content-disposition^@^@^@^@^@proxy-authorization^@^@^@^@^@if-unmodified-since^@^@^@^@^@strict-transport-security^@^@^@^@^@^@^@access-control-allow-origin^@^@^@^@^@head^@^@^@^@http2-1.6.3-Bp7nrXDLFN4GAtMqa4uiIs:Network.HPACK.Token.Token^@^@^@^@http2-1.6.3-Bp7nrXDLFN4GAtMqa4uiIs:Network.HPACK.Token.Token^@^@^@^@http2-1.6.3-Bp7nrXDLFN4GAtMqa4uiIs^@^@^@^@^@^@Network.HTTP2^@^@^@PRI * HTTP/2.0^M

It does seem like it's part of some data file.

cleverca22 commented 7 years ago

where would be the best place to put an override for purescript?, its currently defined in hackage-packages.nix which is 7.8mb and auto-generated?

this override is enough to remove GHC from the closure: https://gist.github.com/cleverca22/af72df113ec7c49ac2377753db819c1a#file-gistfile1-nix-L15-L19

further investigation points towards configuration-common.nix

peti commented 7 years ago

Where would be the best place to put an override for purescript?

I sometimes add these kind of "clean up overrides" for end-user packages into all-packages.nix, i.e. for stack or for cabal2nix so that users can install the minimal version with nix-env -i <name>. I suppose configuration-common.nix could be an appropriate location, too, though.

domenkozar commented 7 years ago

So on darwin the major culprit are things like https://github.com/yesodweb/wai/commit/3e1ffd1689b588c1c8d60591d5b571ad31dade9a and we're back to https://github.com/NixOS/nixpkgs/issues/4504#issuecomment-308439768

domenkozar commented 7 years ago

stack closure size: http://hydra.nixos.org/job/nixpkgs/trunk/stack.x86_64-darwin#tabs-charts cabal-install closure size: http://hydra.nixos.org/job/nixpkgs/trunk/cabal-install.x86_64-darwin#tabs-charts

domenkozar commented 7 years ago

cabal-install closure should be addressed in 56be4bee34ad4913f5697f6c920090ab5651c668

domenkozar commented 7 years ago

I'll close this catch-all issue, summary:

haskell packages might depend on packaging metadata, which can bring back the whole haskell closure (https://github.com/NixOS/nixpkgs/issues/21200#issuecomment-308441063)
dead stripping is the default on linux, on darwin we can't turn it on by default, but justStaticExecutables uses it for now to reduce the closure size (incl any unused packaging metadata information)
multiple outputs will help reducing closure size on darwin: https://github.com/NixOS/nixpkgs/issues/4504#issuecomment-308439768
haskell packages on darwin have 500MB in closure size due to OSXPrivateSDK https://github.com/NixOS/nixpkgs/issues/26710

expipiplus1 commented 3 years ago

This still seems to happen (with warp) for cross compiling to armv7l-linux.

expipiplus1 commented 3 years ago

warp = appendConfigureFlag super.warp "--enable-split-sections"; seems to fix this, still I wonder why this is only necessary when cross compiling...

expipiplus1 commented 3 years ago

Oops, didn't mean to reopen!

NixOS / nixpkgs

statically linked GHC libraries on darwin still depend on haskell libraries #21200