search

GaloisInc / haskell-tor

A Haskell implementation of the Tor protocol.
BSD 3-Clause "New" or "Revised" License
272 stars 19 forks source link

build using cabal sandbox on ghc-8.0.1 #32

Open vu3rdd opened 8 years ago

vu3rdd commented 8 years ago

I could build with the cabal.config from a stackage snapshot and ghc-8.0.1 and the small attached patch and also run the test suite with one failure. I removed the version pinning in the .cabal file and instead let cabal.config (output of cabal freeze) do the dependency resolution. Not sure if this is acceptable. Please have a look. Thanks.

$ cabal test
Preprocessing library haskell-tor-0.1.2...
Preprocessing test suite 'test-tor' for haskell-tor-0.1.2...
Running 1 test suites...
Test suite test-tor: RUNNING...
TorCell Serialization:
  TorAddress round-trips: [OK, passed 100 tests]
  TorAddress makes sensible ByteStrings: [OK, passed 100 tests]
  ExtendSpec serializes: [OK, passed 100 tests]
  DestroyReason serializes (check #1): [OK, passed 100 tests]
  DestroyReason serializes (check #2): [OK, passed 100 tests]
  HandshakeType serializes (check #1): [OK, passed 100 tests]
  HandshakeType serializes (check #2): [OK, passed 100 tests]
  RelayEndReason serializes: [OK, passed 100 tests]
  RelayCell serializes: [OK, passed 100 tests]
  RelayCell serializes w/ digest: [OK, passed 100 tests]
  RelayCell serializes w/ digest: [OK, passed 100 tests]
  Tor certificates serialize: [OK, passed 100 tests]
Hybrid encryption tests:
  Hybrid encryption works when forced: [OK, passed 100 tests]
  Hybrid encryption works in general: [OK, passed 100 tests]
Handshakes:
  TAP Handshake: [Failed]
*** Failed! (after 1 test): 
Exception:
  src/Tor/Link/DH.hs:(12,11)-(15,4): Missing field in record construction params_bits
0
Exception thrown while printing test case:
  No signing key
  CallStack (from HasCallStack):
    error, called at src/Tor/RouterDesc.hs:77:37 in haskell-tor-0.1.2-9P3MKtkGaARLl9OKYgwwOx:Tor.RouterDesc

TorRNG
(used seed 2403893616338533771)
  NTor Handshake: [OK, passed 100 tests]

         Properties   Total       
 Passed  15           15          
 Failed  1            1           
 Total   16           16          
Test suite test-tor: FAIL
Test suite logged to: dist/test/haskell-tor-0.1.2-test-tor.log
0 of 1 test suites (0 of 1 test cases) passed.
erikd commented 8 years ago

@vu3rdd Well done. I like the idea of using a cabal freeze file to specify a set of versions that work together. My only minor gripe is that this cabal freeze file contains many packages (like all of yesod*) that are neither needed or used by haskell-tor. Would it be possible to edit the freeze file to drop all packages haskell-tor doesn't depend on explicitly?

I also wonder if it would be possible (and make sense) to have different versions of the freeze file for different ghc versions.

vu3rdd commented 8 years ago

@erikd Thanks. Great feeling when one gets a pat from an experienced Haskeller.

The cabal.config file I started with was from a stackage nightly snapshot from two days ago. I then did a cabal freeze to create the one I checked in. It might be possible to drop the packages not needed by haskell-tor. and its transitive dependencies I will take a look.

I am not sure about the freeze files for different ghc versions. Stackage lts releases target specific ghc versions. It could be possible with some shell/symlink tricks to switch cabal.config, I think. But I don't know the cabal way of switching the config file. I will try to dig into it as well.

vu3rdd commented 8 years ago

(/cc @erikd)

I am having problems building it on OSX, so even with cabal.config it is not a reproducible build. :(

$ ghc --version
The Glorious Glasgow Haskell Compilation System, version 8.0.1
$ cabal --version
cabal-install version 1.24.0.0
compiled using version 1.24.0.0 of the Cabal library 
$ cabal sandbox init
$ cabal install --only-dependencies --enable-tests --constraint="tls +hans" --constraint="haskell-tor +hans" 
Resolving ...
[..]
Installed attoparsec-0.13.0.2
Building tls-1.3.8...
Failed to install tls-1.3.8
Build log ( /Users/ram/projects/haskell-tor/.cabal-sandbox/logs/tls-1.3.8.log ):
cabal: Entering directory '/var/folders/fj/71brg2ss7h97c28hxvqwgjbr0000gn/T/cabal-tmp-39361/tls-1.3.8'
Configuring tls-1.3.8...
Building tls-1.3.8...
Preprocessing library tls-1.3.8...
[ 1 of 51] Compiling Network.TLS.Util.ASN1 ( Network/TLS/Util/ASN1.hs, dist/dist-sandbox-85f12e97/build/Network/TLS/Util/ASN1.o )
[ 2 of 51] Compiling Network.TLS.ErrT ( Network/TLS/ErrT.hs, dist/dist-sandbox-85f12e97/build/Network/TLS/ErrT.o )
[ 3 of 51] Compiling Network.TLS.Extension.EC ( Network/TLS/Extension/EC.hs, dist/dist-sandbox-85f12e97/build/Network/TLS/Extension/EC.o )
[ 4 of 51] Compiling Network.TLS.RNG  ( Network/TLS/RNG.hs, dist/dist-sandbox-85f12e97/build/Network/TLS/RNG.o )
[ 5 of 51] Compiling Network.TLS.Imports ( Network/TLS/Imports.hs, dist/dist-sandbox-85f12e97/build/Network/TLS/Imports.o )
[ 6 of 51] Compiling Network.TLS.Util.Serialization ( Network/TLS/Util/Serialization.hs, dist/dist-sandbox-85f12e97/build/Network/TLS/Util/Serialization.o )
[ 7 of 51] Compiling Network.TLS.Util ( Network/TLS/Util.hs, dist/dist-sandbox-85f12e97/build/Network/TLS/Util.o )
[ 8 of 51] Compiling Network.TLS.Crypto.ECDH ( Network/TLS/Crypto/ECDH.hs, dist/dist-sandbox-85f12e97/build/Network/TLS/Crypto/ECDH.o )
[ 9 of 51] Compiling Network.TLS.Crypto.DH ( Network/TLS/Crypto/DH.hs, dist/dist-sandbox-85f12e97/build/Network/TLS/Crypto/DH.o )
[10 of 51] Compiling Network.TLS.Types ( Network/TLS/Types.hs, dist/dist-sandbox-85f12e97/build/Network/TLS/Types.o )
[11 of 51] Compiling Network.TLS.X509 ( Network/TLS/X509.hs, dist/dist-sandbox-85f12e97/build/Network/TLS/X509.o )
[12 of 51] Compiling Network.TLS.Session ( Network/TLS/Session.hs, dist/dist-sandbox-85f12e97/build/Network/TLS/Session.o )
[13 of 51] Compiling Network.TLS.Measurement ( Network/TLS/Measurement.hs, dist/dist-sandbox-85f12e97/build/Network/TLS/Measurement.o )
[14 of 51] Compiling Network.TLS.Crypto ( Network/TLS/Crypto.hs, dist/dist-sandbox-85f12e97/build/Network/TLS/Crypto.o )
[15 of 51] Compiling Network.TLS.MAC  ( Network/TLS/MAC.hs, dist/dist-sandbox-85f12e97/build/Network/TLS/MAC.o )
[16 of 51] Compiling Network.TLS.Struct ( Network/TLS/Struct.hs, dist/dist-sandbox-85f12e97/build/Network/TLS/Struct.o )
[17 of 51] Compiling Network.TLS.Hooks ( Network/TLS/Hooks.hs, dist/dist-sandbox-85f12e97/build/Network/TLS/Hooks.o )
[18 of 51] Compiling Network.TLS.Credentials ( Network/TLS/Credentials.hs, dist/dist-sandbox-85f12e97/build/Network/TLS/Credentials.o )
[19 of 51] Compiling Network.TLS.Wire ( Network/TLS/Wire.hs, dist/dist-sandbox-85f12e97/build/Network/TLS/Wire.o )
[20 of 51] Compiling Network.TLS.Cap  ( Network/TLS/Cap.hs, dist/dist-sandbox-85f12e97/build/Network/TLS/Cap.o )
[21 of 51] Compiling Network.TLS.Backend ( Network/TLS/Backend.hs, dist/dist-sandbox-85f12e97/build/Network/TLS/Backend.o )

Network/TLS/Backend.hs:36:1: error:
    Failed to load interface for ‘Hans.NetworkStack’
    Use -v to see a list of the files searched for.
cabal: Leaving directory '/var/folders/fj/71brg2ss7h97c28hxvqwgjbr0000gn/T/cabal-tmp-39361/tls-1.3.8'
cabal: Error: some packages failed to install:
tls-1.3.8 failed during the building phase. The exception was:
ExitFailure 1

Building with a -v flag shows that the hans .so file is getting picked up. A bit puzzled what is going on.

[..]
creating dist/dist-sandbox-85f12e97/build
/usr/local/bin/ghc --make -fbuilding-cabal-package -O -static -dynamic-too -dynosuf dyn_o -dynhisuf dyn_hi -outputdir dist/dist-sandbox-85f12e97/build -odir dist/dist-sandbox-85f12e97/build -hidir dist/dist-sandbox-85f12e97/build -stubdir dist/dist-sandbox-85f12e97/build -i -idist/dist-sandbox-85f12e97/build -i. -idist/dist-sandbox-85f12e97/build/autogen -Idist/dist-sandbox-85f12e97/build/autogen -Idist/dist-sandbox-85f12e97/build -optP-DINCLUDE_NETWORK -optP-DINCLUDE_HANS -optP-DSSLV2_COMPATIBLE -optP-include -optPdist/dist-sandbox-85f12e97/build/autogen/cabal_macros.h -this-unit-id tls-1.3.8-IMnJjryvxF2AmR1NNbcspy -hide-all-packages -no-user-package-db -package-db /Users/ram/projects/haskell-tor/.cabal-sandbox/x86_64-osx-ghc-8.0.1-packages.conf.d -package-db dist/dist-sandbox-85f12e97/package.conf.inplace -package-id asn1-encoding-0.9.4-G3Eu427lfih60n7Hu41ILm -package-id asn1-types-0.3.2-7OJ0HzVkSNEBIwzF1QvaHd -package-id async-2.1.0-J6Pl8k3L4PKGEpjYdgwiIf -package-id base-4.9.0.0 -package-id bytestring-0.10.8.1 -package-id cereal-0.5.3.0-90KodwwwQanJlL7equBhpB -package-id cryptonite-0.19-9J4hBOjSHa16WTMVURplSj -package-id data-default-class-0.1.2.0-FYQpjIylblBDctdkHAFeXA -package-id hans-3.0.0.1-1RKzO37VeVx93uDAZKQt0c -package-id memory-0.13-9YH63fsEcp4AuUH5L0KHp -package-id mtl-2.2.1-6qsR1PHUy5lL47Hpoa4jCM -package-id network-2.6.3.1-G4Up1CPKbp7DeFsnywOnGG -package-id transformers-0.5.2.0 -package-id x509-1.6.3-B8QQl20I2pFHRBKHO0jkKo -package-id x509-store-1.6.1-KlY7Bt3Dyy7JI96TWNq0ar -package-id x509-validation-1.6.3-XxiipkjjQS8gXXOkvi9eO -XHaskell98 Network.TLS Network.TLS.Cipher Network.TLS.Compression Network.TLS.Internal Network.TLS.Extra Network.TLS.Extra.Cipher Network.TLS.Cap Network.TLS.Struct Network.TLS.Core Network.TLS.Context Network.TLS.Context.Internal Network.TLS.Credentials Network.TLS.Backend Network.TLS.Crypto Network.TLS.Crypto.DH Network.TLS.Crypto.ECDH Network.TLS.ErrT Network.TLS.Extension Network.TLS.Extension.EC Network.TLS.Handshake Network.TLS.Handshake.Common Network.TLS.Handshake.Certificate Network.TLS.Handshake.Key Network.TLS.Handshake.Client Network.TLS.Handshake.Server Network.TLS.Handshake.Process Network.TLS.Handshake.Signature Network.TLS.Handshake.State Network.TLS.Hooks Network.TLS.IO Network.TLS.Imports Network.TLS.MAC Network.TLS.Measurement Network.TLS.Packet Network.TLS.Parameters Network.TLS.Record Network.TLS.Record.Types Network.TLS.Record.Engage Network.TLS.Record.Disengage Network.TLS.Record.State Network.TLS.RNG Network.TLS.State Network.TLS.Session Network.TLS.Sending Network.TLS.Receiving Network.TLS.Util Network.TLS.Util.ASN1 Network.TLS.Util.Serialization Network.TLS.Types Network.TLS.Wire Network.TLS.X509 -Wall -fwarn-tabs -fno-warn-unused-imports
vu3rdd commented 8 years ago

@erikd The cabal.config file looks clean (see the second commit in this PR). The cabal freeze cleaned up the cabal file and puts in all the dependencies including transitive ones in the file.

$ cat cabal.config 
constraints: array ==0.5.1.1,
             base ==4.9.0.0,
             ghc-prim ==0.5.0.0,
             rts ==1.0,
             integer-gmp ==1.0.0.1,
             asn1-encoding ==0.9.4,
             asn1-types ==0.3.2,
             bytestring ==0.10.8.1,
             deepseq ==1.4.2.0,
             hourglass ==0.2.10,
             memory ==0.13,
             async ==2.1.0,
             stm ==2.4.4.1,
             attoparsec ==0.13.0.2,
             containers ==0.5.7.1,
             scientific ==0.3.4.9,
             binary ==0.8.3.0,
             hashable ==1.2.4.0,
             text ==1.2.2.1,
             vector ==0.11.0.0,
             primitive ==0.6.1.0,
             transformers ==0.5.2.0,
             base64-bytestring ==1.0.0.1,
             cereal ==0.5.3.0,
             cryptonite ==0.19,
             fingertree ==0.1.1.0,
             hans ==3.0.0.1,
             BoundedChan ==1.0.3.0,
             heaps ==0.3.3,
             monadLib ==3.7.3,
             psqueues ==0.2.2.2,
             random ==1.1,
             time ==1.6.0.1,
             unix ==2.7.2.0,
             network ==2.6.3.1,
             pretty-hex ==1.0,
             pure-zlib ==0.4,
             tls ==1.3.8,
             data-default-class ==0.1.2.0,
             mtl ==2.2.1,
             x509 ==1.6.3,
             asn1-parse ==0.9.4,
             pem ==0.2.2,
             x509-store ==1.6.1,
             x509-validation ==1.6.3,
             byteable ==0.1.1
erikd commented 8 years ago

I'm pretty sure this:

Network/TLS/Backend.hs:36:1: error:
    Failed to load interface for ‘Hans.NetworkStack’

is due to some incompatibility between hans-3.0.0.1 and the tls package. You might try editing the cabal.config file and forcing hans < 3.0.

vu3rdd commented 8 years ago

Thanks @erikd. That worked perfectly.

acw commented 8 years ago

Hmmm. This is looking like it's failing all over Travis. Any idea what's going on?

vu3rdd commented 8 years ago

@acw I made an attempt to fix the travis failures. Still a wip in the branch https://github.com/vu3rdd/haskell-tor/tree/fix-travis-for-ghc-8.0.1. With that, GHC 7.8.4 based build fails to build hans. The library itself builds, but the web-server executable has an ambiguous import. 7.10.3 and 8.0.1 are successfully built but the testsuite fails with one failure. Here are the results: https://travis-ci.org/vu3rdd/haskell-tor/builds/153913806

erikd commented 8 years ago

The cabal.config fine cannot have tight dependencies on base if you want it to compile with more than one version of ghc. GHC 8.0 ships with base-4.9, GHC 7.10 with base-4.8 and so on. For dependency constaint foe base I would actually suggest == 4.*.

vu3rdd commented 8 years ago

I relaxed the dependency in the cabal.config of base to base >= 4.7 && < 5.0. The dependencies seem to be getting resolved okay now.

vu3rdd commented 8 years ago

I changed it as per your suggestion. Thanks @erikd.

erikd commented 8 years ago

I'm pretty sure some of the other packages that are tightly tied to GHC will also be an issue. For example; array, binary, primitive, transformers and others are likely to cause problems.

vu3rdd commented 8 years ago

I did a travis build of my branch. The logs are here.

GHC 7.8.x

This fails with this following error:

[40 of 43] Compiling Hans.Layer.Tcp   ( src/Hans/Layer/Tcp.hs, dist/build/Hans/Layer/Tcp.o )
[41 of 43] Compiling Hans.Layer.Tcp.Socket ( src/Hans/Layer/Tcp/Socket.hs, dist/build/Hans/Layer/Tcp/Socket.o )
[42 of 43] Compiling Hans.NetworkStack ( src/Hans/NetworkStack.hs, dist/build/Hans/NetworkStack.o )
[43 of 43] Compiling Hans.DhcpClient  ( src/Hans/DhcpClient.hs, dist/build/Hans/DhcpClient.o )
In-place registering hans-2.6.0.0...
Preprocessing executable 'web-server' for hans-2.6.0.0...
[1 of 1] Compiling Main             ( web-server/Main.hs, dist/build/web-server/web-server-tmp/Main.o )
web-server/Main.hs:73:34:
    Ambiguous occurrence ‘defaultTimeLocale’
    It could refer to either ‘Data.Time.defaultTimeLocale’,
                             imported from ‘Data.Time’ at web-server/Main.hs:15:1-16
                             (and originally defined in ‘time-1.6.0.1:Data.Time.Format.Locale’)
                          or ‘System.Locale.defaultTimeLocale’,
                             imported from ‘System.Locale’ at web-server/Main.hs:35:1-30
Failed to install hans-2.6.0.0

GHC 7.10.x and GHC 8.0.1

Fails in the cabal test with one error. I am yet to look into that test failure.

Handshakes:
  TAP Handshake: [Failed]
*** Failed! (after 1 test): 
Exception:
  src/Tor/Link/DH.hs:(12,11)-(15,4): Missing field in record construction params_bits
0
Exception thrown while printing test case: 'No signing key'
TorRNG
(used seed -293077498860149041)
  NTor Handshake: [OK, passed 100 tests]
         Properties   Total       
 Passed  15           15          
 Failed  1            1           
 Total   16           16          
Test suite test-tor: FAIL
Test suite logged to: dist/test/haskell-tor-0.1.2-test-tor.log
0 of 1 test suites (0 of 1 test cases) passed.
The command "cabal test" exited with 1.
acw commented 8 years ago

Got it. The test-suite error can be fixed by something in one of my branches (testing, which is waiting for an upstream TLS release). You can either do some fancy pulling and cherry picking, or you can just copy this change: https://github.com/GaloisInc/haskell-tor/commit/6e37c443ef26fce018d670b693a7f72625692539

... or just tell me you're tired of this, and I'll accept the merge and continue to wait for my upstream TLS change to fix the build problem.

erikd commented 8 years ago

@acw Where are your upstream TLS changes? I don't see a PR at https://github.com/vincenthz/hs-tls/ for that.

acw commented 8 years ago

@erikd It's this one:

https://github.com/vincenthz/hs-tls/pull/149

It's been merged, but an updated version hasn't been pushed to Hackage, yet.

erikd commented 8 years ago

Ah right. Thanks @acw !

erikd commented 8 years ago

@vu3rdd You should probably rebase against master. The build output has come compilier warnings that I have already fixed.

vu3rdd commented 8 years ago

@erikd hmm, it seem already on top of the latest master which has some of your commits on compiler warnings merged in. I am wondering if I managed to introduced some new warnings here.