search

mietek / haskell-on-heroku

Buildpack for deploying Haskell apps
BSD 3-Clause "New" or "Revised" License
131 stars 16 forks source link

Compilation error deploying a snap app #10

Closed mgmeier closed 10 years ago

mgmeier commented 10 years ago

I'm trying to deploy a snap project (VierGewinntMM) to Heroku with this buildpack, but upon first git push, I get an error:

...
Inferring sandbox constraints
-----> Inferring sandbox digest... done, a95fa48
-----> Restoring sandbox a95fa48 (VierGewinntMM-0.1)
       Downloading http://s3.halcyon.sh/linux-ubuntu-10-04-x64/halcyon-sandbox-ghc-7.8.2-VierGewinntMM-0.1-a95fa48522adb262188b1796463009a7596209e4.tar.gz... 404
   *** WARNING: sandbox a95fa48 (VierGewinntMM-0.1) is not prepared
   *** ERROR: Compiling failed

       To prepare, use a one-off PX dyno:
       $ heroku run --size=PX prepare
...

heroku prepare selects the hello-snap sandbox, but still the app won't recompile/run.

In my project, I used the cabal.config file from your hello-snap repo, so that appropriate pre-built packages would be selected (without that, inferred constraints don't match any given sandbox). My project's .cabal file looks like this:

Name:                VierGewinntMM
Version:             0.1
Synopsis:            XXXX
License:             GPL-3
Author:              XXX
Maintainer:          XXX
Stability:           Experimental
Category:            Web
Build-type:          Simple
Cabal-version:       >=1.8

Executable VierGewinntMM
  hs-source-dirs: .
  main-is: MainSnap.hs

  Build-depends:
    aeson                     >= 0.7,
    base                      >= 4     && < 5,
    bytestring,
    hashable                  >= 1.2,
    parallel,
    random,
    text,
    vector                    >= 0.10,
    unordered-containers,
    time,
    snap                      >= 0.13,
    snap-core

  if impl(ghc >= 6.12.0)
    ghc-options: -threaded -Wall -funbox-strict-fields -O2
                 -fno-warn-unused-do-bind
  else
    ghc-options: -threaded -Wall -funbox-strict-fields -O2

Could you please help me in clearing up why compilation fails? It shouldn't be a problem to mention a build-depend which has no constraint in cabal.config (e.g. parallel), should it? I'm, really stuck and advice would be much appreciated.

mietek commented 10 years ago

Thanks. You are right — it should not be a problem to omit a constraint for one of your dependencies in your cabal.config. Unfortunately, right now, this is a problem, and I need to fix it.

I assume you tried doing heroku run --size=PX prepare, which finished with a warning similar to:

...
-----> Built sandbox 6a36ed7 (hello-snap-0.1), 70M
-----> Validating sandbox 6a36ed7 (hello-snap-0.1)
   *** WARNING: Actual sandbox digest is a95fa48
   *** WARNING: Unexpected constraints difference:
...

To move forward, you need a cabal.config with constraints for all dependencies. You can generate one by doing cabal freeze. If you do that, then you need to remove the VierGewinntMM ==0.1 self-constraint from the generated cabal.config (haskell/cabal#1908). Finally, push the change, and do heroku run --size=PX prepare again.

Please note copying a cabal.config from the examples is unnecessary — doing heroku run --size=PX prepare will always prepare all necessary packages, even if no packages have been prepared previously. I need to improve the documentation to make this clear.

Note: prepare is now called build.

mietek commented 10 years ago

This is now tracked as #22 and #23.

mgmeier commented 10 years ago

Thanks for the quick response! I tried cabal freeze (while having my project locally insede a cabal sandbox with a Snap version matching the one of the prepared sandbox, 0.13.2.6), yet heroku run prepare outputs as follows:

...
-----> Scoring matched sandboxes
       Ignoring sandbox 662ac34 (Spock-0.5.1.0) as Spock is not needed
       4    sandbox 82fddfc (base-4.7.0.0)
       Ignoring sandbox 67d3a08 (happstack-lite-7.3.3) as happstack-lite is not needed
       Ignoring sandbox 67d3a08 (hello-happstack-lite-0.1) as happstack-lite is not needed
       Ignoring sandbox 24682b4 (hello-scotty-0.1) as ansi-terminal is not needed
       Ignoring sandbox a95fa48 (hello-snap-0.1) as hashable-1.2.1.0 is needed and not 1.2.2.0
       Ignoring sandbox bb067b2 (hello-snap-with-postgres-0.1) as blaze-textual is not needed
       Ignoring sandbox 662ac34 (hello-spock-0.1) as Spock is not needed
       Ignoring sandbox 310049d (hello-yesod-0.1) as ansi-terminal is not needed
       Ignoring sandbox 023ce4c (hello-yesod-platform-0.1) as QuickCheck is not needed
       Ignoring sandbox 24682b4 (scotty-0.7.3) as ansi-terminal is not needed
       Ignoring sandbox fc7e894 (snap-0.13.2.6) as directory-tree-0.11.0 is needed and not 0.12.0
       Ignoring sandbox 359ab77 (snap-core-0.9.6.2) as hashable-1.2.1.0 is needed and not 1.2.2.0
       Ignoring sandbox 310049d (yesod-1.2.5.2) as ansi-terminal is not needed
       Ignoring sandbox 023ce4c (yesod-platform-1.2.11) as QuickCheck is not needed

The problem I see here is that each constraint in cabal.config must match the one on the sandbox, otherwise the sandbox won't get selected. This seems achievable only by trying to deploy ~100 times and updating the offending constraint in cabal.config manually each time... this is why I tried to use a sane cabal.config from one of your repos in the first place.I could see two seemingly simple workarounds:

  1. publishing a cabal.config for each sandbox, so you know what package versions to use/match.
  2. making sandbox selection regard the main package(s) (e.g. snap) and/or its immediate depencies only, and not the whole dependency tree.

I see that a fix to #22 could possibly affect sandbox selection (by omitting many of the constraints in cabal.config or dropping the necessity of it altogether), but until then, what do think about the above ideas? Especially the first one would help a lot ;)

mietek commented 10 years ago

There appears to be some misunderstanding, most likely due to the documentation being incomplete, for which I offer my apologies.

You should not be trying to match an existing sandbox. You should be preparing your own sandbox, using the following command:

heroku run --size=PX prepare

The --size=PX option is important, as it selects a Heroku dyno with 6GB RAM instead of the 512MB default, which is insufficient to compile most Haskell packages.

There is no error in your quoted output. It shows prepare looked at the available sandboxes and found the best available sandbox is 82fddfc (base-4.7.0.0), which is partially matching your cabal.config with a score of 4 packages matched.

As the next step, prepare should build a new sandbox, using the best available sandbox as a starting point. The new sandbox should fully match your cabal.config. Then, prepare should upload the new sandbox to your Amazon S3 bucket. The new sandbox should be automatically used during your next push.

Have you configured your S3 details properly? In order to upload to S3, you must set at least the following three config vars:

heroku config:set HALCYON_AWS_ACCESS_KEY_ID=...
heroku config:set HALCYON_AWS_SECRET_ACCESS_KEY=...
heroku config:set HALCYON_S3_BUCKET=...

Can you paste the entirety of your output?

Note: prepare is now called build.

mietek commented 10 years ago

Thanks for suggesting the sandbox cabal.config files should be published. In fact, they already are, as they are used during the scoring process. You can download them as follows:

$ source haskell-on-heroku.sh 

$ list_prepared linux-ubuntu-10-04-x64 | grep cabal.config
       Listing s3://s3.halcyon.sh/?prefix=linux-ubuntu-10-04-x64... done
linux-ubuntu-10-04-x64/halcyon-sandbox-ghc-7.6.3-base-4.6.0.1-cfd4fabd55f46d3043050bd13bee1740ee134f96.cabal.config
linux-ubuntu-10-04-x64/halcyon-sandbox-ghc-7.8.2-Spock-0.5.1.0-662ac34ee567f41d6ed34fcbb17b7a26f0ec0cca.cabal.config
linux-ubuntu-10-04-x64/halcyon-sandbox-ghc-7.8.2-base-4.7.0.0-82fddfcd641582152aa7dc64e7729724af1bb576.cabal.config
linux-ubuntu-10-04-x64/halcyon-sandbox-ghc-7.8.2-happstack-lite-7.3.3-67d3a083327525865dcd24191257edbc67b0cdb7.cabal.config
linux-ubuntu-10-04-x64/halcyon-sandbox-ghc-7.8.2-hello-happstack-lite-0.1-67d3a083327525865dcd24191257edbc67b0cdb7.cabal.config
linux-ubuntu-10-04-x64/halcyon-sandbox-ghc-7.8.2-hello-scotty-0.1-24682b4d2a15dc4166289034b22debd2736c8385.cabal.config
linux-ubuntu-10-04-x64/halcyon-sandbox-ghc-7.8.2-hello-snap-0.1-a95fa48522adb262188b1796463009a7596209e4.cabal.config
linux-ubuntu-10-04-x64/halcyon-sandbox-ghc-7.8.2-hello-snap-with-postgres-0.1-bb067b237070a6cbfe9cf9308beb6eded85891d7.cabal.config
linux-ubuntu-10-04-x64/halcyon-sandbox-ghc-7.8.2-hello-spock-0.1-662ac34ee567f41d6ed34fcbb17b7a26f0ec0cca.cabal.config
linux-ubuntu-10-04-x64/halcyon-sandbox-ghc-7.8.2-hello-yesod-0.1-310049d61adc7e8469c10288a81eb07639367185.cabal.config
linux-ubuntu-10-04-x64/halcyon-sandbox-ghc-7.8.2-hello-yesod-platform-0.1-023ce4cd5af44d88a7cf2bd6cdc3041ee0a90cbe.cabal.config
linux-ubuntu-10-04-x64/halcyon-sandbox-ghc-7.8.2-scotty-0.7.3-24682b4d2a15dc4166289034b22debd2736c8385.cabal.config
linux-ubuntu-10-04-x64/halcyon-sandbox-ghc-7.8.2-snap-0.13.2.6-fc7e894dca2a74607cf3146c6c05bcc6df69de32.cabal.config
linux-ubuntu-10-04-x64/halcyon-sandbox-ghc-7.8.2-snap-core-0.9.6.2-359ab779a66fd167fec86283586c5c3541807fc1.cabal.config
linux-ubuntu-10-04-x64/halcyon-sandbox-ghc-7.8.2-yesod-1.2.5.2-310049d61adc7e8469c10288a81eb07639367185.cabal.config
linux-ubuntu-10-04-x64/halcyon-sandbox-ghc-7.8.2-yesod-platform-1.2.11-023ce4cd5af44d88a7cf2bd6cdc3041ee0a90cbe.cabal.config

$ download_prepared linux-ubuntu-10-04-x64 halcyon-sandbox-ghc-7.8.2-snap-0.13.2.6-fc7e894dca2a74607cf3146c6c05bcc6df69de32.cabal.config .
       Downloading s3://s3.halcyon.sh/linux-ubuntu-10-04-x64/halcyon-sandbox-ghc-7.8.2-snap-0.13.2.6-fc7e894dca2a74607cf3146c6c05bcc6df69de32.cabal.config... done

$ head -n 1 halcyon-sandbox-ghc-7.8.2-snap-0.13.2.6-fc7e894dca2a74607cf3146c6c05bcc6df69de32.cabal.config
constraints: HUnit ==1.2.5.2,

Please note this is not part of the solution to your issue.

robstewart57 commented 9 years ago

@mietek said:

heroku run --size=PX prepare

Does that mean that a credit card needs to be registered with Heroku? When trying this command..

$ heroku run --size=PX prepare
Running `prepare` attached to terminal... failed
 !    You must add a credit card to run PX size dynos.
 !    https://heroku.com/billing

Is it possible to use haskell-on-heroku with a free Heroku account? Thanks.

mietek commented 9 years ago

@robstewart57: Please continue the discussion in #31.