stack setup

[tonyday567]

I made a wrapper for tidal that uses stack to install tidal. It pins the tidal version to an exact commit (the stack convention) and is very useful for active development.

https://github.com/tonyday567/tidal-stack

I'm happy to document this if it's useful to others, but I wasn't sure exactly where to start (and if at all) with all the activity happening right now.

Tidal is very current and am pleased it works out of the box for ghc-8.6.2. Nice!

[yaxu]

Thanks Tony. I've not really got my head around stack, do you have suggestions for how to make tidal more stack-friendly?

[tonyday567]

stack is a wrapper around cabal, designed to help with dependencies, and to avoid global ghc an library installation. Instead stack encourages isolated project installations (each project installs the libraries it needs and ghc versions are managed by stack in isolated environments).

Tidal is pretty friendly already - didn't take much to get it current. Just found the dependencies that weren't in stackage (hosc and microspec) and it all worked.

I've thought about tidal and stack more generally and it's tricky to strongly recommend a stack install method. You could put it in stackage like Tidal used to be, but I'm not sure what that achieves. Usually, a package produces an executable or is a library primarily designed to be used by other libraries. I can't think of another haskell project where the main interface is, instead, via ghci. So stack install tidal doesn't immediately solve or do anything: it doesn't globally install an executable or globally install a library. Maybe there's an executable idea in BootTidal.hs, but that's a lot of work and I struggle to see what it achieves.

A major design feature of stack is to isolate projects from global state. That's for people who have lots of haskell projects and a need to jump back and forth between ghc versions. Maybe that isn't the case for the majority of users. It kind of made sense to me to wrap Tidal in another package, but it also may be that an easier way is possible. The easiest is just to clone the Github repo, run stack build --test and then everything will work. But it will only work inside the directory - step outside the directory and stack no longer knows about Tidal at all.

My tentative conclusion is that it isn't suitable as the main install method, but is very useful for development, and the tidal-stack example I linked is then more a how-to.

It was very easy to use stack in emacs - just needed to change tidal-interpreter etc. I haven't used atom but imagine it would be a similar story.

[yaxu]

Thanks @tonyday567, a really helpful summary.

Maybe these are other reasons for using stack?

• more reliable than a cabal install (in practise this doesn't necessarily seem to be the case)
• a path to bring installable via other package managers e.g. Debian's

You imply tidal isn't in stackage any more - that's a shame! It'd be good to fix that if that's the case!

Longer term I think tidal will be a binary that either mimics ghci or has a network interface e.g. a websocket api

[tonyday567]

On hold in stackage:

https://github.com/commercialhaskell/stackage/blob/master/build-constraints.yaml#L2689

hosc might be a current blockage.

Latest tidal stackage:

https://www.stackage.org/package/tidal

Looks like tidal got bumped out on ghc-8.4.

[yaxu]

When I attempt to raise a PR to re-enable tidal I'm directed to check that this works first:

stack unpack $package-$version
cd $package-$version
stack init --resolver nightly
stack build --resolver nightly --haddock --test --bench --no-run-benchmarks

With tidal-0.9.10 replacing $package-$version, it ''doesn't'' work.. ~/.cabal/bin/stack init --resolver nightly gives the following:

Looking for .cabal or package.yaml files to use to init the project.
Using cabal packages:
- ./

Selected resolver: nightly-2018-12-15
Downloaded nightly-2018-12-15 build plan.    
Selected resolver: nightly-2018-12-15
Updating package index Hackage (mirrored at https://s3.amazonaws.com/hackage.fpc                                                                                Selected mirror https://s3.amazonaws.com/hackage.fpcomplete.com/
Updating package index Hackage (mirrored at https://s3.amazonaws.com/hackage.fpc                                                                                Downloading root 
Updating package index Hackage (mirrored at https://s3.amazonaws.com/hackage.fpc                                                                                Selected mirror https://s3.amazonaws.com/hackage.fpcomplete.com/
Updating package index Hackage (mirrored at https://s3.amazonaws.com/hackage.fpc                                                                                Downloading timestamp
Updating package index Hackage (mirrored at https://s3.amazonaws.com/hackage.fpc                                                                                Downloading snapshot
Updating package index Hackage (mirrored at https://s3.amazonaws.com/hackage.fpc                                                                                Downloading mirrors
Updating package index Hackage (mirrored at https://s3.amazonaws.com/hackage.fpc                                                                                Cannot update index (no local copy)
Updating package index Hackage (mirrored at https://s3.amazonaws.com/hackage.fpc                                                                                Downloading index
Updating package index Hackage (mirrored at https://s3.amazonaws.com/hackage.fpc                                                                                Updated package index downloaded
Updating package index Hackage (mirrored at https://s3.amazonaws.com/hackage.fpc                                                                                Update complete  
Populated index cache.    
Unable to load cabal files for snapshot

----
Deleting cached snapshot file: /home/alex/.stack/build-plan/nightly-2018-12-15.yaml
Recommendation: try running again. If this fails again, open an upstream issue at:
https://github.com/fpco/stackage-nightly/issues/new
----

Unable to parse cabal file for base-noprelude-4.12.0.0@sha256:dcd5ece10257f79fa16e207d558b31ab3467b205adae5d1976abdee28b1c8b4a,6845: NoParse "reexported-modules" 42

[yaxu]

If I try the same in a checkout of master I get;

Control.Exception.Safe.throwString called with:

Error: Stack configuration file stack.yaml exists, use 'stack solver' to fix the existing config file or '--force' to overwrite it.
Called from:
  throwString (src/Stack/Init.hs:64:9 in stack-1.6.5-FNSY9Wz6MmOG8BH57QdIf5:Stack.Init)

With --force applied I get the same error as above (Unable to parse cabal file for base-noprelude...)

[tonyday567]

Try stack upgrade to make sure you have the latest stack. The first error looks like a transitory stack problem. When I run the commands, I get:

Resolver 'nightly-2018-12-15' does not have all the packages to match your requirements.
    hosc not found
        - tidal requires >=0.16

I can build and test tidal-0.9.10 with the following stack.yaml:

resolver: nightly-2018-12-15

packages:
  - '.'

extra-deps:
  - hosc-0.16
  - microspec-0.2.1.3

https://github.com/commercialhaskell/stackage/issues/3084

[tonyday567]

Once tidal is back in stackage, I think the nix integration in stack is worth playing around with to aim for a one-click install. ref: https://chris-martin.org/2017/nix-for-stack-users

[yaxu]

Yes stack upgrade fixed it (so that it breaks properly now), thanks. @vivid-synth it'd be great to get microspec into hackage

One click installation would be amazing. #282 #283 #284

[hvr]

@tonyday567 I've got to nitpick re "stack is a wrapper around cabal, designed to help with dependencies, and to avoid global ghc an library installation." :-) -- you seem to refer to the legacy cabal mode of operation which installed into a global package database; but this is all a thing of the past now with cabal's Nix-style local builds;

@yaxu regarding cabal compatibility: In my function as Hackage Trustee I've been busy fixing up a lot of Tidal's metadata on Hackage (see e.g. https://github.com/haskell-infra/hackage-trustees/issues/172) ... it would be great for the user experience of Hackage/Cabal users (and also to reduce the workload for Hackage Trustees :) ) if Tidal would be a bit more proactive about cabal compatibility too! I'm happy to advise if you have any questions on how to do this.

[yaxu]

Hi @hvr, ah I had no idea that this work was needed and going on behind the scenes, thanks !

Very happy to be more pro-active, but I'm not clear on what this entails - what have we been doing wrong?

By the way, currently tidal-midi is not compatible with the latest release of tidal, and there is not a plan to update it at the moment, so I think it is considered deprecated. This is because more advanced/stable midi support is available via the superdirt synth.

[tonyday567]

@hvr Thanks for your engagement! It sounds like you still agree with my description of stack, and your point is that stack is unnecessary now that we have nix-style local builds in cabal? In other words, stack competes with cabal v2-* to provide local package builds? If that's not the case, I'd be interested in your opinion on how stack and cabal v2-* functionality differ now and down the track.

Whatever the answer, tidal is a popular haskell project, with a growing user base of largely newcomers to haskell. Meanwhile, the main usage method is via running ghci (within Atom, emacs or command line) assuming that tidal (the library) is installed globally. Any thoughts or ideas on how to give users a smoother install and user experience would be awesome.

[rrodseth]

Voting for Stack-specific instructions if nothing else. I'm a new user who has interest in Haskell independent of Tidal. I was advised by Haskellers to use Stack, but as a result the bootstrap script can't find ghci.

[hvr]

Quoting myself from reddit where I'd strongly recommend newcomers to use ghcup with Cabal:

Follow the simple instructions from https://github.com/haskell/ghcup to setup ghcup (i.e. download ghcup and set $PATH accordingly), and then as the initial setup do

ghcup install-cabal
ghcup install ghc-8.4.4
ghcup set 8.4.4

then you can simply do

cabal update
cabal v2-repl --build-depends tidal

and Cabal will go ahead, figure out the build-plan and compile everything, and you'll be thrown into a GHCi repl with the tidal package in scope. It doesn't get easier than that!

[yaxu]

Now ghcup is suddenly the default way to install haskell for mac users, I'm finding it isn't very easy. In particular, adding to $PATH is difficult if you haven't never heard of that before..

[hvr]

@yaxu well, ghcup follows the mantra to not require admin privileges and avoid trying to perform risky operations (such as modifying your login scripts). Since I wrote the comment, ghcup now has a more beginner oriented intro page at https://www.haskell.org/ghcup/ which follows a similar "pipe some shell script from the internet into your shell" idiom as other language ecosystems tend to offer:

$ curl https://get-ghcup.haskell.org -sSf | sh

Welcome to Haskell!

This will download and install the Glasgow Haskell Compiler (GHC) for 
the Haskell programming language, and the Cabal build tool.

It will add the 'cabal', 'ghc', and 'ghcup' executables to bin directory 
located at: 

  /home/hvr/.ghcup/bin

and create the environment file /home/hvr/.ghcup/env
which you should source in your ~/.bashrc or similar to get the required
PATH components.

To proceed with the ghcup installation press enter, to cancel press ctrl-c.
Note that this script can be re-run at any given time.

and lateron in the installation, you get reminded multiple times (emphasized via terminal colors) about the PATH concern:

...
done
Done installing, run "ghci-8.6.5" or set up your current GHC via: ghcup set 8.6.5
Setting GHC to 8.6.5
Done, make sure "/home/hvr/.ghcup/bin" is in your PATH!
Installing cabal-install-2.4.1.0 into "/home/hvr/.ghcup/bin"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 4025k  100 4025k    0     0  4833k      0 --:--:-- --:--:-- --:--:-- 4827k
Successfully installed cabal-install into
  /home/hvr/.ghcup/bin

You may want to run the following to get the really latest version:
    cabal new-install cabal-install

And make sure that "~/.cabal/bin" comes *before* "/home/hvr/.ghcup/bin"
in your PATH!
Downloading the latest package list from hackage.haskell.org

Installation done!

Don't forget to source /home/hvr/.ghcup/env in your ~/.bashrc or similar.

Do you have a suggestion on how to improve the wording above to make it easier to understand?

[yaxu]

Perhaps we just need a tidal install script that does the above, installs the tidal library too, and also installs and configures atom to call ~/.ghcup/bin/ghci so that the path doesn't need to be updated at all.

[the-frey]

Just chipping in as I've just installed tidal on an ubuntu system (18LTS) and a Mojave install of OSX and things were moderately painful - stack was totally the way to go, so thanks for that pointer in the docs.

Managed to get it working on both systems with atom and the plugin, but emacs requires a bit more gaffa tape I think...

Wondering if it's worth updating the docs with a couple of lines on emacs/tidal/stack interplay?

Will be investigating my emacs setup later based on @tonyday567's comment:

It was very easy to use stack in emacs - just needed to change tidal-interpreter etc. I haven't used atom but imagine it would be a similar story.

Happy to write a couple of lines of docs if/when I work out what's awry. Assume it's that emacs/tidal is using the wrong version of ghci, but hey.

[hvr]

@the-frey It would be very helpful to know what you perceived as painful on Ubuntu and OSX in order to improve the situation with Haskell's standard tooling and avoid needing to resort to commercial 3rd party tooling.

[the-frey]

Possibly my unfamiliarity with stack but still not got my emacs settings worked out.

Sorry, by 3rd party you mean -? AFAICT stack is MIT?

[bradrn]

Not sure how progress is going with this issue, but as a stack user I found that a simple stack build tidal worked well for Tidal installation. I then found the following Emacs configuration to work well:

(setq tidal-interpreter "stack"
      tidal-interpreter-arguments '("ghci" "--package" "tidal")
      tidal-boot-script-path
      (concat (substring
               (shell-command-to-string "stack exec --package tidal -- bash -c \"ghc-pkg describe $(ghc-pkg latest tidal) | grep data-dir | cut -f2 -d' '\"")
               0 -1)
              "/BootTidal.hs"))

This just sets the interpreter to stack ghci --package tidal, then gets the tidal-boot-script-path using stack exec --package tidal -- bash -c "ghc-pkg describe $(ghc-pkg latest tidal) | grep data-dir | cut -f2 d' '". (I need the explicit bash -c since I’m on Windows; a Mac or Linux user should be fine with leaving that out.)

[yaxu]

Closing this for now, but please reopen if there's still an issue here.

[FranklinChen]

@bradrn As of the most recent Stackage LTS 15.11 (if not earlier), the format of the file containing the data-dir changed so that I am using

(setq tidal-boot-script-path
    (concat (substring
                (shell-command-to-string
                    "stack exec ghc-pkg describe $(stack exec ghc-pkg latest tidal) | grep -A 1 data-dir | grep tidal | cut -b 5-") 0 -1)
        "/BootTidal.hs")
    )