Open seagreen opened 5 years ago
Is there a way to do this, perhaps what haskell-code-explorer.mfix.io does?
In order to index packages for haskell-code-explorer.mfix.io, I wrote a stack script (https://docs.haskellstack.org/en/stable/GUIDE/#script-interpreter) that downloads, builds and indexes a subset of packages from a Stackage snapshot. I plan to release that script (I need to clean it up a bit first).
I'd like to index not just the package I'm working on, but also the rest of the packages in the stack.yaml's snapshot, as well as the packages in extra-deps.
I think the easiest way is to write a simple shell script.
For example, this command downloads the source code of all dependencies of a stack project to PATH:
stack ls dependencies | tr " " - | xargs -i stack unpack {} --to PATH
This command tries to build (and index) each package in PATH with stack using RESOLVER (e.g., lts-13.13):
for p in PATH/*; do (cd "$p" && stack init --resolver=RESOLVER --force && stack build && PATH_TO_HASKELL_CODE_INDEXER -p .); done
(It should work for most packages except for ghc, base, ghc-prim, etc.)
Sweet, that worked. I've got an example up here: https://github.com/seagreen/hermetic/tree/master/explore
Some thoughts:
Like you warned it doesn't work for base
. How'd you get base
indexed for haskell-code-explorer.mfix.io?
It copies in this premake stack.yaml
instead of using stack init
:
resolver: lts-13.0
allow-newer: true
The project being indexed uses lts-13.0
, so I know that's right without having to run stack init
. And without the allow-newer
I had a lot of packages fail to build due to a restrictive upper bound on QuickCheck
. I'm guessing LTS 13 has a setting in there to override them, which is why they work together in that LTS but don't work when trying to build them individually. I didn't look into it deeply though because since I already know those packages work together I can set allow-newer
.
The application consists of two local packages, hermetic
and json-relay
. I placed all their dependencies in a deps
directory, but decided to index those two in place, because if I copied them to deps
it would be easy to let them get out of sync as the project develops. In order to use haskell-code-server
with both deps
and those two local packages I used this PR: https://github.com/alexwl/haskell-code-explorer/pull/31
You can see how this worked in action here: https://github.com/seagreen/hermetic/blob/669504a346b2d99f90100527bca10e681ac9e695/explore/serve
I used cabal-install
(cabal
executable) and globally installed GHC to build base
. Building base
requires certain system libraries and tools that are checked during the configuration step (e.g., cabal configure -f integer-gmp
). Some libraries may need to be installed manually.
I'm guessing LTS 13 has a setting in there to override them, which is why they work together in that LTS but don't work when trying to build them individually.
It seems to be the case. For example, LTS 13.0 contains QuickCheck-2.12.6.1
and text-1.2.3.1
, and at the same time build component tests
of text-1.2.3.1
package depends on QuickCheck >= 2.7 && < 2.11
(the upper bound is too restrictive).
Now, I see that copying stack.yaml
with allow-newer:true
is a better approach than using stack init
command.
I like the new feature (PR #31).
Glad you like it. Thank you for fixing the PR!
I used cabal-install (cabal executable) and globally installed GHC to build base. Building base requires certain system libraries and tools that are checked during the configuration step (e.g., cabal configure -f integer-gmp). Some libraries may need to be installed manually.
Ah, interesting. I'll take a stab at this sometime and update my example.
@alexwl
For example, this command downloads the source code of all dependencies of a stack project to PATH:
stack ls dependencies | tr " " - | xargs -i stack unpack {} --to PATH This command tries to build (and index) each package in PATH with stack using RESOLVER (e.g., lts-13.13):
for p in PATH/*; do (cd "$p" && stack init --resolver=RESOLVER --force && stack build && PATH_TO_HASKELL_CODE_INDEXER -p .); done (It should work for most packages except for ghc, base, ghc-prim, etc.)
I do not understand this approach. When a project is built, all dependencies are already built. Why do you want to explicitly download all sources and build them again?
@rikvdkleij
The indexer uses GHC API (http://hackage.haskell.org/package/ghc-8.6.1) to collect information from Haskell AST (Abstract Syntax Tree). The only way to get an AST of a Haskell module is to initialize a GHC API session (https://gitlab.haskell.org/ghc/ghc/wikis/commentary/compiler/api) and to load a source file using setTargets
or addTargets
function (http://hackage.haskell.org/package/ghc-8.6.1/docs/GHC.html#v:setTargets).
Each already-built dependency is a collection of interface (.hi) files (https://gitlab.haskell.org/ghc/ghc/wikis/commentary/compiler/iface-files) and object files. It is not impossible to get an AST from an interface file (there is a project https://gitlab.haskell.org/ghc/ghc/wikis/hie-files that should fix it in new versions of GHC).
I'd like to index not just the package I'm working on, but also the rest of the packages in the
stack.yaml
's snapshot, as well as the packages inextra-deps
.Is there a way to do this, perhaps what haskell-code-explorer.mfix.io does?