Small local generation?

[JonnyRa]

Hi!

I'd recently had a go at using stack hoogle and was delighted to see that it works, however the build times with --rebuild are really terrible and make it almost unusable, so I started digging a little bit to see what stack actually does when you call hoogle... which led me to https://github.com/commercialhaskell/stack/issues/55#issuecomment-155186311.

Even though that issue is pretty old, looking through the current source it seems about right. The bit that is really slow on the project I'm working on is building all the haddocks which I'm not really interested in anyway - I just want to use the type search, our code doesn't have api docs anyway really.

If I just run stack exec -- hoogle generate --local --database=.stack-work/hoogle then I get more tolerable build times of about 10 seconds, however it doesn't seem to actually be able to search any of our local types, it also seems to still be generating lots of stuff for dependencies.

So a few (possibly stupid!) questions:

• Is building the haddock stuff necessary for hoogle to work?
• Is it possible to not bother making stuff for dependencies and just generate the local stuff? I can just use the online hoogle or stackage's hoogle for non local stuff.
• It looks like you can pass arguments to local from the source but the formats in the directory pointed at in tests isn't familiar to me, what do you point it at + how do you pass and seperate multiple arguments? I'd be happy to update the docs for this and do a pull request

The tool is great and pretty much does everything you'd want it to, I just need to understand it a bit better to get the build times down a bit for our code.

Any help would be much appreciated :)

Cheers

Jonny

[ndmitchell]

I'm afraid the docs for executing Hoogle locally aren't great. In truth, I never do it, so I don't even know what the flags do! Any help with docs would be most welcome.

What are the build times with --rebuild? Have you done +RTS -N2, which might double the speed. Are you invoking it via Stack or directly, so you can measure each part in isolation?

The only way Hoogle indexes the types is by running Haddock to extract the documentation and types - you have to run Haddock I'm afraid or it's not going to work.

You can configure Hoogle to work only with local packages (not sure how, but I think you can) - but does that help if Haddock is the bottleneck?

It looks like you can pass arguments to local from the source

Don't really understand the question - can you give an example?

[JonnyRa]

Ok, that's helpful. I'll have a go at calling stuff directly. I suppose really I should look at trying to build less haddocks in that case.

Sorry that was a bit badly phrased, it should have probably said

It looks like you can pass arguments to the local flag based on my reading of the source.

eg instead of just using it like --local. You can see it branching on that in Action/Generate.hs around line 201.

The other thing I was on about there was the Action/Test.hs file, which sets local to "misc/sample-data". Is the stuff in there haddock output?

    actionGenerate defaultGenerate{database=sample, local_=["misc/sample-data"]}

The type (CmdLine/Generate) has local_ down as a list of FilePath which is what I was curious about as I can't see any documentation/examples for how to pass multiple things (I'm not even sure if that's what I want/need to do!), don't stress about it though if you're not that familiar, I can do some more digging. I'll post my progress on here