search

ndmitchell / hoogle

Haskell API search engine
http://hoogle.haskell.org/
Other
738 stars 134 forks source link

show progress with `data all` #109

Open erikkaplun opened 9 years ago

erikkaplun commented 9 years ago

Currently, I have no idea how much more my hoogle data all command is going to take; it's already been running for ~3 hours and I don't know if I should wait another 30 min or just give up.

Also, would it be possible to make the whole data all process faster?

ndmitchell commented 9 years ago

In Hoogle 5 it takes ~50s, so once that is out, expect it to be massively faster. It also gives progress. I'm working towards Hoogle 5, but there are still some features not yet ready, see http://neilmitchell.blogspot.co.uk/2015/01/hoogle-5-is-coming.html for some details.

erikkaplun commented 9 years ago

That looks like great news! Meanwhile, any idea how much longer do I have to wait? It's been running for 3:53 on an i7 with 16G of RAM (it's at 3.67GB right now and the usage seems to have reached a fix point).

ndmitchell commented 9 years ago

I suspect the GHC heap limited itself to 3.67Gb, and it's just furiously garbage collecting. I have no idea how long it might take. Increasing the memory with +RTS -M8G might make it go faster, or not.

erikkaplun commented 9 years ago

OK, I'll just leave it running overnight, or if that won't suffice, just wait for 5.0! Thanks for the assistance!

ndmitchell commented 9 years ago

In Hoogle 5 I've got this down to 45s when using 2 threads, and about 60s on 1 thread.