Cannot build a standalone executable

[Zert]

It is a bit complicated to build a standalone gitit executable with plugins.

I've built one, then deployed to another host and what I have is:

# ./gitit -f my.conf
Loading plugin 'plugins/Dot'...
gitit: /root/.stack/programs/x86_64-linux/ghc-8.0.1/lib/ghc-8.0.1/settings: openFile: does not exist (No such file or directory)

How I can make a built-in plugins, like dot, for example? I don't need a compiler on the hosting.

[jefdaj]

I had a similar problem and ended up disabling plugin support and hardcoding them. It might not be the "right" way, but it's a fairly simple code change. Look for compiledPlugins in my Plugins.hs and Initialize.hs.

[Zert]

@jefdaj Thank you! I've merged a recent changes from upstream and added MTable.plugin. Here is my fork https://github.com/Zert/gitit/tree/master

[3e4]

Could anyone report what the install size is for the standalone executable is?

I compiled on Debian using stack (recommended in the README), and all in all it became 3GB, which I feel is too much for a Wiki installation.

[jgm]

PS. That was on macos. On a debian-based system the size of my gitit executable is 99MB.

[jgm]

Mine is 197MB, or 147MB stripped. The size could be reduced further by compiling with --split-objs.

Are you sure you're counting just the size of the compiled executable, and not the dependencies installed by stack (including the ghc compiler + all all the libraries gitit depends on)?

Since ghc uses static linking by default, you don't need the dependent libraries to run gitit -- just the executable and the data files.

+++ 3e4 [Jun 01 17 17:24 ]:

Could anyone report what the install size is for the standalone executable is?

I compiled on Debian using stack (recommended in the README), and all in all it became 3GB, which I feel is too much for a Wiki installation.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, [1]view it on GitHub, or [2]mute the thread.

References

1. https://github.com/jgm/gitit/issues/599#issuecomment-305655891
2. https://github.com/notifications/unsubscribe-auth/AAAL5Fr9DCJhyXnRQpVWgqARk0c9_3vaks5r_1YpgaJpZM4L-lGc

[3e4]

The git-cloned gitit directory has 357 MB after compiling. 3.5 MB before. The compiled program seems to be in gitit/.stack-work/install/x86_64-linux/lts-8.0/8.0.2/ (140 MB) with folders bin doc flag-cache lib pkgdb share.

Then I have the stackroot directory that I created at ../stackroot:

stack install --stack-root /var/www/gitit/stackroot

The stackroot directory has 2.9 GB and I first thought that I can delete it. So I renamed it once and run gitit, which worked fine until I tried to export a page as odt, at which point I got the following error:

gitit: Could not find data file /var/www/gititwiki/stackroot/snapshots/x86_64-linux/lts-8.0/8.0.2/share/x86_64-linux-ghc-8.0.2/pandoc-1.19.2.1/data/templates/default.opendocument

Then I tried to compile with

stack install --stack-root /var/www/gititwiki/stackroot/ --ghc-options='-optl-static -optl-pthread'

but got an exit 1 because of

[22 of 22] Compiling Network.Gitit    ( src/Network/Gitit.hs, .stack-work/dist/x86_64-linux/Cabal-1.24.2.0/build/Network/Gitit.o )
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/4.9/crtbeginT.o: relocation R_X86_64_32 against `__TMC_END__' can not be used when making a shared object; recompile with -fPIC
/usr/lib/gcc/x86_64-linux-gnu/4.9/crtbeginT.o: error adding symbols: Bad value
collect2: error: ld returned 1 exit status
`gcc' failed in phase `Linker'. (Exit code: 1)`

[jgm]

Egads. See commercialhaskell/stack#1005, which is this precise issue: the data files that the executable depends on are not installed properly. This issue (from 2015) points to another, still open issue, commercialhaskell/stack#848. I'm surprised that stack has this problem; data files have always worked well with cabal.

Perhaps you should start over, using cabal instead of stack to build the project.

cabal update
cabal install gitit

This procedure should put the executable and the required data files in the same subdirectory (~/.cabal or perhaps ~/Library/Haskell, I can't remember the default, but this can be configured with a --prefix option I believe).

[jgm]

Please report back @3e4 if the cabal method works for you. We may want to modify the advice in the README.

[jgm]

Perhaps a more robust fix would be to provide a flag to bake the data files into the binary, as we do with pandoc.

[3e4]

Just trying on Debian 8:

sudo apt install haskell-platform
cabal install gitit

And i get error because of pandoc dependency issues:

1.2.1, 1.2, 1.1, 1.0.0.1, 1.0, 0.46, 0.45, 0.44, 0.43, 0.42, 0.41, 0.4
(conflict: gitit => pandoc>=1.12.4 && <1.20)
rejecting: pandoc-1.15.0.1, 1.15, 1.14.0.4, 1.14.0.3, 1.14.0.2, 1.14.0.1, 1.14
(conflict: pandoc-types==1.17.0.5, pandoc => pandoc-types>=1.12.4 && <1.13)
Backjump limit reached (change with --max-backjumps).

So i find this and try

cabal install gitit --max-backjumps=-1

But there is no progress after more than 1 hour. Just showing Resolving dependencies....

[jgm]

Did you do cabal update? It ought to find a more recent version of pandoc than it seems it is finding.

+++ 3e4 [Jun 02 17 23:31 ]:

Just trying on Debian 8: sudo apt install haskell-platform cabal install gitit

And i get error because of pandoc dependency issues: 1.2.1, 1.2, 1.1, 1.0.0.1, 1.0, 0.46, 0.45, 0.44, 0.43, 0.42, 0.41, 0.4 (conflict: gitit => pandoc>=1.12.4 && <1.20) rejecting: pandoc-1.15.0.1, 1.15, 1.14.0.4, 1.14.0.3, 1.14.0.2, 1.14.0.1, 1.14 (conflict: pandoc-types==1.17.0.5, pandoc => pandoc-types>=1.12.4 && <1.13) Backjump limit reached (change with --max-backjumps).

So i find [1]this and try cabal install gitit --max-backjumps=-1

But there is no progress after more than 1 hour. Just showing Resolving dependencies....

— You are receiving this because you commented. Reply to this email directly, [2]view it on GitHub, or [3]mute the thread.

References

1. https://stackoverflow.com/questions/29939644/backjump-limit-reached-change-with-max-backjumps
2. https://github.com/jgm/gitit/issues/599#issuecomment-305955470
3. https://github.com/notifications/unsubscribe-auth/AAAL5Bkv88iwOrPOK8TOln1DCyu_3z0Pks5sAP3ZgaJpZM4L-lGc

[3e4]

Yes I did. The full error trace looks like this:

trying: pandoc-types-1.17.0.5 (dependency of gitit-0.12.2.1)
next goal: pandoc (dependency of gitit-0.12.2.1)
rejecting: pandoc-1.19.2.1, 1.19.2, 1.19.1, 1.19, 1.18 (conflict:
base==4.6.0.1/installed-8aa..., pandoc => base>=4.7 && <5)
rejecting: pandoc-1.17.2, 1.17.1, 1.17.0.3, 1.17.0.2, 1.17.0.1, 1.17,
1.16.0.2, 1.16.0.1, 1.16 (conflict: pandoc-types==1.17.0.5, pandoc =>
pandoc-types>=1.16 && <1.17)
rejecting: pandoc-1.15.2.1, 1.15.2, 1.15.1.1, 1.15.1, 1.15.0.6, 1.15.0.5,
1.15.0.4, 1.15.0.3, 1.15.0.2, 1.14.1, 1.13.2.1, 1.13.2 (conflict:
pandoc-types==1.17.0.5, pandoc => pandoc-types>=1.12.4 && <1.13)
rejecting: pandoc-1.13.1, 1.13.0.1, 1.13, 1.12.4.2, 1.12.4 (conflict:
text==1.2.2.2, pandoc => text>=0.11 && <1.2)
rejecting: pandoc-1.12.3.3, 1.12.3.2, 1.12.3.1, 1.12.3, 1.12.2.1, 1.12.2,
1.12.1, 1.12.0.2, 1.12.0.1, 1.12, 1.11.1, 1.11, 1.10.1, 1.10.0.5, 1.10.0.4,
1.10.0.3, 1.10.0.2, 1.10.0.1, 1.10, 1.9.4.5, 1.9.4.4, 1.9.4.3, 1.9.4.2,
1.9.4.1, 1.9.4, 1.9.3, 1.9.2, 1.9.1.2, 1.9.1.1, 1.9.1, 1.9.0.5, 1.9.0.4,
1.9.0.3, 1.9.0.2, 1.9, 1.8.2.1, 1.8.2, 1.8.1.2, 1.8.1.1, 1.8.1, 1.8.0.3,
1.8.0.2, 1.8.0.1, 1.8, 1.6.0.1, 1.6, 1.5.1.1, 1.5.1, 1.5.0.1, 1.5, 1.4, 1.3,
1.2.1, 1.2, 1.1, 1.0.0.1, 1.0, 0.46, 0.45, 0.44, 0.43, 0.42, 0.41, 0.4
(conflict: gitit => pandoc>=1.12.4 && <1.20)
rejecting: pandoc-1.15.0.1, 1.15, 1.14.0.4, 1.14.0.3, 1.14.0.2, 1.14.0.1, 1.14
(conflict: pandoc-types==1.17.0.5, pandoc => pandoc-types>=1.12.4 && <1.13)
Backjump limit reached (change with --max-backjumps).

I am running

$ cabal -V
cabal-install version 1.20.0.3
using version 1.20.0.2 of the Cabal library
$ ghc -v
Glasgow Haskell Compiler, Version 7.6.3, stage 2 booted by GHC version 7.6.3

In my .cabal/packages/hackage.haskell.org I have 00-index.cache with (full cache):

pref-ver: pandoc <1.14 || >1.14 && <1.14.0.1 || >1.14.0.1 && <1.14.0.2 || >1.14.0.2 && <1.14.0.3 || 
>1.14.0.3 && <1.14.0.4 || >1.14.0.4 && <1.15 || >1.15 && <1.15.0.1 || >1.15.0.1

[jgm]

Ah. Yes, recent versions of pandoc require ghc 7.8 or greater. You have a very old ghc.

+++ 3e4 [Jun 03 17 16:41 ]:

Yes I did. The full error trace looks like this: trying: pandoc-types-1.17.0.5 (dependency of gitit-0.12.2.1) next goal: pandoc (dependency of gitit-0.12.2.1) rejecting: pandoc-1.19.2.1, 1.19.2, 1.19.1, 1.19, 1.18 (conflict: base==4.6.0.1/installed-8aa..., pandoc => base>=4.7 && <5) rejecting: pandoc-1.17.2, 1.17.1, 1.17.0.3, 1.17.0.2, 1.17.0.1, 1.17, 1.16.0.2, 1.16.0.1, 1.16 (conflict: pandoc-types==1.17.0.5, pandoc => pandoc-types>=1.16 && <1.17) rejecting: pandoc-1.15.2.1, 1.15.2, 1.15.1.1, 1.15.1, 1.15.0.6, 1.15.0.5, 1.15.0.4, 1.15.0.3, 1.15.0.2, 1.14.1, 1.13.2.1, 1.13.2 (conflict: pandoc-types==1.17.0.5, pandoc => pandoc-types>=1.12.4 && <1.13) rejecting: pandoc-1.13.1, 1.13.0.1, 1.13, 1.12.4.2, 1.12.4 (conflict: text==1.2.2.2, pandoc => text>=0.11 && <1.2) rejecting: pandoc-1.12.3.3, 1.12.3.2, 1.12.3.1, 1.12.3, 1.12.2.1, 1.12.2, 1.12.1, 1.12.0.2, 1.12.0.1, 1.12, 1.11.1, 1.11, 1.10.1, 1.10.0.5, 1.10.0.4, 1.10.0.3, 1.10.0.2, 1.10.0.1, 1.10, 1.9.4.5, 1.9.4.4, 1.9.4.3, 1.9.4.2, 1.9.4.1, 1.9.4, 1.9.3, 1.9.2, 1.9.1.2, 1.9.1.1, 1.9.1, 1.9.0.5, 1.9.0.4, 1.9.0.3, 1.9.0.2, 1.9, 1.8.2.1, 1.8.2, 1.8.1.2, 1.8.1.1, 1.8.1, 1.8.0.3, 1.8.0.2, 1.8.0.1, 1.8, 1.6.0.1, 1.6, 1.5.1.1, 1.5.1, 1.5.0.1, 1.5, 1.4, 1.3, 1.2.1, 1.2, 1.1, 1.0.0.1, 1.0, 0.46, 0.45, 0.44, 0.43, 0.42, 0.41, 0.4 (conflict: gitit => pandoc>=1.12.4 && <1.20) rejecting: pandoc-1.15.0.1, 1.15, 1.14.0.4, 1.14.0.3, 1.14.0.2, 1.14.0.1, 1.14 (conflict: pandoc-types==1.17.0.5, pandoc => pandoc-types>=1.12.4 && <1.13) Backjump limit reached (change with --max-backjumps).

I am running $ cabal -V cabal-install version 1.20.0.3 using version 1.20.0.2 of the Cabal library $ ghc -v Glasgow Haskell Compiler, Version 7.6.3, stage 2 booted by GHC version 7.6.3

In my .cabal/packages/hackage.haskell.org I have 00-index.cache with ([1]full cache): pref-ver: pandoc <1.14 || >1.14 && <1.14.0.1 || >1.14.0.1 && <1.14.0.2 || >1.14. 0.2 && <1.14.0.3 ||

1.14.0.3 && <1.14.0.4 || >1.14.0.4 && <1.15 || >1.15 && <1.15.0.1 || >1.15.0.1

— You are receiving this because you commented. Reply to this email directly, [2]view it on GitHub, or [3]mute the thread.

References

1. https://gist.github.com/3e4/7afdb68e58e64a5d3c16ed31871993a0
2. https://github.com/jgm/gitit/issues/599#issuecomment-306008021
3. https://github.com/notifications/unsubscribe-auth/AAAL5GQycQk4fCJXWaoTBg_07Mg0bisQks5sAe8qgaJpZM4L-lGc

[3e4]

Finally?

I installed haskell platform from their repository with ghc 7.10.3 and the dependency resolution worked within a few seconds.

Now I got the following binaries in my ~/.cabal/bin directory:

 2.8M expireGititCache
124M gitit
 13M json2yaml
 64M pandoc
 13M yaml2json

So that'll be around 200 MB. Its my first time I got in contact with Haskell. Didn't expect it to be that messy... Thank you for the help and maybe this will be helpful to others.

[jgm]

Glad it worked. Note that because everything is statically linked, you don't need json2yaml, yaml2json, or pandoc to use gitit. You just need the one executable and the data files it depends on.

[3e4]

Not sure if related because my build process used some bad libraries, but I get alignment issues with line numbers:

Here is some page with haskell code using line numbers, with some clear misalignment. (gitit version 0.12.2.1 -plugins)

The same source file but compiled with pandoc from the command line pandoc markd -s --highlight-style pygments -o markd.html (pandoc 1.19.2.1 Compiled with pandoc-types 1.17.0.5, texmath 0.9.4, skylighting 0.1.1.5)

Misalignment visible in Firefox and Chromium.

Here is the source file:

---
format: Markdown
...

# Welcome to Gitit!

This is the front page of your new gitit wiki.  You can edit this
page by clicking on the "edit" tab at the top of the screen.
For instructions on how to make a link to another wiki page, see [the
Help page](Help#wiki-links). To create a new wiki page, just create a
link to it and follow the link.

Help is always available through the "Help" link in the sidebar.
More details on installing and configurating gitit are available
in the [Gitit User’s Guide]().

Some code here[Gooooogle](https://google.com)

~~~ {.haskell .numberLines}
-- | Inefficient quicksort in haskell.
qsort :: (Enum a) => [a] -> [a]
qsort []     = []
qsort (x:xs) = qsort (filter (< x) xs) ++ [x] ++
               qsort (filter (>= x) xs) 

Lets try creating a new page: [Alternative top page]()

[jgm]

It could be due to older versions of pandoc (or of the highlighting library) that were used in compiling gitit. Or it could just be an issue with some CSS gitit is using, in which case it could easily be fixed by adding something to your local static/ directory.

+++ 3e4 [Jun 05 17 02:06 ]:

Not sure if related because my build process used some bad libraries, but I get alignment issues with line numbers:

[jeffmcneill]

So, should folks try using cabal to install and not stack? Sounds like it worked above...

[3e4]

Sorry I should have given another update. At least as of writing my post on June 2nd, Stack from debian repository was not working.

I the meantime I setup cabal from their bootstrap.sh script, then used that cabal to install/compile stack (from stack.sh), and then used that stack to compile gitit.

stack --version
Version 1.4.0, Git revision e714f1dd3fade19496d91bd6a017e435a96a6bcd (4640 commits) x86_64 hpack-0.17.0

cabal --version
cabal-install version 2.1.0.0
compiled using version 2.1.0.0 of the Cabal library 

ghc --version
The Glorious Glasgow Haskell Compilation System, version 8.0.2

It was messy with several failures and dependency hells (had to purge everything twice and repeat), and I wouldn't be able to produce detailed instructions how to repeat it. But I got myself a single binary (~120 MB) that is working.