Build issue

[PatrickMassot]

I tried to follow the blog instructions to test building the stacks web site. I get

INFO:__main__:Assigning chapters to parts
Traceback (most recent call last):
  File "/home/pmassot/.virtualenvs/gerby/lib/python3.6/site-packages/peewee.py", line 5463, in get
    return clone.execute()[0]
  File "/home/pmassot/.virtualenvs/gerby/lib/python3.6/site-packages/peewee.py", line 3355, in __getitem__
    return self.row_cache[item]
IndexError: list index out of range

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "update.py", line 170, in <module>
    Part.create(part=Tag.get(Tag.type == "part", Tag.ref == part), chapter=Tag.get(Tag.type == "chapter", Tag.ref == chapter))
  File "/home/pmassot/.virtualenvs/gerby/lib/python3.6/site-packages/peewee.py", line 5082, in get
    return sq.get()
  File "/home/pmassot/.virtualenvs/gerby/lib/python3.6/site-packages/peewee.py", line 5468, in get
    (clone.model, sql, params))
gerby.database.TagDoesNotExist: <class 'gerby.database.Tag'> instance matching query does not exist:
SQL: SELECT "t1"."tag", "t1"."label", "t1"."active", "t1"."ref", "t1"."type", "t1"."html", "t1"."name" FROM "tag" AS "t1" WHERE (("t1"."type" = ?) AND ("t1"."ref" = ?)) LIMIT 1 OFFSET 0
Params: ['part', '9']

I can try to investigate but first I'd like to know whether you see something obvious here (like you could know that the blog instructions is not up to date).

By the way, I don't see any kind of continuous integration on this repository. Don't you think you could configure Travis to try to build the stacks project each time someone pushes to this repo? I could probably help a bit in setting this up.

[chngr]

At some point, we decided that documents fed into Gerby should be split into parts. However, this is not yet handled either in plasTeX or well in update.py: there is nothing that really creates rows corresponding to parts in the tags database. So right now, when you run update.py, it will hit line 170, try to find parts in the tags database, and then fail.

This is definitely something to be automated, but one temporary thing you can do to get everything working is the following: manually insert rows for the 9 parts into the database. You might also need to manually insert rows for sections 108 and 109 for things to work, too.

Thanks for raising this as a formal complaint. I think @pbelmans and @aisejohan have some personal scripts that already automate away this issue, but until they enlighten us about such things, this is a fix.

[aisejohan]

The current version (updated yesterday) of the Stacks project has a target web in the Makefile which creates book.tex, tags, and my.bib files that can be parsed by plastex/gerby and the update script in gerby-website. At least it works for me locally on my machine at home. Your results may vary.

[PatrickMassot]

Thank you to both of you. @chngr you wrote "Thanks for raising this as a formal complaint." but I'm not complaining about anything. I simply try to build the thing to see if I can play with it. Mostly I would like to convince you to go for server side mathjax rendering. But obviously I can't set up a demo if I can't build the current version.

@aisejohan I'll try to use this make target.

[chngr]

@PatrickMassot, sounds great! Looking forward to your demo (: And it appears that, somehow, I have been influenced to use the word "complain" more liberally than most others... ahem.

[PatrickMassot]

Now update.py seem to hang forever after printing INFO:__main__: Created tag 00MY (it created thousands of tags before stopping)

[pbelmans]

Sorry, the instructions are a little out of date, given the recent developments. We'll try to fix that soon, once I get back from holidays.

Using Travis sounds interesting. I have no experience with it, but I'd love to learn more. We can use it both for pdflatex and plastex testing I presume? We also need about 3 gigabyte of memory at the moment.

Regarding parts: right now they are not yet in the tags file, as the current website doesn't deal with them. Gerby strictly speaking doesn't need parts to work, if the top level contains chapters it'll just revert to using those, but the Stacks project is meant to be used with parts. So that's another cause of confusion at the moment.

I don't think you need to deal with sections 108 and 109 manually though. At least I never did.

Regarding serverside MathJax rendering: I did experiment with it a little bit, but first we needed to get everything else up and running. It's no good to have serverside MathJax if the other features aren't there :). Now that the majority of things has been implemented we should certainly think about serverside MathJax. My only concern is that it will significantly increase the runtime for update.py, but maybe my initial experiments with mathjax-node weren't performed in an optimal way.

Regarding the last message: I have no idea why that happened. Sometimes the update script doesn't output anything for a while (e.g. when building statistics, at the very end).

[aisejohan]

OK, it is impossible to keep straight what you have to do to get it to work (eg what I said above wasn't OK because I forgot about something). I have a bash script that keeps changing, but it sort of works. Don't just run this script but read it and do the corresponding commands for your setup.

#! /bin/bash

mkdir GERBY
cd GERBY

# Install plastex/gerby
git clone https://github.com/pbelmans/plastex.git
cd plastex
git checkout gerby
# sudo apt install python3-pip
pip3 install --editable .
cd ..

# Install dvipng
#sudo apt-get install dvipng

# Create the book.tex file and files needed for plastex/gerby
mkdir WEB
git clone https://github.com/stacks/stacks-project.git
cd stacks-project
echo "Say yes below to continue."
python scripts/add_tags.py
make web
cd ../WEB
echo "Make sure to that extra tags for parts have been added. Done (y/n)?"
read ANSWER
if [ ! $ANSWER == "y" ]; then exit; fi
echo "Give full path to plastex executable:"
read PLASTEX
$PLASTEX --renderer=Gerby book.tex
cd ..

# Install needed python3 packages
#pip3 install flask
#pip3 install peewee
git clone https://github.com/live-clones/pybtex.git
wget https://bitbucket.org/pybtex-devs/pybtex/issues/attachments/110/pybtex-devs/pybtex/1514284299.07/110/no-protected-in-math-mode.patch
cd pybtex
git apply ../no-protected-in-math-mode.patch
pip3 install --editable .
cd ..
#pip3 install flask_profiler
#pip3 install feedparser
#pip3 install markdown
#pip3 install mdx_bleach
#pip3 install python-markdown-math
#pip3 install validators

# Install gerby-website
git clone https://github.com/pbelmans/gerby-website.git
cd gerby-website
pip3 install --editable .

# Install XyJax
cd gerby/static
git clone https://github.com/sonoisa/XyJax.git
sed -i -e 's@\[MathJax\]@/static/XyJax@' XyJax/extensions/TeX/xypic.js

# Install jquery-bonsai --- this works but I don't know why
rm -rf jquery-bonsai
git clone https://github.com/aexmachina/jquery-bonsai
cp jquery-bonsai/jquery.bonsai.css css/

# Make links to things created by plastex run
cd ../tools
ln -s ../../../WEB/book
mv book stacks
ln -s ../../../WEB/book.paux
mv book.paux stacks.paux
ln -s ../../../WEB/tags
mv tags stacks.tags

# Run update.py
echo "Remove the last section from update.py starting with "
echo "PATH_TO_BOOK_PDF = ..."
echo "and replace it with "
echo "BookStatistic.create(statistic="pages", value='6000')"
echo "Done? (y/n)"
read ANSWER
if [ ! $ANSWER == "y" ]; then exit; fi
python3 update.py

# Start flask server
echo "Give full path to flask executable: "
read FLASK
FLASK_APP=gerby $FLASK run

[PatrickMassot]

Thanks. Could you tell me what you do after your script asks "Make sure to that extra tags for parts have been added. Done (y/n)?"

Yesterday I went to bed while update.py hanged and something actually happened during the night. I think I the silent part was post-processing tags. And then it crashed on missing parts, but this is clearly expected.

[aisejohan]

I visually inspect the tags file to see if at the very end it has added tags for parts. Something like

0EGT,book-part-topics-in-geometry
0EGU,book-part-schemes
0EGV,book-part-preliminaries
0EGW,book-part-topics-in-moduli-theory
0EGX,book-part-algebraic-stacks
0EGY,book-part-algebraic-spaces
0EGZ,book-part-miscellany
0EH0,book-part-topics-in-scheme-theory
0EH1,book-part-deformation-theory

should be in the tags file plastex is going to use....

[chngr]

@PatrickMassot and @aisejohan: I have begun to add some flexibility to the update script. In particular, you can now specify a flag --noParts to bypass the parts assignment. Of course, it will be prettier and better if you do assign some parts by updating the tags file as above, but it is not strictly necessary.

As a side note, you can make the database creation process quicker by not creating a search table and skipping the computation of statistics. This can be done by passing the flags --noSearch and --noTagStats, respectively.

[pbelmans]

I think we can close this now: #82 gives precise and up-to-date instructions on how to install things, #71 will lead to a more flexible updating script.