search

rougier / scientific-visualization-book

An open access book on scientific visualization using python and matplotlib
https://www.labri.fr/perso/nrougier/
Other
10.66k stars 986 forks source link

Create recent book version from scratch #51

Closed labdmitriy closed 2 years ago

labdmitriy commented 2 years ago

Hi @rougier,

Could you please tell is it possible to create recent version of the book using the repository by myself?

I started to install required system/python packages and have an errors when running make all, for example:

Latexmk: 'xelatex': source file 'ean13.tex' doesn't exist. I'll try making it...

There are a lot of fixes in repository which are not in pdf in the book folder, and it will be great to include these fixes in pdf version.

Thank you.

rougier commented 2 years ago

Yes, you can compile the book from the repository but you'll need to install some supplementary packages. I'm using TeXLive that install pretty much everything. The ean13 error is package for generating barcodes.

labdmitriy commented 2 years ago

Great, thank you!

labdmitriy commented 2 years ago

Hi @rougier,

Sorry but I can't build the book even after the following steps (I am using Ubuntu 20.04):

However I have the following error in the terminal:

mkdir -p pdf
Building tex/main.tex
Building tex/00-dedication.tex
Building tex/00-preface.tex
Building tex/00-introduction.tex
Building tex/00-acknowledgments.tex
Rc files read:
  NONE
Latexmk: Run number 1 of rule 'xelatex'
This is XeTeX, Version 3.141592653-2.6-0.999994 (TeX Live 2022) (preloaded format=xelatex)
 \write18 enabled.
entering extended mode
Latexmk: Getting log file 'book.log'
Latexmk: 'xelatex': source file 'book.bbl' doesn't exist. I'll try making it...
make[1]: Entering directory '/home/dlabazkin/education/scientific-visualization-book/tex'
make[1]: Leaving directory '/home/dlabazkin/education/scientific-visualization-book/tex'
Latexmk: Summary of warnings from last run of *latex:
  Latex failed to resolve 122 reference(s)
Collected error summary (may duplicate other messages):
  xelatex: Command for 'xelatex' gave return code 1
      Refer to 'book.log' for details

And tex/book.log file has the following lines with Error word:

cat tex/book.log | grep Error

! LaTeX Error: \begin{mdframed} on input line 367 ended by \end{DUadmonition}.
! LaTeX Error: \begin{mdframed} on input line 367 ended by \end{DUclass}.
! LaTeX Error: \begin{mdframed} on input line 1150 ended by \end{DUadmonition}.
! LaTeX Error: \begin{mdframed} on input line 1150 ended by \end{DUclass}.
! LaTeX Error: \begin{mdframed} on input line 1741 ended by \end{DUadmonition}.
! LaTeX Error: \begin{mdframed} on input line 1741 ended by \end{DUclass}.
! LaTeX Error: \begin{mdframed} on input line 1741 ended by \end{document}.

Also I attached full text of book.log - book.log

Could you please help with this problem?

If it is possible to solve and build successfully then I can create PR and prepare instructions how to create it for Ubuntu if you want.

Thank you.

rougier commented 2 years ago

Sorry, I'm very late in answer. Did you fix the problem?

labdmitriy commented 2 years ago

Hi, no problem :) Unfortunately no because I didn’t have any idea how to resolve it. Could you please check which additional step do I need? Thank you.

rougier commented 2 years ago

Yes, I'll look into it.

rougier commented 2 years ago

Which version of docututils do you have by the way ? I think I remember we had the problem on the private repo and I had to stick to a precise version. I'll search for the version number.

labdmitriy commented 2 years ago

I can check the version and any other ideas tomorrow and will definitely answer to you.

rougier commented 2 years ago

It seems docutils 0.18 fails while docutils 0.17 is ok. Tell me if it is the problem.

rougier commented 2 years ago

With GitHub action, we have:

     run: sudo apt-get install --yes python && sudo pip install docutils==0.17 # docutils 0.18 breaks the build
labdmitriy commented 2 years ago

Ok, I will check tomorrow and tell you about result, thanks a lot!

labdmitriy commented 2 years ago

Hi Nicolas,

I reproduced all the steps from scratch and using version 0.17 of docutils and few other steps I can generate the book successfully, thank you!

Below I am describing all the steps for Ubuntu 20.04 which I did to make a successful build.

  1. Install TeX Live This guide was helpful to install it - https://www.tug.org/texlive/quickinstall.html Here I used the latest version of TeX Live (20220621)
cd ~/Downloads/
wget https://mirror.ctan.org/systems/texlive/tlnet/install-tl-unx.tar.gz
tar -xzvf install-tl-unx.tar.gz
cd install-tl-20220621/
sudo perl install-tl --no-interaction

# Add to ~/.bashrc
export PATH="$PATH:/usr/local/texlive/2022/bin/x86_64-linux"

source ~/.bashrc

# Check installation
which latex
  1. Install Proj 9.0.0 for cartopy I used several steps from this answer on Stack Overflow - https://stackoverflow.com/a/72442088
sudo apt-get update
sudo apt-get install libgeos-dev cmake sqlite3 curl libcurl4-openssl-dev libtiff5-dev

cd ~/Downloads
wget https://download.osgeo.org/proj/proj-9.0.0.tar.gz
tar -xzvf proj-9.0.0.tar.gz
cd proj-9.0.0/
mkdir build && cd build
cmake ..
cmake --build .
sudo cmake --build . --target install
  1. Configure project & build the book For cartopy I used the latest version from GitHub because for some reason I had the errors with metadata for cartopy package from PyPI repository. 
# Clone the repository
mkdir -p ~/education/sci-viz
cd ~/education/sci-viz
git clone https://github.com/rougier/scientific-visualization-book
cd scientific-visualization-book

# Create virtual environment
python -m venv ~/venvs/scientific-visualization-book
source ~/venvs/scientific-visualization-book/bin/activate
python -m pip install --upgrade pip setuptools wheel 
pip install numpy scipy matplotlib docutils==0.17 scikit-image tqdm shapely python-dateutil GitPython glm imageio mpmath noise
pip install git+https://github.com/SciTools/cartopy.git

# Fix the version of Python packages
pip freeze > requirements.txt

# Build the book
make clean
make all

Also I attached my requirements.txt file: requirements.txt

Maybe all these Python packages are not required for building the book itself but can help resolve #33 which I used for creating the list with the packages.

Probably these instructions and this file can help other people to quickly build the latest version of the book.

rougier commented 2 years ago

Thank you very much for the detailed build instructions. Actually, you could put them in a how-to-build.txt and make a PR (if you want).

labdmitriy commented 2 years ago

I think I can create PR this week, could you please tell which format of this file should be? Maybe we can use Markdown file? The versions of TeX Live and cartopy in the description above can change over time, should I fix it somehow? I can also replace exact folder names for downloading and for the project with template names.

rougier commented 2 years ago

Perfect, thanks! Markdown would be fine. As for versions, maybe it's safer to freeze versions even though TeX Live might be more stable. Not sur what you mean with exact folder name but go ahead and we can discuss it on your PR.

labdmitriy commented 2 years ago

Ok, I will create PR this week to discuss the details and propose possible description. Maybe it will be useful to implement it along with issue #33? Then it would be complete guide how to create the book and prepare to use all the code used in the book. But then your help will be required to check if all required libraries are specified for reproducibility.

labdmitriy commented 2 years ago

It would be great if you could answer about the scope of PR (only building the book or with code reproducibility but with your help) described above, so I can start prepare PR this Friday. I updated previous message so maybe there were no additional notifications about it, therefore I decided to repeat the question.

rougier commented 2 years ago

Basically, what you wrote in the issue would be a very good start. For the requirements file, I wrote in the book which package I used we can use your file provided the book compile, including the various python scripts. Else, we would need to freeze versions to a lower version. Also, maybe we could use a dedicated virtual environment just for the book such as not to pollute user's environment.

labdmitriy commented 2 years ago

Ok, thank you, then I will share my thoughts in PR and you will decide if it is correct or we need to change it in some way.

labdmitriy commented 2 years ago

Hi Nicolas, I spent a lot of time to check all the steps and prepare Pull Request but it was failed.

A few comments about it:

After all these steps I have the following error when building the book:

Biber error: [345] Utils.pm:409> ERROR - Error: Found biblatex control file version 3.9, expected version 3.8.

I don't know why this new error has happened but I didn't find how to resolve it. I saw that biblatex was updated recently (22.06.2022) but I am not sure if this is the reason.

Because I didn't expect to spend so much time to prepare PR, I will just show you updated steps.

Install TeX Live https://www.tug.org/texlive/quickinstall.html

cd ~/Downloads/
wget https://mirror.ctan.org/systems/texlive/tlnet/install-tl-unx.tar.gz
tar -xzvf install-tl-unx.tar.gz
cd install-tl-*/

export TEXLIVE_INSTALL_PREFIX=$HOME/texlive
export TEXLIVE_INSTALL_TEXDIR=$HOME/texlive/2022
perl install-tl --no-interaction

# Add to ~/.bashrc
echo -e '\nexport PATH="$PATH:$HOME/texlive/2022/bin/x86_64-linux"\n' >> ~/.bashrc
source ~/.bashrc

# Check installation
which latex

Create virtual environment

sudo apt-get -y install python3-pip python3-venv
python3 -m venv ~/venvs/scientific-visualization-book
source ~/venvs/scientific-visualization-book/bin/activate
python3 -m pip install --upgrade pip setuptools wheel 
pip install docutils==0.17

Clone the repository

sudo apt-get -y install git 
mkdir -p ~/education/sci-viz
cd ~/education/sci-viz
git clone https://github.com/rougier/scientific-visualization-book
cd scientific-visualization-book

Build the book

make clean
make all
labdmitriy commented 2 years ago

I think that the problem is in repository which is used for installing packages. As I understand by default the repository is chosen automatically based on internal logic and can be different for subsequent installation executions, so it is not deterministic and you need to find the repository without this problem and define it while installing packages. I think that it was useful and I will stop here :) Thank you!

labdmitriy commented 2 years ago

Hi @rougier, I wrote the article about the improved solution for the building of your book - https://labdmitriy.github.io/blog/building-scientific-visualization-book/

rougier commented 2 years ago

Many many thanks for all your effort in making the build straightforward and sorry for late answer. I also had issue with biber at some point and had to remove everything to compile it from scratch. I'll merge your PR. Again, many thanks.

labdmitriy commented 2 years ago

Hi @rougier,

I added information in the article about further builds (after initial configuration and the first execution), and PDF compression (I found that you have pdf-compressed option in Makefile and decided to include it for completeness).

Thank you.