sphinx-doc / sphinx

The Sphinx documentation generator
https://www.sphinx-doc.org/
Other
6.22k stars 2.04k forks source link

when a right aligned image extends the next headline #3289

Open Blendify opened 7 years ago

Blendify commented 7 years ago

Subject: when a right aligned image extends the next headline text gets aligned wrong

Problem

Procedure to reproduce the problem

Not sure how to make this happen in a small test case but make a pdf version of our manual by running:

svn checkout https://svn.blender.org/svnroot/bf-manual/trunk/blender_docs
make pdf

Error logs / results

Expected results

The text should smoothly wrap around the image.

Reproducible project / your project

Environment info

tk0miya commented 7 years ago

Sorry, the docs of blender is too large to investigate the problem. I would appreciate it if you could make simple and reproducible project.

jfbu commented 7 years ago

Thanks for reporting. It appears your document is a convincing demonstration that wrapfig LaTeX package is not usable as general automatized way to wrap text around figures. The problem here is mainly that the figures are somewhat tall, and the produced tex file always contains a list (description or itemize), and it is well-known in LaTeX world (and pointed out in wrapfig doc) that wrapfig does not interact well with any kind of LaTeX list.

For the time being all I can propose is that you drop the :align: options, but I guess you want to keep it for html. I need to think about this more.

Looking at your document I had some other issues with LaTeX. Here is a reduced summary of errors. Unfortunately, AUCTeX displays line numbers >= 10000 as 1....

 blender_manual.tex          42 Error   LaTeX Error: Option clash for package geometry.
 blender_manual.tex        1747 Error   Package inputenc Error: Unicode char ✛ (U+271B)
 blender_manual.tex        2014 Error   Package inputenc Error: Unicode char ▼ (U+25BC)
 blender_manual.tex        2014 Error   Package inputenc Error: Unicode char ► (U+25BA)
 blender_manual.tex        2375 Error   Package inputenc Error: Unicode char ◂ (U+25C2)
 blender_manual.tex        2375 Error   Package inputenc Error: Unicode char ▸ (U+25B8)
 blender_manual.tex        1... Error   Package inputenc Error: Unicode char ⏮ (U+23EE)
 [...]
 blender_manual.tex        1... Error   LaTeX Error: Unknown graphics extension: .svg.
 blender_manual.tex        1... Error   LaTeX Error: Unknown graphics extension: .svg.
 blender_manual.tex        1... Error   LaTeX Error: Unknown graphics extension: .gif.
 blender_manual.tex        1... Error   LaTeX Error: Unknown graphics extension: .gif.
 blender_manual.tex        1... Error   LaTeX Error: Something's wrong--perhaps a missing \item.
 blender_manual.tex        1... Error   LaTeX Error: Something's wrong--perhaps a missing \item.
 [...]
 blender_manual.tex        1... Error   LaTeX Error: Unknown graphics extension: .svg.
 blender_manual.tex        1... Error   LaTeX Error: Unknown graphics extension: .svg.
 blender_manual.tex        1... Error   LaTeX Error: Unknown graphics extension: .svg.
 [...]
 blender_manual.tex        2... Error   Package inputenc Error: Unicode char ✗ (U+2717)
 [...]
 blender_manual.tex        2... Error   LaTeX Error: Something's wrong--perhaps a missing \item.
 [...]
 blender_manual.tex        6... Error   Dimension too large.
 blender_manual.tex        6... Error   Dimension too large.
 blender_manual.tex        6... Error   Dimension too large.
 [...]
 blender_manual.tex        6... Error   LaTeX Error: Unknown graphics extension: .gif.
 blender_manual.tex        6... Error   LaTeX Error: Unknown graphics extension: .gif.
 blender_manual.tex        7... Error   LaTeX Error: Unknown graphics extension: .gif.
 blender_manual.tex        7... Error   LaTeX Error: Unknown graphics extension: .gif.
 blender_manual.tex        7... Error   LaTeX Error: Unknown graphics extension: .svg.
 blender_manual.tex        7... Error   LaTeX Error: Unknown graphics extension: .svg.
 blender_manual.tex        8... Error   Package inputenc Error: Unicode char ✔ (U+2714)
 blender_manual.tex        8... Error   Package inputenc Error: Unicode char ✗ (U+2717)
 [...]
 blender_manual.tex        9... Error   Package inputenc Error: Unicode char   (U+2007)
 blender_manual.tex        9... Error   Package inputenc Error: Unicode char   (U+2007)
 blender_manual.tex        9... Error   LaTeX Error: Unknown graphics extension: .svg.
 [...]
 blender_manual.tex        9... Error   LaTeX Error: Unknown graphics extension: .gif.
 blender_manual.tex        1... Error   Package inputenc Error: Unicode char └ (U+2514)
 blender_manual.tex        1... Error   Package inputenc Error: Unicode char ├ (U+251C)
 blender_manual.tex        1... Error   Package inputenc Error: Unicode char ├ (U+251C)
 blender_manual.tex        1... Error   Package inputenc Error: Unicode char ├ (U+251C)
 blender_manual.tex        1... Error   Package inputenc Error: Unicode char └ (U+2514)
 blender_manual.tex        1... Error   Package inputenc Error: Unicode char └ (U+2514)
 blender_manual.tex        1... Error   Package inputenc Error: Unicode char ├ (U+251C)
 blender_manual.tex        1... Error   Package inputenc Error: Unicode char ├ (U+251C)
 blender_manual.tex        1... Error   Package inputenc Error: Unicode char └ (U+2514)
 blender_manual.tex        1... Error   Package inputenc Error: Unicode char ├ (U+251C)
 blender_manual.tex        1... Error   Package inputenc Error: Unicode char └ (U+2514)
 blender_manual.tex        1... Error   Package inputenc Error: Unicode char ├ (U+251C)
 blender_manual.tex        1... Error   Package inputenc Error: Unicode char └ (U+2514)
 blender_manual.tex        1... Error   Package inputenc Error: Unicode char └ (U+2514)
 blender_manual.tex        1... Error   Package inputenc Error: Unicode char └ (U+2514)
 blender_manual.tex        1... Error   Package inputenc Error: Unicode char ├ (U+251C)
 blender_manual.tex        1... Error   Package inputenc Error: Unicode char └ (U+2514)
 blender_manual.tex        1... Error   Package inputenc Error: Unicode char ├ (U+251C)
 blender_manual.tex        1... Error   Package inputenc Error: Unicode char └ (U+2514)
 blender_manual.tex        1... Error   Package inputenc Error: Unicode char ├ (U+251C)
 blender_manual.tex        1... Error   Package inputenc Error: Unicode char └ (U+2514)
 blender_manual.tex        1... Error   Package inputenc Error: Unicode char └ (U+2514)
 blender_manual.tex        1... Error   Package inputenc Error: Unicode char └ (U+2514)
 blender_manual.tex        1... Error   Package inputenc Error: Unicode char ├ (U+251C)
 blender_manual.tex        1... Error   Package inputenc Error: Unicode char └ (U+2514)

Analysis:

  1. the worst error is Dimension too large. From brief look it comes from jpeg files with gigantic dimensions such as physics_smoke_type_domain_vorticity.jpg. Unfortunately the xelatex log does not give the same information in the pdflatex log which I currently have discarded but I recall some dimensions like 32000pt. TeX handles at maximum 16384pt.

  2. There are errors from trying to import gif or svg files into PDFlatex. I recommend you convert all your image files into pdf format. This should make errors from 1. and 2. go away.

  3. There are errors with some Unicode characters not being known to LaTeX. You would need some suitable font and identify the searched-for characters in suitable encodings, I am no expert in this. Hence I tried with xelatex engine and it seems to work (see below).

  4. Finally there are all the missing \item errors. In my brief examination they are all caused by the wrapfig presence trying to wrap text around figures.

Also there was an error with geometry option clash: in your conf.py I recommend you customize geometry rather this (new) way. (since 1.5.3).

# Additional stuff for the LaTeX preamble.
#  'preamble': r'\usepackage[left=0.75in, right=0.75in, top=1.0in, bottom=1.0in]{geometry}',

  'sphinxsetup': 'hmargin=0.75in, vmargin=1in',

To fix the Unicode errors you could use:

latex_engine = 'xelatex'

latex_elements = {
    'fontpkg': '''\
\\setmainfont{DejaVu Serif}
\\setsansfont{DejaVu Sans}
\\setmonofont{DejaVu Sans Mono}''',
}

I have not checked in output PDF, but at least errors are gone. As I use DejaVu Sans Mono in my Emacs buffer I knwo that almost all characters should be OK. But some would be missing.

I will try to come back later after investigating more the issue you reported. But it is an intrinsic fact that wrapping text around objects in LaTeX2e has no automatic solution and that wrapfig suffers from big incompatibility with one of the main LaTeX construct: lists.

Often your images causing issues with wrapfig would barely fit in the page, they are rather tall. I think currently the best way is, for LaTeX to drop the :align: option.

If I get any better idea, I will keep you informed.

By the way the conf.py contains

# The name of an image file (relative to this directory) to place at the top of
# the title page.
latex_logo = "../resources/theme/sidebar_logo.png"

but this file is absent from svn checkout.

There is a minor problem in the Makefile which has for pdf target sphinx-build:

pdf: .FORCE
    QUICKY_CHAPTERS=$(QUICKY_CHAPTERS) \
    sphinx-build -b latex ./manual "$(BUILDDIR)/latex"
    make -C "$(BUILDDIR)/latex" LATEXOPTS="-interaction nonstopmode"

it should be $(SPHINXBUILD) for easier use of another Sphinx version.

I tried it with 1.6-release branch, unfortunately it seems the invocation as above

    make -C "$(BUILDDIR)/latex" LATEXOPTS="-interaction nonstopmode"

is now broken. One needs make -C build/latex all-pdf, or perhaps

make -C "build/latex" all-pdf LATEXOPTS="--interaction=nonstopmode"

but the nonstopmode in this way is not obeyed by latexmk.

===========Latexmk: Missing input file: 'editors_sequencer_strips_introduction_strip-graphic.svg' from line
  '! Unable to load picture or PDF file 'editors_sequencer_strips_introduction_strip-graphic.svg'.'
Latexmk: Missing input file: 'editors_sequencer_strips_introduction_strip-graphic.svg' from line
  '! Unable to load picture or PDF file 'editors_sequencer_strips_introduction_strip-graphic.svg'.'
Latexmk: Log file says output to 'blender_manual.xdv'
Latexmk: Errors, so I did not complete making targets
Collected error summary (may duplicate other messages):
  xelatex: Command for 'xelatex' gave return code 1
      Refer to 'blender_manual.log' for details
Latexmk: Use the -f option to force complete processing,
 unless error was exceeding maximum runs of latex/pdflatex.
make: *** [Makefile:28: blender_manual.pdf] Error 12

so I used

make -C "build/latex" all-pdf LATEXOPTS="-f --interaction=nonstopmode"

and it did produce the pdf. Here is the end of the console output:

blender_manual.xdv -> blender_manual.pdf
[1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24][25][26][27][28][29][30][31][32][33][34][35][36][37][38][39][40][41][42][43][44][45][46][47][48][49][50][51][52][53][54][55][56][57][58][59][60][61][62][63][64][65][66][67][68][69][70][71][72][73][74][75][76][77][78][79][80][81][82][83][84][85][86][87][88][89][90][91][92][93][94][95][96][97][98][99][100][101][102][103][104][105][106][107][108][109][110][111][112][113][114][115][116][117][118][119][120][121][122][123][124][125][126][127][128][129][130][131][132][133][134][135][136][137][138][139][140][141][142][143
xdvipdfmx:warning: JPEG: Inconsistent resolution may have specified in Exif and JFIF: 182.88x182.88 - 183x183
][144][145][146][147][148][149][150][151][152][153][154][155][156][157][158][159][160][161][162][163][164][165][166][167][168][169][170][171][172][173][174][175][176][177][178][179][180][181][182][183][184][185][186][187][188][189][190][191][192][193][194][195][196][197][198][199][200][201][202][203][204][205][206][207][208][209][210][211][212][213][214][215][216][217][218][219][220][221][222][223][224][225][226][227][228][229][230][231][232][233][234][235][236][237][238][239][240][241][242][243][244][245][246][247][248][249][250][251][252][253][254][255][256][257][258][259][260][261][262][263][264][265][266][267]
xdvipdfmx:warning: Invalid CMap mapping entry. (ignored)
No Unicode mapping available: GID=3304
No Unicode mapping available: GID=3305
No Unicode mapping available: GID=3306
No Unicode mapping available: GID=3291
No Unicode mapping available: GID=3292
No Unicode mapping available: GID=3293

xdvipdfmx:warning: Invalid CMap mapping entry. (ignored)
No Unicode mapping available: GID=3159
No Unicode mapping available: GID=3160
No Unicode mapping available: GID=3161

7216636 bytes written
Latexmk: Errors, in force_mode: so I tried finishing targets
Collected error summary (may duplicate other messages):
  xelatex: Command for 'xelatex' gave return code 1
      Refer to 'blender_manual.log' for details
make: *** [Makefile:28: blender_manual.pdf] Error 12

Unfortunately as you see this stops after a ridiculously small number of 267 pages whereas your document has about 2000 pages... the end of the log is

[265]
Overfull \hbox (228.81696pt too wide) in paragraph at lines 16177--16177
[][][] 
 []

Overfull \hbox (228.81696pt too wide) in paragraph at lines 16179--16179
[][][] 
 []

! Unable to load picture or PDF file 'editors_sequencer_strips_introduction_str
ip-graphic.svg'.
<to be read again> 
                   }
l.16190 ...strips_introduction_strip-graphic}.svg}

Here is how much of TeX's memory you used:
 24041 strings out of 493608
 499426 string characters out of 6150099
 559608 words of memory out of 5000000
 27277 multiletter control sequences out of 15000+600000
 13811 words of font info for 100 fonts, out of 8000000 for 9000
 1040 hyphenation exceptions out of 8191
 36i,18n,67p,10390b,567s stack positions out of 5000i,500n,10000p,200000b,80000s

Output written on blender_manual.xdv (267 pages, 4249496 bytes).

showing that xelatex stopped at the problem with the svg file which it does not know how to import in pdf ntaively.

I will need to investigate how to pass option to Latexmk to get further in processing and ignore such errors. (this has now been done for 1.6 release at #3695, 99c5d79)

jfbu commented 7 years ago

Only robust workaround I can propose is the following. Add to the conf.py this config:

latex_elements = {
# The paper size ('letterpaper' or 'a4paper').
  'papersize': 'a4paper',

# The font size ('10pt', '11pt' or '12pt').
  'pointsize': '10pt',

# Additional stuff for the LaTeX preamble.

  'sphinxsetup': 'hmargin=0.75in, vmargin=1in',

  'preamble': r'\renewenvironment{wrapfigure}[2]{\begin{figure}[H]}{\end{figure}}',

}

This way you can still have the :align: options used in html builds. But for PDF, the wrapping is simply dropped, and replaced by inclusion of the graphics centered on the page.

Can you try it out ?

Your document systematically has lists following very soon rather tall pictures: LaTeX package wrapfig is simply unable to handle that to satisfaction. In fact the package was thought out for small pictures and manual tuning by user... it is a limitation, not so much of TeX perhaps, but of LaTeX2e, that it simply has no robust solution for flowing automatically text around graphics, and particularly in a way compatible with lists. For example the wrapfig manual says not use it near bottom of page if figure is too tall, but how are we supposed to detect that automatically !?!

With the above, the only errors remaining in your documents are:

  1. the missing Unicode characters: there is perhaps a pure pdflatex solution, but at least xelatex or lualatex with suitable OpenType font do solve it,

  2. use of .gif and .svg graphics which are not supported by the pdflatex backend,

  3. use of .jpeg graphics with too big dimensions. The maximal dimension for TeX is not much more than 5meters. The pixel size should be set-up so that the pictures do not create Dimension too large error.

Blendify commented 7 years ago

Yes, I will test in the next couple of days.

jfbu commented 7 years ago

Your document also displays the problem #3532. I tested the proposed fix #3686, and it does solve the issues with captions of figures in table cells one sees in your document.

I also noticed in passing the #3702 issue.

Blendify commented 7 years ago

Only robust workaround I can propose is the following. Add to the conf.py this config:

Committed and will test in a bit

the missing Unicode characters: there is perhaps a pure pdflatex solution, but at least xelatex or lualatex with suitable OpenType font do solve it,

Can you provide more information on how to do this? I am not familiar with all the latex options.

use of .gif and .svg graphics which are not supported by the pdflatex backend,

We are waiting for the patch to automatically convert them to be merged but can try removing them.

jfbu commented 7 years ago

About the missing Unicode characters:

  1. partial solution with xelatex was mentioned in an earlier comment

    latex_engine = 'xelatex'
    
    latex_elements = {
       'fontpkg': '''
    \\setmainfont{DejaVu Serif}
    \\setsansfont{DejaVu Sans}
    \\setmonofont{DejaVu Sans Mono}
    ''',
    }

    But this doesn't work for all characters. I think the main problematic ones were the media control characters used in timeline.rst file. On my Mac OS they are rendered using Apple Emoji font, but for some reason not all of them. Anyway, now that I looked more closely at the PDF (section Player Control) I see that it would not be good to use the colored emoticons of Apple here. So see next item for alternative.

  2. finally I spent some time on this and found a complete solution (in the sense of avoiding all errors) with pdflatex. Inside latex_elements, from conf.py, use:

      'preamble': r'''
    \renewenvironment{wrapfigure}[2]{\begin{figure}[H]}{\end{figure}}
    % Fix missing Unicode characters
    \usepackage{pmboxdraw}% BOX DRAWINGS characters
    \DeclareUnicodeCharacter{2007}{\nobreakspace}% FIGURE SPACE
    \usepackage{pifont}
    \usepackage{newunicodechar}
    \newunicodechar{✓}{\ding{51}}
    \newunicodechar{✗}{\ding{55}}
    \newunicodechar{✔}{\ding{52}}
    \newunicodechar{✛}{\ensuremath{+}}% (U+271B)
    \newunicodechar{▼}{\ensuremath{\blacktriangledown}}% (U+25BC)
    \newunicodechar{►}{\ensuremath{\blacktriangleright}}% use (U+25B6) instead
    %  note: a \blackpointerright is provided by stix package,
    %  \newunicodechar{►}{\ensuremath{\blackpointerright}}% (U+25BA)
    \newunicodechar{◂}{{\small\ensuremath{\blacktriangleleft}}}% (U+25C2)
    \newunicodechar{▸}{{\small\ensuremath{\blacktriangleright}}}% (U+25B8)
    \newunicodechar{◀}{\ensuremath{\blacktriangleleft}}% (U+25C0)
    \newunicodechar{▶}{\ensuremath{\blacktriangleright}}% (U+25B6)
    % from timeline.rst
    % .. |first| unicode:: U+023EE
    % .. |last|  unicode:: U+023ED
    % .. |rewind| unicode:: U+025C0
    % .. |play|   unicode:: U+025B6
    % .. |previous| unicode:: U+023EA
    % .. |next|     unicode:: U+023E9
    % .. |pause| unicode:: U+023F8
    % if the characters are available as small png files:
    % \DeclareUnicodeCharacter{23EE}{\includegraphics{mediacontrol23EE.png}}%
    % else here is desperate alternative
    \def\poormanmediacontrol#1%
      {{\fboxsep1pt\ttfamily\fbox{\vbox{\hsize1sp\tiny\baselineskip0pt\hbox{media}\hbox{#1}}}}}
    \DeclareUnicodeCharacter{23EE}{\poormanmediacontrol{first}}
    \DeclareUnicodeCharacter{23ED}{\poormanmediacontrol{last}}
    \DeclareUnicodeCharacter{25C0}{\poormanmediacontrol{rewi.}}
    \DeclareUnicodeCharacter{25B6}{\poormanmediacontrol{play}}
    \DeclareUnicodeCharacter{23EA}{\poormanmediacontrol{prev.}}
    \DeclareUnicodeCharacter{23E9}{\poormanmediacontrol{next}}
    \DeclareUnicodeCharacter{23F8}{\poormanmediacontrol{pause}}
    ''',
    

With this all "missing Unicode characters" errors are gone. Check the output in particular for the media control ones in section Player Control of PDF. As indicated, better would be that you have png's prepared to match the actual things displayed in figure "Player Controls" at start of this section, and use the definitions with \includegraphics I indicated above.

The BOX DRAWINGS characters for displaying a tree layout (from pipline/index.rst) render almost ok (at least on page 1851... ), but unfortunately it seems the (U+2500) from the source gets converted by Sphinx into an hyphen -, which is a pity because if I insert the directly in the tex file, then package pmboxdraw which I added to the premable does its job fine. But the tex file does not receive the original character but the hyphen.

jfbu commented 7 years ago

The problem with (U+2500, BOX DRAWINGS LIGHT HORIZONTAL) comes from https://github.com/sphinx-doc/sphinx/blob/master/sphinx/util/texescape.py#L43

You would need to monkey patch this for the time being.

What do you think @tk0miya, it looks weird to have one and only BOX DRAWINGS character treated especially. And it is bad for xelatex, lualatex where the user may have a font such as DejaVu which contains them. And even for pdflatex there is pmboxdraw package which covers these characters (admittedly, then text search in PDF and copy-paste form PDF does not work with them; there could be a font for pdftex which has them and would be better solution, I did not investigate more after having found package pmboxdraw which provides BOX DRAWINGS and other things via "TeX horizontal and vertical rules").

Also

    ('‣', r'\(\rightarrow\)'),
    ('✓', r'\(\checkmark\)'),

are problematic. Because pifont package provides the checkmark as \ding{51} and the heavy check mark as \ding{52}. And it seems arbitrary to map to \rightarrow.

jfbu commented 7 years ago

Update: there is also BOX DRAWINGS LIGHT VERTICAL at https://github.com/sphinx-doc/sphinx/blob/master/sphinx/util/texescape.py#L48, and also BOX DRAWINGS LIGHT DIAGONAL UPPER LEFT TO LOWER RIGHT

jfbu commented 7 years ago

regarding the image files

quote:

  use of .gif and .svg graphics which are not supported by the pdflatex backend,

We are waiting for the patch to automatically convert them to be merged but can try removing them.

You don't have to remove them, just make sure png's (for gif's) or pdf's for (svg) are also available but then you should use the globbing character * as extension: http://www.sphinx-doc.org/en/stable/rest.html?highlight=image#images

jfbu commented 7 years ago

The media control characters are almost all available via FontAwesome, for which there is also pdflatex support. I am adding this just in case it might help other people visiting this thread, but in your specific case, you should better use the fonts or icons of the original Player Controls from Blender.

Anyway, here is first how to handle this using xelatex:

latex_engine = 'xelatex'

latex_elements = {
# The paper size ('letterpaper' or 'a4paper').
  'papersize': 'a4paper',

# The font size ('10pt', '11pt' or '12pt').
  'pointsize': '10pt',

# Additional stuff for the LaTeX preamble.

  'sphinxsetup': 'hmargin=0.75in, vmargin=1in',

  'classoptions': ',openany,oneside',
#  'babel': '\\usepackage[english]{babel}',
# note that xelatex will use polyglossia by default, but if 'babel' key is redefined like above, it will use babel package.

    'fontpkg': r'''
\setmainfont{DejaVu Serif}
\setsansfont{DejaVu Sans}
\setmonofont{DejaVu Sans Mono}
''',

   'preamble': r'''
\renewenvironment{wrapfigure}[2]{\begin{figure}[H]}{\end{figure}}
\newfontfamily{\FA}{FontAwesome}
\catcode`^^^^23ee\active \protected\def^^^^23ee{{\FA\symbol{"F049}}} % {\faFastBackward}
\catcode`^^^^23ea\active \protected\def^^^^23ea{{\FA\symbol{"F04A}}} % {\faBackward}
\catcode`^^^^25c0\active \protected\def^^^^25c0{\reflectbox{^^^^25b6}} % {\reflectbox{\faPlay}}
\catcode`^^^^25b6\active \protected\def^^^^25b6{{\FA\symbol{"F04B}}} % {\faPlay}
\catcode`^^^^23e9\active \protected\def^^^^23e9{{\FA\symbol{"F04E}}} % {\faForward}
\catcode`^^^^23ed\active \protected\def^^^^23ed{{\FA\symbol{"F050}}} % {\faFastForward}
\catcode`^^^^23f8\active \protected\def^^^^23f8{{\FA\symbol{"F04C}}} % {\faPause}
''',

}

I used DejaVu fonts, many other OpenType fonts will do. The point is that DejaVu does have all the Box Drawings characters and other special Unicode characters in your document, except the Media Control characters. And unfortunately for the tree with box drawing characters, there is issue #3706. Also the checkmark is affected by this issue, hence uses a math font.

Regarding the directory tree,it is not too bad, because the hyphen is actually at the correct height above baseline, contrarily with the pdflatex situation using \pmboxdraw.

The remaining characters are picked in the private area used by FontAwesome. I checked the latex package documentation and support files to figure out the codepoints (one can get them from copying-pasting the glyphs from the PDF of FontAwesome doc to Emacs Buffer actually).

I avoided loading package fontawesome because it defines hundreds of macros, but if one does load it, then this is a bit easier:

\usepackage{fontawesome}
\catcode`^^^^23ee\active \protected\def^^^^23ee{\faFastBackward}

as one does not need to go into support files to discover the code-points, the PDF doc telling the macro names.

Further alternative is to do

\usepackage{fontawesome}
\usepackage{newunicodechar}
\newunicodechar{⏮}{\faFastBackward}
% etc..

which is even more condensed.

It is possible to use package fontawesome with traditional pdflatex engine. I did it this way then:

\usepackage{fontawesome}
\DeclareUnicodeCharacter{23EE}{\faFastBackward}% first
\DeclareUnicodeCharacter{23EA}{\faBackward}% prev.
\DeclareUnicodeCharacter{25C0}{\reflectbox{\faPlay}}% rewi. No mirror of "Play" in fontawesome ?
\DeclareUnicodeCharacter{25B6}{\faPlay}% play
\DeclareUnicodeCharacter{23E9}{\faForward}% next
\DeclareUnicodeCharacter{23ED}{\faFastForward}% last
\DeclareUnicodeCharacter{23F8}{\faPause}% pause

but one can also avoid loading package fontawesome in its entirety, but one has to dig into the support files to figure out what to use.. ok let's not complicate. The above replaces the \poormanmediacontrol macro from my earlier comment.

Anyway, for your document, the remaining errors are the ones from bad format of graphics files for latex.

jfbu commented 7 years ago

ah sorry I had overlooked missing characters with xelatex+DejaVu:

$ grep "Missing character" blender_manual.log | sort | uniq
Missing character: There is no ✔ in font DejaVu Serif/OT:script=latn;language=D
Missing character: There is no ✗ in font DejaVu Serif/OT:script=latn;language=D
Missing character: There is no ✛ in font DejaVu Serif/OT:script=latn;language=D

so I need to extend a bit

xelatex

(edited because LaTeX \checkmark can not be boldened! (easily)) (edited again because the line with loading fontawesome was missing)

   'preamble': r'''
\renewenvironment{wrapfigure}[2]{\begin{figure}[H]}{\end{figure}}

\usepackage{newunicodechar}
\usepackage{pifont}
\newunicodechar{✔}{\ding{52}}% 
% nota bene Sphinx already replaced ✓ by \checkmark, which uses a math font 
% and is provided by amssymb. But it can not be boldened! (easily)
\newunicodechar{✗}{\ding{55}}
\newunicodechar{✛}{\ding{59}}

\usepackage{fontawesome}
\newunicodechar{⏮}{\faFastBackward}
\newunicodechar{⏪}{\faBackward}
\newunicodechar{▶}{\faPlay}
\newunicodechar{⏩}{\faForward}
\newunicodechar{⏭}{\faFastForward}
\newunicodechar{⏸}{\faPause}
\newunicodechar{◀}{\reflectbox{▶}}
''',

pdflatex

So the pdflatex setting becomes:

   'preamble': r'''
\renewenvironment{wrapfigure}[2]{\begin{figure}[H]}{\end{figure}}
% Fix missing Unicode characters
\usepackage{pmboxdraw}% BOX DRAWINGS characters
\DeclareUnicodeCharacter{2007}{\nobreakspace}% FIGURE SPACE
\usepackage{pifont}
\usepackage{newunicodechar}
\newunicodechar{✓}{\ding{51}}% but is escaped by Sphinx to \(\checkmark\)
% so the character is never encountered actually
\newunicodechar{✗}{\ding{55}}
\newunicodechar{✔}{\ding{52}}%
\newunicodechar{✛}{\ding{59}}% (U+271B) OPEN CENTRE CROSS
\newunicodechar{▼}{\ensuremath{\blacktriangledown}}% (U+25BC)
\newunicodechar{►}{\ensuremath{\blacktriangleright}}% use (U+25B6) instead
%  note: a \blackpointerright is provided by stix package,
%  \newunicodechar{►}{\ensuremath{\blackpointerright}}% (U+25BA)
\newunicodechar{◂}{{\small\ensuremath{\blacktriangleleft}}}% (U+25C2)
\newunicodechar{▸}{{\small\ensuremath{\blacktriangleright}}}% (U+25B8)
\newunicodechar{◀}{\ensuremath{\blacktriangleleft}}% (U+25C0)
\newunicodechar{▶}{\ensuremath{\blacktriangleright}}% (U+25B6)
% from timeline.rst
% .. |first| unicode:: U+023EE
% .. |last|  unicode:: U+023ED
% .. |rewind| unicode:: U+025C0
% .. |play|   unicode:: U+025B6
% .. |previous| unicode:: U+023EA
% .. |next|     unicode:: U+023E9
% .. |pause| unicode:: U+023F8
\usepackage{fontawesome}
\DeclareUnicodeCharacter{23EE}{\faFastBackward}% first
\DeclareUnicodeCharacter{23EA}{\faBackward}% prev.
\DeclareUnicodeCharacter{25C0}{\reflectbox{\FAtwo\symbol{217}}}% rewi. No mirror of "Play" in fontawesome ?
\DeclareUnicodeCharacter{25B6}{\faPlay}% play
\DeclareUnicodeCharacter{23E9}{\faForward}% next
\DeclareUnicodeCharacter{23ED}{\faFastForward}% last
\DeclareUnicodeCharacter{23F8}{\faPause}% pause
''',
Blendify commented 7 years ago

I tried doing your modifications but I get:

Encoding error:
'ascii' codec can't decode byte 0xe2 in position 132: ordinal not in range(128)
The full traceback has been saved in /tmp/sphinx-err-ndMMro.log, if you want to report the issue to the developers.
# Sphinx version: 1.5.5
# Python version: 2.7.12 (CPython)
# Docutils version: 0.13.1 release
# Jinja2 version: 2.9.6
# Last messages:
#   
#   resolving references...
#   /home/aaron/Documents/Open Source Projects/Blender Foundation/blender_docs/manual/physics/collision.rst:79: WARNING: undefined label: fig-collision-soft-plane (if the link has no caption the label must precede a section header)
#   writing...
#   /home/aaron/Documents/Open Source Projects/Blender Foundation/blender_docs/manual/getting_started/about/community.rst:: WARNING: unusable reference target found: irc://irc.freenode.net/blender
#   /home/aaron/Documents/Open Source Projects/Blender Foundation/blender_docs/manual/getting_started/about/community.rst:: WARNING: unusable reference target found: irc://irc.freenode.net/blenderchat
#   /home/aaron/Documents/Open Source Projects/Blender Foundation/blender_docs/manual/getting_started/about/community.rst:: WARNING: unusable reference target found: irc://irc.freenode.net/blendercoders
#   /home/aaron/Documents/Open Source Projects/Blender Foundation/blender_docs/manual/getting_started/about/community.rst:: WARNING: unusable reference target found: irc://irc.freenode.net/blenderpython
#   /home/aaron/Documents/Open Source Projects/Blender Foundation/blender_docs/manual/getting_started/about/community.rst:: WARNING: unusable reference target found: irc://irc.freenode.net/gameblender
#   /home/aaron/Documents/Open Source Projects/Blender Foundation/blender_docs/manual/getting_started/about/community.rst:: WARNING: unusable reference target found: irc://irc.freenode.net/blenderwiki
# Loaded extensions:
#   youtube (unknown version) from /home/aaron/Documents/Open Source Projects/Blender Foundation/blender_docs/exts/youtube.pyc
#   sphinx.ext.mathjax (1.5.5) from /usr/local/lib/python2.7/dist-packages/sphinx/ext/mathjax.pyc
#   googleanalytics (0.1) from /home/aaron/Documents/Open Source Projects/Blender Foundation/blender_docs/exts/googleanalytics.pyc
#   alabaster (0.7.10) from /usr/local/lib/python2.7/dist-packages/alabaster/__init__.pyc
#   vimeo (unknown version) from /home/aaron/Documents/Open Source Projects/Blender Foundation/blender_docs/exts/vimeo.pyc
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/sphinx/cmdline.py", line 296, in main
    app.build(opts.force_all, filenames)
  File "/usr/local/lib/python2.7/dist-packages/sphinx/application.py", line 333, in build
    self.builder.build_update()
  File "/usr/local/lib/python2.7/dist-packages/sphinx/builders/__init__.py", line 246, in build_update
    self.build(['__all__'], to_build)
  File "/usr/local/lib/python2.7/dist-packages/sphinx/builders/__init__.py", line 322, in build
    self.write(docnames, list(updated_docnames), method)
  File "/usr/local/lib/python2.7/dist-packages/sphinx/builders/latex.py", line 130, in write
    docwriter.write(doctree, destination)
  File "/usr/local/lib/python2.7/dist-packages/docutils/writers/__init__.py", line 80, in write
    self.translate()
  File "/usr/local/lib/python2.7/dist-packages/sphinx/writers/latex.py", line 162, in translate
    self.output = visitor.astext()
  File "/usr/local/lib/python2.7/dist-packages/sphinx/writers/latex.py", line 576, in astext
    return LaTeXRenderer().render(DEFAULT_TEMPLATE, self.elements)
  File "/usr/local/lib/python2.7/dist-packages/sphinx/util/template.py", line 25, in render
    return self.env.get_template(template_name).render(context)
  File "/usr/local/lib/python2.7/dist-packages/jinja2/environment.py", line 1008, in render
    return self.environment.handle_exception(exc_info, True)
  File "/usr/local/lib/python2.7/dist-packages/jinja2/environment.py", line 780, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python2.7/dist-packages/sphinx/templates/latex/content.tex_t", line 51, in top-level template code
    <%= printindex %>
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 132: ordinal not in range(128)
jfbu commented 7 years ago

I do all my work with Python 3. With Python 2, I confirm your error but this is simply due to the use of a Unicode string in conf.py without uprefix. So, for the xelatex config for example it would be something like

   'preamble': u'''
\\renewenvironment{wrapfigure}[2]{\\begin{figure}[H]}{\\end{figure}}
\\usepackage{newunicodechar}

\\usepackage{pifont}
\\newunicodechar{✔}{\\ding{52}}
\\newunicodechar{✗}{\\ding{55}}
\\newunicodechar{✛}{\\ding{59}}

\\usepackage{fontawesome}
\\newunicodechar{⏮}{\\faFastBackward}
\\newunicodechar{⏪}{\\faBackward}
\\newunicodechar{▶}{\\faPlay}
\\newunicodechar{⏩}{\\faForward}
\\newunicodechar{⏭}{\\faFastForward}
\\newunicodechar{⏸}{\\faPause}
\\newunicodechar{◀}{\\reflectbox{▶}}
''',

where I added u prefix and doubled the backslashes. I vaguely think one can use ur without doubling the backslashes but I have an overdue deadline today and can't check. I just checked that with this modification the error you report is gone.

(edited to correct the line with the , and move the pifont line before it) (and again to add missing line to load package fontawesome)

jfbu commented 7 years ago

(I did it now and as page numbers go in the thousands, there is some overflow in page margin, but this can be fixed using e.g. tocloft package, see there, particularly the lines

\\usepackage[titles]{tocloft}
\\cftsetpnumwidth {1.25cm}\\cftsetrmarg{1.5cm}

)

Blendify commented 7 years ago

Now that sphinx 1.6.1 is out I had another test but I am still getting errors:

build succeeded.
make -C "build/latex" LATEXOPTS="-interaction nonstopmode"
make[1]: Entering directory '/home/aaron/Documents/Open Source Projects/Blender Foundation/blender_docs/build/latex'
latexmk -pdf -dvi- -ps-  'blender_manual.tex'
Latexmk: This is Latexmk, John Collins, 1 January 2015, version: 4.41.
Latexmk: applying rule 'pdflatex'...
Rule 'pdflatex': Rules & subrules not known to be previously run:
   pdflatex
Rule 'pdflatex': The following rules & subrules became out-of-date:
      'pdflatex'
------------
Run number 1 of rule 'pdflatex'
------------
------------
Running 'xelatex -interaction nonstopmode  -recorder  "blender_manual.tex"'
------------
This is XeTeX, Version 3.14159265-2.6-0.99992 (TeX Live 2015/Debian) (preloaded format=xelatex)
 restricted \write18 enabled.
entering extended mode
(./blender_manual.tex
LaTeX2e <2016/02/01>
Babel <3.9q> and hyphenation patterns for 3 language(s) loaded.
(./sphinxmanual.cls
Document Class: sphinxmanual 2017/03/26 v1.6 Document class (Sphinx manual)
(/usr/share/texlive/texmf-dist/tex/latex/base/report.cls
Document Class: report 2014/09/29 v1.4h Standard LaTeX document class
(/usr/share/texlive/texmf-dist/tex/latex/base/size10.clo)))
(/usr/share/texlive/texmf-dist/tex/latex/cmap/cmap.sty

Package cmap Warning: pdftex not detected - exiting.

) (/usr/share/texlive/texmf-dist/tex/latex/fontspec/fontspec.sty
(/usr/share/texlive/texmf-dist/tex/latex/l3kernel/expl3.sty
(/usr/share/texlive/texmf-dist/tex/latex/l3kernel/expl3-code.tex)
(/usr/share/texlive/texmf-dist/tex/latex/l3kernel/l3xdvipdfmx.def))
(/usr/share/texlive/texmf-dist/tex/latex/l3packages/xparse/xparse.sty)
(/usr/share/texlive/texmf-dist/tex/latex/fontspec/fontspec-xetex.sty
(/usr/share/texlive/texmf-dist/tex/latex/base/fontenc.sty
(/usr/share/texlive/texmf-dist/tex/latex/euenc/eu1enc.def)
(/usr/share/texlive/texmf-dist/tex/latex/euenc/eu1lmr.fd))
(/usr/share/texlive/texmf-dist/tex/xelatex/xunicode/xunicode.sty
(/usr/share/texmf/tex/latex/tipa/t3enc.def
(/usr/share/texlive/texmf-dist/tex/latex/euenc/eu1lmss.fd))
(/usr/share/texlive/texmf-dist/tex/latex/graphics/graphicx.sty
(/usr/share/texlive/texmf-dist/tex/latex/graphics/keyval.sty)
(/usr/share/texlive/texmf-dist/tex/latex/graphics/graphics.sty
(/usr/share/texlive/texmf-dist/tex/latex/graphics/trig.sty)
(/usr/share/texlive/texmf-dist/tex/latex/latexconfig/graphics.cfg)
(/usr/share/texlive/texmf-dist/tex/xelatex/xetex-def/xetex.def
(/usr/share/texlive/texmf-dist/tex/generic/oberdiek/infwarerr.sty)
(/usr/share/texlive/texmf-dist/tex/generic/oberdiek/ltxcmds.sty)))))
(/usr/share/texlive/texmf-dist/tex/latex/fontspec/fontspec.cfg)))
(/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsmath.sty
For additional information on amsmath, use the `?' option.
(/usr/share/texlive/texmf-dist/tex/latex/amsmath/amstext.sty
(/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsgen.sty))
(/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsbsy.sty)
(/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsopn.sty))
(/usr/share/texlive/texmf-dist/tex/latex/amsfonts/amssymb.sty
(/usr/share/texlive/texmf-dist/tex/latex/amsfonts/amsfonts.sty))
(/usr/share/texlive/texmf-dist/tex/latex/polyglossia/polyglossia.sty
(/usr/share/texlive/texmf-dist/tex/latex/etoolbox/etoolbox.sty)
(/usr/share/texlive/texmf-dist/tex/latex/makecmds/makecmds.sty)
(/usr/share/texlive/texmf-dist/tex/latex/xkeyval/xkeyval.sty
(/usr/share/texlive/texmf-dist/tex/generic/xkeyval/xkeyval.tex
(/usr/share/texlive/texmf-dist/tex/generic/xkeyval/xkvutils.tex)))
(/usr/share/texlive/texmf-dist/tex/generic/oberdiek/ifluatex.sty)
(/usr/share/texlive/texmf-dist/tex/generic/ifxetex/ifxetex.sty))
(/usr/share/texlive/texmf-dist/tex/latex/polyglossia/gloss-english.ldf)
(/usr/share/texlive/texmf-dist/tex/latex/fncychap/fncychap.sty) (./sphinx.sty
(/usr/share/texlive/texmf-dist/tex/latex/fancyhdr/fancyhdr.sty)
(/usr/share/texlive/texmf-dist/tex/latex/base/textcomp.sty
(/usr/share/texlive/texmf-dist/tex/latex/base/ts1enc.def))
(/usr/share/texlive/texmf-dist/tex/latex/titlesec/titlesec.sty)

 Package Sphinx Info: 
 **** titlesec 2.10.1 successfully patched for bugfix ****

(/usr/share/texlive/texmf-dist/tex/latex/tabulary/tabulary.sty
(/usr/share/texlive/texmf-dist/tex/latex/tools/array.sty))
(/usr/share/texlive/texmf-dist/tex/latex/tools/longtable.sty)
(/usr/share/texlive/texmf-dist/tex/latex/varwidth/varwidth.sty)
(./sphinxmulticell.sty)
(/usr/share/texlive/texmf-dist/tex/latex/base/makeidx.sty)
(/usr/share/texlive/texmf-dist/tex/latex/framed/framed.sty)
(/usr/share/texlive/texmf-dist/tex/latex/xcolor/xcolor.sty
(/usr/share/texlive/texmf-dist/tex/latex/latexconfig/color.cfg))
(/usr/share/texlive/texmf-dist/tex/latex/fancyvrb/fancyvrb.sty
Style option: `fancyvrb' v2.7a, with DG/SPQR fixes, and firstline=lastline fix 
<2008/02/07> (tvz)) (./footnotehyper-sphinx.sty)
(/usr/share/texlive/texmf-dist/tex/latex/float/float.sty)
(/usr/share/texlive/texmf-dist/tex/latex/wrapfig/wrapfig.sty)
(/usr/share/texlive/texmf-dist/tex/latex/parskip/parskip.sty)
(/usr/share/texlive/texmf-dist/tex/latex/base/alltt.sty)
(/usr/share/texlive/texmf-dist/tex/latex/upquote/upquote.sty)
(/usr/share/texlive/texmf-dist/tex/latex/capt-of/capt-of.sty)
(/usr/share/texlive/texmf-dist/tex/latex/needspace/needspace.sty)
(./sphinxhighlight.sty)
(/usr/share/texlive/texmf-dist/tex/latex/oberdiek/kvoptions.sty
(/usr/share/texlive/texmf-dist/tex/generic/oberdiek/kvsetkeys.sty
(/usr/share/texlive/texmf-dist/tex/generic/oberdiek/etexcmds.sty))))
(/usr/share/texlive/texmf-dist/tex/latex/geometry/geometry.sty
(/usr/share/texlive/texmf-dist/tex/generic/oberdiek/ifpdf.sty)
(/usr/share/texlive/texmf-dist/tex/generic/oberdiek/ifvtex.sty))
(/usr/share/texlive/texmf-dist/tex/latex/hyperref/hyperref.sty
(/usr/share/texlive/texmf-dist/tex/generic/oberdiek/hobsub-hyperref.sty
(/usr/share/texlive/texmf-dist/tex/generic/oberdiek/hobsub-generic.sty))
(/usr/share/texlive/texmf-dist/tex/latex/oberdiek/auxhook.sty)
(/usr/share/texlive/texmf-dist/tex/latex/hyperref/pd1enc.def)
(/usr/share/texlive/texmf-dist/tex/latex/latexconfig/hyperref.cfg)
(/usr/share/texlive/texmf-dist/tex/latex/hyperref/puenc.def)
(/usr/share/texlive/texmf-dist/tex/latex/url/url.sty))

Package hyperref Message: Driver (autodetected): hxetex.

(/usr/share/texlive/texmf-dist/tex/latex/hyperref/hxetex.def
(/usr/share/texlive/texmf-dist/tex/generic/oberdiek/stringenc.sty)
(/usr/share/texlive/texmf-dist/tex/latex/oberdiek/rerunfilecheck.sty))
(/usr/share/texlive/texmf-dist/tex/latex/oberdiek/hypcap.sty)
(/usr/share/texlive/texmf-dist/tex/latex/newunicodechar/newunicodechar.sty)
(/usr/share/texlive/texmf-dist/tex/latex/psnfss/pifont.sty
(/usr/share/texlive/texmf-dist/tex/latex/psnfss/upzd.fd)
(/usr/share/texlive/texmf-dist/tex/latex/psnfss/upsy.fd))

! LaTeX Error: File `fontawesome.sty' not found.

Type X to quit or <RETURN> to proceed,
or enter new name. (Default extension: sty)

Enter file name: 
! Emergency stop.
<read *> 

l.52 \newunicodechar
                    {⏮}{\faFastBackward}^^M
No pages of output.
Transcript written on blender_manual.log.
Latexmk: Missing input file: 'fontawesome.sty' from line
  '! LaTeX Error: File `fontawesome.sty' not found.'
Latexmk: Errors, so I did not complete making targets
Collected error summary (may duplicate other messages):
  pdflatex: Command for 'pdflatex' gave return code 256
Latexmk: Use the -f option to force complete processing,
 unless error was exceeding maximum runs of latex/pdflatex.
Makefile:32: recipe for target 'blender_manual.pdf' failed
make[1]: *** [blender_manual.pdf] Error 12
make[1]: Leaving directory '/home/aaron/Documents/Open Source Projects/Blender Foundation/blender_docs/build/latex'
Makefile:94: recipe for target 'pdf' failed
make: *** [pdf] Error 2
jfbu commented 7 years ago

! LaTeX Error: File `fontawesome.sty' not found.

Hi, you need to enhance your TeX install with the fontawesome package...

http://www.ctan.org/pkg/fontawesome

but installing a LaTeX font package is sometimes complicated. Best is to proceed via the package manager of your tex install.

... if I remember correctly (I am sort of offline now and can't check the thread on the web interface) I proposed an alternative for the missing "media control characters" and mentioned the best for blender docs would be to have icons like those of the graphics with all of them

btw, 1.6.1,(as far as I followed) does not fix the wrapfig issue. You would still need the \renewcommand{wrapfigure} etc...