search

hendricius / the-sourdough-framework

Open source book dedicated to helping you to make the best possible sourdough bread at home.
https://breadco.de/book
Creative Commons Attribution Share Alike 4.0 International
2.57k stars 128 forks source link

History of sourdough looks gibberish on website #273

Closed hendricius closed 7 months ago

hendricius commented 8 months ago

Seems like something is messing up the image generation during the website build: image

The flowcharts are not impacted, just the tikz images it seems.

cedounet commented 8 months ago

image

mon my phone (Firefox…)

how does epub look ?

hendricius commented 8 months ago

In the epub from https://the-bread-code.io/book.epub this is what I get using Apple books:

image
hendricius commented 8 months ago

And when built locally - this is how it looks:

image 
uname -a
Darwin MacBook-Pro-von-Hendrik.local 22.5.0 Darwin Kernel Version 22.5.0: Thu Jun  8 22:22:20 PDT 2023; root:xnu-8796.121.3~7/RELEASE_ARM64_T6000 arm64

/bin/sh --version
GNU bash, version 3.2.57(1)-release (arm64-apple-darwin22)
Copyright (C) 2007 Free Software Foundation, Inc.

latexmk --version
Latexmk, John Collins, 7 Jan. 2023. Version 4.79

lualatex --version
This is LuaHBTeX, Version 1.16.0 (TeX Live 2023)
Development id: 7567

Execute  'luahbtex --credits'  for credits and version details.

There is NO warranty. Redistribution of this software is covered by
the terms of the GNU General Public License, version 2 or (at your option)
any later version. For more information about these matters, see the file
named COPYING and the LuaTeX source.

LuaTeX is Copyright 2022 Taco Hoekwater and the LuaTeX Team.

tex4ebook --version
tex4ebook v0.3i

make4ht --version
make4ht version v0.3m

tidy -version
HTML Tidy for Mac OS X released on 31 October 2006 - Apple Inc. build 10443

lacheck --version
LaCheck (TeX Live) 1.30
$Id: lacheck.l 63190 2022-04-30 22:15:57Z karl $
License GPLv1+: GNU GPL version 1 or later <https://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Written by Kresten Krab Thorup and Per Abrahamsen.

chktex --version
ChkTeX v1.7.8 - Copyright 1995-96 Jens T. Berger Thielemann.
Compiled with POSIX extended regex support.

make --version
GNU Make 3.81
Copyright (C) 2006  Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.

This program built for i386-apple-darwin11.3.0

biber -version
biber version: 2.19

ruby --version
ruby 3.2.1 (2023-02-08 revision 31819e82c8) [arm64-darwin22]

convert --version
Version: ImageMagick 7.1.1-15 Q16-HDRI aarch64 21298 https://imagemagick.org
Copyright: (C) 1999 ImageMagick Studio LLC
License: https://imagemagick.org/script/license.php
Features: Cipher DPC HDRI Modules OpenMP(5.0)
Delegates (built-in): bzlib fontconfig freetype gslib heic jng jp2 jpeg jxl lcms lqr ltdl lzma openexr png ps raw tiff webp xml zlib
Compiler: gcc (4.2)

rsync --version
rsync  version 2.6.9  protocol version 29
Copyright (C) 1996-2006 by Andrew Tridgell, Wayne Davison, and others.
<http://rsync.samba.org/>
Capabilities: 64-bit files, socketpairs, hard links, symlinks, batchfiles,
              inplace, IPv6, 64-bit system inums, 64-bit internal inums

rsync comes with ABSOLUTELY NO WARRANTY.  This is free software, and you
are welcome to redistribute it under certain conditions.  See the GNU
hendricius commented 8 months ago

@cedounet also just realised the full website build is broken locally now 😅 . It seems my mac is very far behind already. I'll see if I can update it and get the build to work locally again.

hendricius commented 8 months ago

Probably also related to #184 and #183

cedounet commented 8 months ago

😡

  1. my laptop (MacOs ) look same as you for the gibberish... question mark on the phone(iOS) (firefox for both)
  2. I like to think i would have seen broken ebooks when I did the B&W version, but I cannot be 100% sure as i did not keep CI built ones
  3. it is broken on my local build... but as long as it worked on CI I did not care too much about ebooks.

C/

cedounet commented 8 months ago

https://github.com/hendricius/the-sourdough-framework/tree/225-ebook-situation now has the timeline in the MWE.

hendricius commented 7 months ago

Thanks! I wonder if generating pngs instead of svgs would somehow solve the whole situation. Couldn't find a way to do it yet. Back then I fixed this by including generated PDFs directly in the main book.

hendricius commented 7 months ago

@cedounet added an example in your branch showing that when we standalone generate the figures, they are properly displayed:

image
cedounet commented 7 months ago

Makes sense because we don’t run dvisvgm in that case. But it works on my Debian machine(s) so something is wrong with the docker image.. not sure what.

hendricius commented 7 months ago

Same with chemfig:

image

How about we just generate all the figures as separate documents instead? Personally I think the SVG steps are overrated in this case too, since the resulting code is gibberish anyways. It does not contain the proper text of the source SVGs. All the text is represented as lines. Then we also have one less dependency on the dvisvgm.

hendricius commented 7 months ago

Updated #225 with the example.

cedounet commented 7 months ago

Why not? You will need a macro so that it doesn’t do external when you build the pdf but yeah if flowcharts are not searchable and the font is fixed it makes sense.

hendricius commented 7 months ago

How would we go about doing that? Like I did in the past? Maybe a pre build step that builds the dependencies? Would however also increase compile times 😅

On Fri 17. Nov 2023 at 12:56, cedounet @.***> wrote:

Why not? You will need a macro so that it doesn’t do external pdf in that case but yeah if flowcharts are not searchable and the font is fixed it makes sense.

— Reply to this email directly, view it on GitHub https://github.com/hendricius/the-sourdough-framework/issues/273#issuecomment-1817223850, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGHNWZ367CORBXEKYHGBOLYE7TQJAVCNFSM6AAAAAA7B2H526VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMJXGIZDGOBVGA . You are receiving this because you authored the thread.Message ID: @.***>

cedounet commented 7 months ago

There already is support in the makefile https://github.com/hendricius/the-sourdough-framework/blob/6b2153b27ac79ce7b22337493ae495cbfbeac485/book/makefile#L71

that is largely untested.

All is needed is macro for input either the tex file when ran through latex or the pdf/png when ran through luaLaTeX or htlatex.

I still think this would better be fixed in the docker image, but right now it is broken so we ought to at least have decent ebooks and website.

cedounet commented 7 months ago

Would however also increase compile times 😅

Not if you write the makefile correctly, it can be run in parallel of other large tasks…. Should be barely noticeable if at all.

cedounet commented 7 months ago

we are getting mixed up between both bugs this one and https://github.com/hendricius/the-sourdough-framework/issues/279

anyway for this one this is the reason... if you look at the source code we have

<!--  l. 1  -->
<p class='noindent' id='-sourdough-microbiology-timeline'><img alt='JaFMAMJuJuASOND51MJuOONDDDDDDDnebapanluepcoe.4.1anccoeeeeeeerrygtvc51rettvccccccc 2 4 1 2 3 1 1 2 3 3 3bb5254479111illill5--ioio--F--Y------nnFiFFeFPDFSLiFrsiruaiaiiooyyrsirtsnsrsnnrsuueets btgtstgotrisarar mta mi des hdssa oca ooiaauuoPrrtrnnn brmuaigaei loesagstnritilaasag gnhteimiamnnuios iues oeddrnnrmnsse J ibsa txosa wlanotirocinc rndlatetdesifcatrhtttneia no a (dursp2 yaka3enlor:5adeft9s ai fu:r (n5ceug5hki)aaeryotes) ' src=
'[book0x.svg](view-source:file:///home/ced/Documents/the-sourdough-framework/book/static_website_html/book0x.svg)' /> <a id='x2-1001r1'></a></p>
<figcaption class='caption'><span class='id'>Figure&nbsp;1.1:&nbsp;</span><span class='content'>Timeline giberrish on website</span></figcaption>
<!--  tex4ht:label?: x2-1001r1   --></div>
</figure>

but book0x.svg does not exist...

If I do:

cp book-6fe744e7dd7ffbd43f85f9c5a0efb765.svg book0x.svg

tada !!!

image

cedounet commented 7 months ago

https://github.com/hendricius/the-sourdough-framework/pull/283 but I am not sure why that option was for in the first place and why this is the only one which breaks with it?

hendricius commented 7 months ago

🤩 sweet. I assume then this is also broken on the website. Fixing the copying should fix it. Did you get it to run on your Debian? I could docker your Debian as base image then.

On Sat 18. Nov 2023 at 10:57, cedounet @.***> wrote:

283 https://github.com/hendricius/the-sourdough-framework/pull/283

— Reply to this email directly, view it on GitHub https://github.com/hendricius/the-sourdough-framework/issues/273#issuecomment-1817640164, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGHNW5CESXKOY7WZRTNB7TYFEONBAVCNFSM6AAAAAA7B2H526VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMJXGY2DAMJWGQ . You are receiving this because you authored the thread.Message ID: @.***>

cedounet commented 7 months ago

I just disabled svgmhashes plugin... which I am not sure what is here for?

You added it so now :eyes: at what you have to say for your defense :imp: ?

hendricius commented 7 months ago

Delete, exterminate (in Dalek voice)

On Sat 18. Nov 2023 at 11:17, cedounet @.***> wrote:

I just disabled svgmhashes plugin... which I am not sure what is here for?

You added it so now 👀 at what you have to say for your defense 👿 ?

— Reply to this email directly, view it on GitHub https://github.com/hendricius/the-sourdough-framework/issues/273#issuecomment-1817645045, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGHNW2XEOLYSKODN7UZMRDYFEQXLAVCNFSM6AAAAAA7B2H526VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMJXGY2DKMBUGU . You are receiving this because you authored the thread.Message ID: @.***>

cedounet commented 7 months ago

I will risk a merge then... Alea jecta est.

cedounet commented 7 months ago

It keeps getting worst... I reverted my patch, but now it does not build anymore. not sure why it built before ?

023-11-18T21:56:48.4864559Z [ERROR]   mkutils: File book-bc9c43619ddaa460468696034970c72f.svg doesn't exist
2023-11-18T21:56:48.4880947Z cp: cannot stat 'book-bc9c43619ddaa460468696034970c72f.svg': No such file or directory
2023-11-18T21:56:48.4885158Z [INFO]    make4ht-lib: parse_lg process file: book-aa7904c8e4bfddd14ef106f2ee762d30.svg
20
hendricius commented 7 months ago

Seems like it is still looking for the hashes somehow? :-/

hendricius commented 7 months ago

Looks like it is still going into this code somehow: https://github.com/michal-h21/make4ht/blob/1d1e71166ad147627fac92cea6420ce70506297c/extensions/make4ht-ext-dvisvgm_hashes.lua#L17

cedounet commented 7 months ago

I "reverted the revert" we have no flowcharts in either website nor ebooks... but at least it builds.

I am afraid we reached the point of entshitification here... it now breaks just by looking at it funny.

It works on my local machine though so it points towards incompatibility between code and tools... so there is hope :sweat_smile:

cedounet commented 7 months ago

Logs are telling us something (not sure what)

2023-11-19T13:48:48.6765303Z Entering book.lg
2023-11-19T13:48:48.6766053Z System call: dvisvgm -n -p 2 --exact -c 1.4,1.4 -s book.idv > book0x.svg  
2023-11-19T13:48:48.6766605Z --- Warning --- System return: 65024
2023-11-19T13:48:48.6767134Z System call: dvisvgm -n -p 4 --exact -c 1.4,1.4 -s book.idv > book1x.svg  
2023-11-19T13:48:48.6767642Z --- Warning --- System return: 65024
2023-11-19T13:48:48.6768159Z System call: dvisvgm -n -p 6 --exact -c 1.4,1.4 -s book.idv > book2x.svg  
2023-11-19T13:48:48.6768675Z --- Warning --- System return: 65024
2023-11-19T13:48:48.6769200Z System call: dvisvgm -n -p 8 --exact -c 1.4,1.4 -s book.idv > book3x.svg  
2023-11-19T13:48:48.6769700Z --- Warning --- System return: 65024
[...]
2023-11-19T13:48:48.6787802Z System call: dvisvgm -n -p 43 --exact -c 1.4,1.4 -s book.idv > book21x.svg  
2023-11-19T13:48:48.6788442Z --- Warning --- System return: 65024
2023-11-19T13:48:48.6788955Z System call: dvisvgm -n -p 44 --exact -c 1.4,1.4 -s book.idv > book22x.svg  
2023-11-19T13:48:48.6789627Z --- Warning --- System return: 65024
2023-11-19T13:48:48.6789928Z Entering book.css
2023-11-19T13:48:48.6790181Z Entering book.tmp
2023-11-19T13:48:48.6790340Z

Later on:

2023-11-19T13:48:50.1238503Z [INFO]    make4ht-lib: parse_lg process file: book0x.svg
2023-11-19T13:48:50.1239461Z [INFO]    make4ht: outdir: static_website_html/
2023-11-19T13:48:50.1240063Z [INFO]    mkutils: Copy: cp "book0x.svg" "static_website_html"
2023-11-19T13:48:50.1260592Z [INFO]    make4ht-lib: parse_lg process file: book1x.svg
2023-11-19T13:48:50.1261708Z [INFO]    make4ht: outdir: static_website_html/
2023-11-19T13:48:50.1262741Z [INFO]    mkutils: Copy: cp "book1x.svg" "static_website_html"
2023-11-19T13:48:50.1281800Z [INFO]    make4ht-lib: parse_lg process file: book2x.svg
[ ... ]
2023-11-19T13:48:50.1693223Z [INFO]    make4ht-lib: parse_lg process file: book22x.svg
2023-11-19T13:48:50.1694238Z [INFO]    make4ht: outdir: static_website_html/
2023-11-19T13:48:50.1695292Z [INFO]    mkutils: Copy: cp "book22x.svg" "static_website_html"
2023-11-19T13:48:50.1712244Z [STATUS]  make4ht: Conversion finished
2023-11-19T13:48:50.1855337Z cp static_website_html/book.html static_website_html/index.html
2023-11-19T13:48:50.1904511Z cd ../website && ruby modify_build.rb

upload is looking great though... but somehow the svg files are not being uploaded (because empty ?)

https://***.s3.amazonaws.com/static_html_root/bg.jpg,https://***.s3.amazonaws.com/static_html_root/bibname.html,https://***.s3.amazonaws.com/static_html_root/book.css,https://***.s3.amazonaws.com/static_html_root/book.html,https://***.s3.amazonaws.com/static_html_root/book0x.svg,https://***.s3.amazonaws.com/static_html_root/book10x.svg,https://***.s3.amazonaws.com/static_html_root/book11x.svg,https://***.s3.amazonaws.com/static_html_root/book12x.svg,https://***.s3.amazonaws.com/static_html_root/book13x.svg,https://***.s3.amazonaws.com/static_html_root/book14x.svg,https://***.s3.amazonaws.com/static_html_root/book15x.svg,https://***.s3.amazonaws.com/static_html_root/book16x.svg,https://***.s3.amazonaws.com/static_html_root/book17x.svg,https://***.s3.amazonaws.com/static_html_root/book18x.svg,https://***.s3.amazonaws.com/static_html_root/book1x.svg,https://***.s3.amazonaws.com/static_html_root/book20x.svg,https://***.s3.amazonaws.com/static_html_root/book21x.svg,https://***.s3.amazonaws.com/static_html_root/book22x.svg,https://***.s3.amazonaws.com/static_html_root/book23x.svg,https://***.s3.amazonaws.com/static_html_root/book2x.svg,https://***.s3.amazonaws.com/static_html_root/book3x.svg,https://***.s3.amazonaws.com/static_html_root/book4x.svg,https://***.s3.amazonaws.com/static_html_root/book5x.svg,https://***.s3.amazonaws.com/static_html_root/book6x.svg,https://***.s3.amazonaws.com/static_html_root/book7x.svg,https://***.s3.amazonaws.com/static_html_root/book8x.svg,https://***.s3.amazonaws.com/static_html_root/book9x.svg,https://***.s3.amazonaws.com/static_html_root/bread.jpg,https://***.s3.amazonaws.com/static_html_root/close.svg,https://***.s3.amazonaws.com
[...]
cedounet commented 7 months ago

When it works (on my local machine i get)

t4ht.c (2018-07-04-14:25 kpathsea)
t4ht book.dvi 
(/usr/share/texlive/texmf-dist/tex4ht/base/unix/tex4ht.env)
Entering book.lg
System call: dvisvgm -n -p 2 --exact -c 1.4,1.4 -s book.idv > book0x.svg  
System return: 0
System call: dvisvgm -n -p 4 --exact -c 1.4,1.4 -s book.idv > book1x.svg  
System return: 0
cedounet commented 7 months ago

https://github.com/hendricius/the-sourdough-framework/pull/287#issuecomment-1825348918