search

sile-typesetter / sile

The SILE Typesetter — Simon’s Improved Layout Engine
https://sile-typesetter.org
MIT License
1.62k stars 97 forks source link

Support Math — somehow (general discussion of approaches and status) #220

Closed akavel closed 2 years ago

akavel commented 8 years ago

I'm aware MathJax is listed on the wiki as "up-for-grabs", but I just wanted to start by asking: have you maybe tried pushing some stuff from the MathJax's unpacked/ subdir through some js-to-lua translator (or other)?

I'm on Windows, so personally, before I maybe start trying to contribute, I have to first manage to compile the stuff (yes, I've seen the issue tracking this)...

(By the way, I got here by accidentally watching the 2015 FOSDEM video of yours; very entertaining and interesting :) although I've apparently already +1'ed your project at some point, only this video made me aware of many things about it (both feature demos, architecture, and how it relates and improves over *TeX) and thus more interested.)

edit: A few years later, after some prototyping and experiments, I'm expanding this initial post with more detailed current status of the project. The path I've eventually picked to try and implement this is based on ASCIIMath, not MathJax.

Current TODO tasks

Roughly in priority/painfulness order. I'd be especially grateful if someone could help with the ones marked "HELP NEEDED", I don't know how to proceed with those unfortunately :(. Jumping in to help with any task is enthusiastically encouraged!

Current example result

akavel commented 7 years ago

Notes from the first failed attempt: I tried translating KaTeX with castl. The translation "succeeded" (using KaTeX v0.7.1 from git and castl 1.2.4), with commands roughly as below:

$ cd katex
$ git checkout v0.7.1
$ for f in katex.js src/*.js; do castl $f -o --debug --node; done

However, trying to run the following Lua script afterwards with Lua 5.2:

local katex = assert(loadfile 'katex.js.lua')()

failed with a Lua error: [...] too many C levels (limit is 200) [...], which seems to mean there's too deep nesting in the code.

(Reported on castl project as https://github.com/PaulBernier/castl/issues/18. TODO: try some next translator, maybe colony.)

akavel commented 7 years ago

After some more looking around, I want to try my luck with ASCIIMath. It seems small enough, that even in case js2lua converters won't help, a manual port seems feasible. It does not implement Full Parity With TeX™; but as we all know too well I think, TeX is Turing-complete, and full parity is not necessarily even desirable. Anyway, even if I succeed, in Future™ someone can always just ditch the ASCIIMath and/or incrementally replace it with something More Powerful™ or whatever.

Now, with that said, there's one thing I now realized I'm not really clear on. Namely, what would be the best format to output from a math module for SILE to consume? ASCIIMath I think produces MathML; KaTeX claims to emit HTML+CSS; MathJaX is I believe super-duper-dynamic-adaptive-autodetect and can produce MathML, SVG, possibly images, and I don't even know what else. What would SILE like best? Imagine we're free of constraints; would it be PDF? SVG? Lua rendering code? glyphs + coords + scales? PostScript? glues penalties and some other weird stuff like this? (not even sure if it's there for math). On the other hand: is there/are there some simple format(s) (hopefully, vector) that SILE could reasonably accept for a reasonable first prototype/PoC?

As a side note for future: in case of failure, I think one more direction to research would be troff's eqn processor.

suhr commented 7 years ago

I want to try my luck with ASCIIMath.

It doesn't have \cfact though, not even talking about commutative diagrams.

Anyway, good luck. It is hard to live without even basic math.

alerque commented 7 years ago

PDF would seem to make the most sense. That way the output can be directly embeded without much chance for scale or other interpretive errors.

An SVG route would be an entirely different pipeline and introduce some other pain points, but if there was a Math→SVG processor we might be able to bolt that on entirely outside of Lua.

akavel commented 7 years ago

@suhr Did you ever try groff's eqn? Do you think it's currently more powerful or less, compared to ASCIIMath?

If PDF sounds best, then I think eqn sounds like more bang for the buck for a first shot. With various caveats, first among them GPL I suppose (or alternatively GPL-incompatible Lucene license for Plan9's troff+eqn; not sure about PDF support there too). Or do you know by chance of someone already trying to integrate eqn with SILE and stumbling upon some problems?

EDIT: Hmm; it seems groff emits PDF by means of a sub-tool named "gropdf", which seems to be written in Perl. I'm not really convinced requiring Perl is a good match for SILE. On the other hand, for the time being it could be treated as a "plugin" to SILE, and as such not part of it, so Perl could be acceptable then.

Still, now it's not clear to me how would one approach "merging" PDF into SILE as input. It seems to me that PDF describes whole pages (or am I not right?). But equations are usually just small fragments on a page. Can SILE cut fragments of PDFs? Do you maybe know of some already preexisting mechanisms for something like that, of which I'm not aware yet? @alerque Do you know if there's maybe a PDF parser or something in SILE?

EDIT 2: @alerque Hm, but there seems to be already some machinery for SVG in SILE, no?

akavel commented 7 years ago

Prototype translation of ASCIIMathML.js to Lua is done and tested. I used the very cool castl translator. Also, given that it's not my first port, I must shout out, that comprehensive end-to-end tests do really boost porting speed awesomely!

TODO: test if svgmath will work reasonably well to translate the resulting MathML to SVG. It seems quite a small and readable MIT codebase, so should be feasible to hand-port to Lua if it's working well. I imagine it should be possible to migrate the SVG emitter to a PDF-or-whatever emitter more fitting for SILE later, should the PoC prove successful.

alerque commented 7 years ago

@akavel SILE already has a way to embed PDFs inside PDFs. Use the \img[src=file.pdf] function passing it a PDF as the input image. This is easy and reliable because the PDF doesn't have to be edited in any way, it can basically just be wrapped up and packaged inside the final output PDF with a location where it should be included.

Yes there is also rudimentary SVG support, but this isn't really up to snuff yet and involves being able to render whatever is in the SVG and convert it to a post script representation that can be embedded in the PDF. This is nowhere near as reliable as just nesting PDF content. It works alright for some things, but doesn't understand a lot of SVG instructions yet.

akavel commented 7 years ago

@alerque Do you know if it's possible to create a PDF with "tiny paper size", so that it would have size of exactly the bounding box of an equation (i.e. 2cm x 1cm, or something like that)? Or does \img[src=...] crop the contents to extract only the "black" contents, ignoring paper size?

alerque commented 7 years ago

Embeding a PDF as an image does nothing in the way of cropping or finding contents, it just uses the bounding box and inserts that at the requested overall width, hence scaling the source's paper size to the embeded size.

Obviously if you're trying to bold on some other render you might end up with an awkward mating like this, but ideally of course a native Lua parser would transform the formula to native drawables that could be passed to the output engine (which at this point is basically always the PDF output engine but keep in mind that isn't supported to be a hard coded limitation).

akavel commented 7 years ago

Ok, did a quick test with groff and indeed this direction seems to (kinda) work already? Main initial caveats:

For anyone interested, here comes a quick & rough transcript of my session. Note I'm using Nix package manager:

$ nix-channel --list
nixos http://nixos.org/channels/nixos-17.03

$ nix-shell -p sile groff

[nix-shell:~]$ cat try3.troff    # NOTE: encoded in UTF-8, as expected for `-k` below IIUC
.PRINTSTYLE TYPESET
.PAGE 1i 2i .05i .05i .05i .05i
.FAMILY T \" Times Roman
.START
.EQ
∇²ϕ = 0
.EN

[nix-shell:~]$ cat eqn-pdf.sil
\begin[class=book,papersize=129mm x 198mm]{document}
\script[src=packages/image]
...
Foo bar \img[src=try3.pdf] baz.
...
\end{document}

[nix-shell:~]$ pdfmom -e -k try3.troff > try3.pdf
[mom]: Your chosen bottom margin for running text is too close to the footer margin.
       No footers or bottom-of-page page numbers will be printed.
       Please reset B_MARGIN or FOOTER_MARGIN to allow enough space.
       If no footers or bottom-of-page page numbers are required,
       invoke .FOOTER_MARGIN 0 before .START
troff: environment stack underflow

[nix-shell:~]$ sile eqn-pdf.sil
This is SILE 0.9.4
</vm-share/eqn-pdf.sil>

! Font 'Gentium' not available, falling back to 'DejaVu Sans'
[1] [2]

screenshot

akavel commented 7 years ago

For the record, a "more advanced" groff equation worked as well:

sum from i=0 to {inf} x sup i { sqrt {alpha + beta} } over {sin (x)}

screenshot

alerque commented 7 years ago

how to get information about the baseline in the original PDF?

What do you mean by "original" here? The baseline in the SILE PDF should be straight forward enough. The trick is going to be leaning where the baseline of an equation PDF is in the event of an inline equation such as the ones in your examples. Once you do learn that you can offset the image in SILE using \lower[height=Y]{}. Combined with the right leading mechanism (fit-glyph?) this should allow lines to move around to accommodate formulas.

akavel commented 7 years ago

(@alerque Yes, I meant exactly what you mean by "learning where the baseline of an equation PDF is".)

Personally, at this point I still plan to pursue the path of: ASCIIMath → MathML → SVG → SILE. I believe it's more promising in the long run. That said, groff (specifically, eqn + pic) seems totally more useful in the short run, especially given that their language is much more powerful and expressive than ASCIIMath (I imagine commutative diagrams should be totally doable with eqn or eqn+pic). But as far as my current plans go, I'm leaving perfecting of the groff path as an... cue ominous thunder... "excercise for the reader".

akavel commented 7 years ago

Cool, SVGMath seems to work! Below is a quick screenshot of: 

sum_(i=0)^oo x^i (sqrt(alpha + beta))/sin(x)

after passing through:

  asciimath.lua —(MathML)→ math2svg.py —(SVG)→ Firefox

(Note: I had to add <html><head></head><body> boilerplate + MathML namespace on the <math> element.)

The screenshot:

screenshot

It complained about some missing metrics and whatnot, so I suspect this could look better with some caring hand. (Still positively surprised super much that it actually worked, and so well so soon!)

simoncozens commented 7 years ago

I don't have much to add to this but I'm really happy it's being worked on. The baseline/leading issues will be quite easy to fix when then time comes.

akavel commented 7 years ago

@simoncozens How about the bounding box? If groff emitted a big, white page with a (comparably) small equation in some quasi-random place on it, would you find it easy to extract just the "dirty" area? If so, it would make the task much easier on the groff side, I think — it should be just deleting all page decorations, and voilà, the thing's done?

alerque commented 7 years ago

@akavel I don't think SILE should be responsible for parsing PDF data. It really isn't setup for that. It knows how to write it and it can take a chunk of PDF data and embed it somewhere, but parsing through the PDF data and doing a virtual rendering to find the bounds of actual content in it is more than a little outside the purview of the typesetting engine.

That being said it may not have to be groff that does the work. There are other tools for examining and modifying PDFs. Given that groff is already an external dependency it wouldn't be entirely out of the question to include another tool in the pipeline. However I'm concerned that even if we found such a tool the result would still be a hack. What happens when the formula is so large that it overflows the random spacious page size you assign? Is there really no way to have groff match the CropBox value to the resulting content it generates?

akavel commented 7 years ago

(@alerque Sorry, but can you please remove my "assignee" status from this issue? I don't want this to start feeling like work to me; I'm doing this just out of fun and because I want to help and like this project. A subjectively perceived connotation of "responsibility" and "obligation" coming from the assignment status unfortunately heavily impacts on my fun and is counter-productive for me working on this. Also, I don't want to make anyone feel like "this issue is occupied so I won't waste my time contributing to it"; I'm totally ok with someone else even finishing it before me and closing the issue (I don't lack things to do). Finally, I never promised I'll complete this, so I don't want anybody to think so by accident. I just fancy to have a stupid plan to try my luck at this, with 100% undefined schedule. As also shown by a ~1.5yr gap between my first and second comment in this thread. TIA! I'd do this myself, but can't find how.)

alerque commented 7 years ago

@akavel Of course. I didn't intend it in the sense of responsibility at all only that because you've got code kicking around in this department it would make sense to coordinate with you before launching into this one.

Speaking of which do you have a branch somewhere with your work in progress? If it was posted and linked from here that would give somebody else the chance to come along and extend or clean it up.

akavel commented 7 years ago

@alerque Thanks!

Re: branch link — sure, good idea. Summarizing the currently explored approach:

Recapping possible alternatives I thought of, with some pros & cons I'm aware of:

All/some of the above alternatives could potentially peacefully coexist so that they can be played to their relative strengths.

akavel commented 7 years ago

The Lua port of SVGMath is now giving results, and very similar ones to what the original Python version gave. On the available test data/examples (*.mml), some have minor/medium spacing issues and there are some non-italic π characters (fixed). Also, 2 examples fail, but they fail in Python too (test18 and test20).

Example Lua results (testdata/*.out.svg):

Reference Python results (testdata/*.svg):

At this point I'd like to ask for help integrating this as an experimental plugin/feature into SILE. Can you possibly give me some guidance on how to start? I'd be very grateful for hints, would considerably speed this up for me. Please note, that I have no practical knowledge/experience of SILE, other than reading the manual.

akavel commented 7 years ago

Uh, oh; do I see correctly that SVG import seems not yet working as of current state of the SILE repo?

simoncozens commented 7 years ago

Try it... We use the nanosvg parser to render SVG to a set of paths, and then there's code in src/svg.c that turns it into PDF drawing operators. There may be incompleteness in either of these steps, but if there is, file a bug and I'll work on adding support.

Look in packages/svg.lua for an SVG package which loads in a .svg file. If you have a string containing SVG instead of a file, you will need to add another command to svg.lua based on \include-svg-file.

akavel commented 7 years ago

@simoncozens Ok, thanks. I've reported #468. My initial suspicion is that maybe nanosvg ignores SVG's <text> elements... :/

EDIT: Yep, seems so :/ no "text" in the relevant block in nanosvg.h, or anywhere else in the file as a matter of fact. Funk.

simoncozens commented 7 years ago

OK. Well, I would reconsider the use of nanosvg, if it's not actually going to help us render real-world SVGs. If we want to do this right (which we do) then I guess we need to be looking at librsvg or cairo or something.

akavel commented 7 years ago

Alternatively, we could try extending nanosvg to support <text>? Or is it too much of a stretch? If that's the only missing thing (? no idea), maybe it wouldn't be very stupid?

(EDIT: I'm continuing discussion of this subtopic in #468.)

akavel commented 7 years ago

Tried pushing the SVG through inkscape -z -f test13.svg --export-pdf test13.pdf, then doing \img[src=test13.pdf]. Seems to "work", though font sizes in the equation are totally inadequate w.r.t. surrounding text (kinda 2x bigger; truly displaymath...)

akavel commented 7 years ago

svgmath seems to be producing only a limited set of SVG primitives, namely: path, rect, line, <g translate=...>, text, circle. I think I could just parse this "SVG" by hand in Lua, reserve a properly sized hbox, then try emitting appropriate SILE/pdf primitives, probably on outputYourself. (Where) could I find functions to emit appropriate primitives in SILE? or would it be in libtexpdf?

simoncozens commented 7 years ago

This should do what you want:

local pdf = require("justenoughlibtexpdf")
...
pdf.add_content("q")
pdf.add_content(stuff)
pdf.add_content("Q")
akavel commented 6 years ago

Status update: the main integration step, or leap, is now done! 😃

The following screenshot:

was rendered from the following sample.sil file:

\begin{document}
Sample inital text.

Another line before.

\script[src=mathsvg]

Sample final text.

Sample more more more text.

\asciimath{E=mc^2}

----

Some examples from ASCIIMath homepage of Peter Jipsen:

- complex subscripts are bracketed, displayed under lim

. \asciimath{d/dxf(x)=lim_(h->0)(f(x+h)-f(x))/h}
\bigskip

- f^((n))(a) must be bracketed, else the numerator is only a

. \asciimath{f(x)=sum_(n=0)^oo(f^((n))(a))/(n!)(x-a)^n}
\bigskip

% FIXME(akavel): error
%- piecewise defined functions are based on matrix notation
%
%. \asciimath{x/x=\{(1,if x!=0),(text\{undefined\},if x=0):\}}
%\bigskip

% FIXME(akavel): error
%- angle brackets and invisible brackets
%
%. \asciimath{<< a,b >> and \{:(x,y),(u,v):\}}
%\bigskip

- symbols can be stacked

. \asciimath{stackrel"def"= or \\stackrel\{\\Delta\}\{=\}" "("or ":=)}
\bigskip

- prescripts simulated by subsuperscripts

. \asciimath{\{::\}_(\\ 92)^238U}
\bigskip

\end{document}

The full repo at its current state is at: https://github.com/akavel/silemath. If you try to use it, you may need to edit paths to TTF fonts on your system, currently hardcoded in svgmath/config.lua, l.20-23. Fonts to TTF files should now be auto-detected from SILE (after recent commit in silemath). It should hopefully now work out of the box — just clone the https://github.com/akavel/silemath repo, and run sile sample.sil — please report here if it worked for you or not!

Obviously, there's still a lot of work to be done. But it already starts to be somewhat usable! And you're now oficially and formally encouraged to jump in and try to help if this work matters to you! :) The main obstacles are now crushed, what's needed now is to patiently, step by step smooth the many rough edges, one by one, towards ultimate mathy victory!

akavel commented 6 years ago

Current TODO tasks

EDIT: moved the list of tasks into the first post in this thread/issue (at the top of this page), as suggested by @alerque

alerque commented 6 years ago

Pro-tip: if you edit that TODO list into the first post in this thread Github has some features showing overall issue progress. Also especially for the held needed items I would recommend splitting them out into separate issues so they can be tackled and tracked independently, and just cross-linking the issue in the todo list for this issue. And no, I wouldn't recommend using a different repo for this since the goal is to have it all integrated into the sile core.

When I get get a chance I'll definitely have a look and see if there are any items I can help with.

alerque commented 6 years ago

Re: not having to escape { and similar characters in math functions: I already did most of the work needed on the LPEG parser to make this happen and it's in SILE core. I can help add handling for the math tags so they get the same treatment as Lua code now does. There will be one caveat for the math use case and that is that the inline version of tags need balanced numbers of opening and closing braces if it stands a chance of finding the right closing mark for the tag. Using a "math mode" syntax ala LaTeX would be one way to alleviate that gotcha situation.

akavel commented 6 years ago

@alerque Regarding parsing, personally I was thinking it might be nice if this parser feature you mention (i.e. some kind of "plugging in with new syntax into the parser") could be maybe available to any modules. My argument is such, that ASCIIMath is a controversial choice, and people might want to like to use some other math engine in future (e.g. port of the classical LaTeX engine?), so there should ideally be some possibility for swapping them in the middle of a document. Also even ASCIIMath may incorporate some changes with time, resulting in different rendering of some expressions, so it would be nice to be able to version it (and pick a particular version of ASCIIMath in parts of a doc). On the other hand, there's a question if giving too much parser influence power to plugins could have bad consequences (see TeX...). So maybe just $ ... $ and \[ ... \] could be the only special cases, and math engines would connect to some API hook in SILE to be able to capture/handle them? Sounds like the safer way maybe for start...

I.e. some kind of:

\mathengine[asciimath]
Foo bar $ ...this is parsed as ASCIIMath... $ baz bing.
\mathengine[latexmath]
Boo far $ ...this is parsed as LaTeX math... $ faz fing.
akavel commented 6 years ago

Fixed another sub-issue; current example result:

[...]

edit: another fix brings even bigger improvement! see:

ctrlcctrlv commented 3 years ago

I've read this issue and #578 (+856 SLOC!), and really don't understand why so much ... I don't know how to put it, perhaps, work? ... is being done on this, yet nothing in core since 2015.

578 seems basically abandoned by @harrysummer, which is a shame, given how much time has obviously been put into it. This idea as well, doesn't seem to be going anywhere.

Also, this issue is misnamed. @akavel gave up on MathJax, instead opting for ASCIIMath.

So, I'd like to propose recalibrating. Early on here, KaTeX and MathJax were jettisoned because they couldn't be converted to Lua. I feel like nobody asked the obvious question: so what?

Node.js is available in most distros, making it an optional build dependency to me, quite honestly, seems like no big deal at all. After all, if lilypond (#435) and TiKZ (#437) are being considered, then I really don't think "can't be done in pure Lua" is a logical blocker.

So, my proposed solution would be, a mathjax.lua to call out to node mathjax_wrapper.js, which would actually import MathJax and set up command line options for controlling its output, outputting SVG onto stdout. Then you can just put \mathjax{} in your document using the #947 hack, and it uses the svg package to draw the math into your SILE document.

Now yes, this is less elegant, sure. But it has a key advantage of not being an enormous amount of work, meaning, it's actually likely to be finished by someone without superhuman stamina :-)

akavel commented 3 years ago

@ctrlcctrlv Those are the highs and lows of open-source 🙂 you are always welcome to try your idea and see if it works, and then try to contribute it back and see if the owner is OK with accepting it to the main codebase 🙂 and once you try, you might start to understand and grudgingly appreciate why this is as is 😆 and some day someone maybe will have enough patience and drive to push some version of it all the way to a successful conclusion.

ctrlcctrlv commented 3 years ago

@akavel Well sure, that's why I wrote the above, to spur discussion on what to do about this problem. 😄

It's not a criticism of you, rather, I'm wondering, given that two highly passionate developers burned out on this project approaching it one way, whether or not it makes sense to come at the problem a different way.

alerque commented 3 years ago

I don't see any conflict with perusing multiple approaches and if either/both become usable then great. A more native approach using SILE to do the rendering of individual nodes would bring a lot of advantages, but is clearly the longer road to travel. A more modular approach of embedding a different language in the content of a tag and shipping it off to an external dependency to render an SVG and embedding the result is something we've shown to be workable too. The former does make more sense to attach ASCIIMath, MathML, and/or TeX math compatibility, the latter makes more sense for MathJAX. Both could happily survive in parallel and serve different needs.

For the latter approach one SVG blob per command would be a good starting point. Bonus points would be an option that returned the resulting graphic broken up into lines that SILE could then use as frame break points.

Lets split up tracking issues though.

alerque commented 2 years ago

I haven't re-read through this issue and the todo list yet, but it needs an overhaul now that #1103 has been merged.

akavel commented 2 years ago

I just noticed the merge of #1103, I'm super happy and excited about it!!!!!!

From quickly glancing #1103, do I understand correctly it currently provides a MathML parser & translator, + a basic Tex-like parser? Do we know how much of MathML is supported, are there some/any limitations in the parser & translator?

If the MathML support is complete enough, I feel it should be pretty easy for me to cut this part out of my PR my module, and keep just the ASCIIMath->MathML translation part, thus maybe adding a new command \asciimath for those who might prefer that syntax. I'm really curious how much of it might be working with the present MathML renderer!!!! 🤩

OlivierNicole commented 2 years ago

I'm enthusiastic too!

The supported MathML tags are summarized in the checklist in #1103. Basic tags of the presentation markup are supported, but some are not supported (notably mtext), as well as some options (such as styling). No semantic tag is supported (but I believe that this part of MathML is much less used).

It should be pretty simple to define a command with a custom parser. You can look at the definition of the math and mathml commands in packages/math/typesetter.lua: the only difference between them is that math first converts its input into MathML.

akavel commented 2 years ago

Hm; @alerque is there some Windows nightly build I could download to try and work on this on MSWin? My main machine is Windows, so it'd be much easier for me this way; but if it's not possible, I'd be grateful if you could let me know and I'll try to undust my 2ndary NixOS laptop and see what I can do there...

alerque commented 2 years ago

@akavel The state of our Windows builds are in shambles. The CI runner needs somebody with expertise and interest in fixing it, it's been broken for a while. Nothing too major has changed architecturally since it was working, but I'm sure the Cmake definitions are well behind. I know there are quite a few people waiting for Windows builds and an installer, it is just so far outside the realm of my own production needs (and expertise) that it's hard for me to invest time it its upkeep.

I know NixOS works, in fact 0.11.1 is packaged there and I just used it for something (I'm looking at using NixOS to build the examples for the SILE website).

akavel commented 2 years ago

@alerque got it, thanks! IIUC I'll probably need to tweak the Nix expression to fetch from current master, not from 0.11.1, so that I could have the features that were needed in core for the mathml package to work, right?

alerque commented 2 years ago

@akavel True. Actually I would be really interested to know how to build a HEAD package for Nix too, but we should probably ask that on their issue tracker or look into other venues for how such a thing is done. Arch Linux has a sile-git package in the AUR which you can get running pretty easily. I can post a Dockerfile to get that running if that would help you.

Math will land in a tagged release soon, really the only hold up is wanting to have examples fixed on the website before we attract attention with the math release announcement.

OlivierNicole commented 2 years ago

I've been trying to compile HEAD again on Arch but ran into problems, as I posted on gitter.im.

akavel commented 2 years ago

@alerque as to building HEAD package first steps towards building HEAD package, what I did and seemed to work was:

  1. copy the default.nix from nixpkgs into the local clone of the SILE git repo (found the path in nixpkgs thanks to "The NixOS Hound")

  2. tweak the Nix expression's args list to work outside nixpkgs, by making it by default pull packages from <nixpkgs>:

    { nixpkgs ? import <nixpkgs> {}
, lib ? nixpkgs.lib
, stdenv ? nixpkgs.stdenv
, darwin ? nixpkgs.darwin
, fetchurl ? nixpkgs.fetchurl
, makeWrapper ? nixpkgs.makeWrapper
, pkg-config ? nixpkgs.pkg-config
, poppler_utils ? nixpkgs.poppler_utils
, gitMinimal ? nixpkgs.gitMinimal
, harfbuzz ? nixpkgs.harfbuzz
, icu ? nixpkgs.icu
, fontconfig ? nixpkgs.fontconfig
, lua ? nixpkgs.lua
, libiconv ? nixpkgs.libiconv
, makeFontsConf ? nixpkgs.makeFontsConf
, gentium ? nixpkgs.gentium
}:

  3. $ nix-build - worked for me!

  4. $ ./result/bin/sile

    SILE v0.11.1 (Lua 5.2)
>

Haven't tried to do anything more advanced with it yet.

(Please note I'm kinda early-intermediate with Nix, definitely not an expert.)

EDIT: Uhh, this will actually probably still build the 0.11.1 version fetched from github. Need to do some tweaks still to load from local code; sorry, as I said above, I'm still learning Nix :/

I think the next thing to do is to change the src attribute to something like src = ./.; but I'm not 100% sure, and this starts to give me some errors; can't work on it now, will see when I can try pushing this further.

suhr commented 2 years ago

Don't forget to update the hash, otherwise nix might pull a wrong source.

suhr commented 2 years ago

Anyway, you want this if you want to build from master:

  src = fetchFromGitHub {
    owner = "sile-typesetter";
    repo = "sile";
    rev = "190683a82eabff2aeccc29fc36d3c1d610b70a87";
    sha256 = "sha256-vStmwsdqwD7yIdNYCk3Vu853fXhU661xFuRmzsa3N2M=";
  };