Closed PHPirates closed 5 years ago
Hey @PHPirates, sorry for the belated reply! I've been busy with some exams and some travel.
You're quite right in that they're internal minted commands. As part of this dissertation work, I wrote a style which I'm guessing you already found over at https://github.com/lol768/id7-pygments-style/
https://github.com/gpoore/minted/issues/230 is indeed the issue motivating the TeX I ended up writing. The basic problem is:
My code samples are not just code, they use a bunch of LaTeX inside the code sample to highlight things. For example, here's a SQL injection vulnerability with the vulnerable part of the code highlighted:
The red box is a \colorbox
and an \faIcon
within the code sample. Unfortunately, only some lexers provided by Pygments correctly handle this. There is a long-standing bug in Pygments that was raised 4 years ago which means that I can't use escapeinside
for languages like C# or Java. Instead, I manually run Minted on just the bare source code, copy the generated output (from _minted-document
) and then manually paste it into the LaTeX document already-highlighted. From here, I can then add my annotations and other custom code.
But! This doesn't "just work". The problem is that Pygments lazy-loads the \PYG<x>
commands. When you do what I just did and try and paste the generated output and remove the original minted
environment, nothing is loaded! Hence, the compile fails because it doesn't understand any of the commands.
The solution is to include the code twice. Once with no annotations, in a way that's hidden from the final document (imagine display: none
, but for LaTeX, if you're familiar with CSS) in order to trick Minted into thinking the commands are necessary and loading them. And then another time, which appears in the final document, with all the necessary annotations and modifications.
The custom id7
Pygments style (which is named after the university's 7th brand iteration) complicates things a bit further, but ensures that the code samples look pretty and are consistent with the rest of the document. You need to have the style installed for any of this to work, since we use \usemintedstyle{idseven}
to load it in the preamble.
The reason your example doesn't work is because the \PYGidseven{k+kd}
definition isn't loaded. In Pygments land, k
is a keyword and kd
is a Keyword.Declaration
. So we can make your minimal example compile by doing something like this:
\documentclass{article}
\usepackage{minted}
\usemintedstyle{idseven}
% Compile using XeLaTeX with -shell-escape
\begin{document}
\begin{minted}{java}
public class MyClass {
// exists solely to pre-load the k+kd definition
}
\end{minted}
\begin{Verbatim}[commandchars=\\\{\}]
\PYGidseven{k+kd}{public}
\end{Verbatim}
\end{document}
In this example, we first have minted compile a simple java class. The public
keyword in this code sample is enough for minted to realise that it needs to load the k+kd
style, and the subsequent Verbatim
environment works just fine.
In addition to this fun hackery, you'll notice that some code samples are highlighted using an environment like this:
\begin{minted}{rrtlex.py:RrtLexer -x}
% ...
\end{minted}
This one is hilarious - minted doesn't support custom lexers. So, we take advantage of a command injection 'vulnerability' in the minted LaTeX package to inject an additional argument into the pygmentize
command that's invoked to do the actual nitty-gritty highlighting work. This allows us to specify our own lexer file to use, which was written specifically for the dissertation.
[Edit] I hope it's no problem for your supervisor that I am trying to compile it... I won't distribute the result, of course.
@mbg - for context, @PHPirates contributes to the LaTeX IntelliJ plugin I used to write most of the report (which you should totally check out one day :stuck_out_tongue:). I added him here so we could try and reproduce some bugs I'd been experiencing.
From my perspective, I don't see a problem with you doing what you want with the report, @PHPirates. I intend on open sourcing all of this work later on.
I see no problem with this!
No worries about any delay, I'm not in any hurry, and your exams are more important!
Oh, wow. That's some advanced minted use :smile: Thanks for the in-depth explanation, I see the general idea of what you're doing now, I had not realised that the \PYG
commands were right there in the generated output, but I see that when I copy them back they indeed start working when the original minted environment is also there (hidden or not).
I guess this would be useful information if you want to open source things :)
But, I'm still not sure I am using the custom style properly. I just installed it to my system-wide Python, but I get the following error when trying to run the example file you give:
*** Error while highlighting:
AttributeError: module 'id7' has no attribute 'IDSevenStyle'
(file "/usr/lib/python3.7/site-packages/id7-1.1-py3.7.egg/id7/__init__.py", line 3, in <module>)
*** If this is a bug you want to report, please rerun with -v.
So it seems to my it does find id7-1.1-py3.7.egg
but somehow it's not doing what it expects. And the pygments output still contains only \PYG
and no \PYGidseven
.
Yeah, collaborator of TeXiFy and author of https://github.com/PHPirates/travis-ci-latex-pdf ... not sure as which of these persons I am interested in your dissertation. But it's proving to be one of the better test files I've seen so far! (testing using avenir next and ubuntu mono on Travis atm, I think I should be able to make that work somehow)
https://github.com/PHPirates/travis-ci-latex-pdf - ah, this is cool! I wanted CI -> PDF to work for this project. I did try using Tectonic but eventually gave up because I don't think it does -shell-escape
. Will investigate your repo a bit more thoroughly in due course.
The error you're seeing seems to be related to the Python version. I can reproduce with v3, but not v2.
I think I've fixed this in the style repo. Reinstalling the package and running xelatex -shell-escape example.tex
seems to work for me now! I think my previous install must've used Python2 by default.
(testing using avenir next and ubuntu mono on Travis atm, I think I should be able to make that work somehow)
Those two are probably fair enough to get working. The sans font is in a bit of a .. grey area.
Thanks for the fix! The little example works now for me too, so that's great.
Yeah and it's unlikely that Tectonic would work with such a complicated document anyway. I think it would be best to stay as close as possible to a local installation, so installing tex live using the install script and then cache it.
But, I still don't even manage to compile the document locally. You spoke about the \faIcons, but where are those commands coming from? I don't see them in the minted output. I still get things like
./motivation.tex:88: Undefined control sequence.
<argument> \faExclamationTriangle
The good thing is, once it works on my machine it's likely to work on Travis as well :)
[Edit] I just realised those must be font awesome icons, I probably did something wrong there. But I installed the fontawesome package and have it in my fonts folder. Investigating...
The actual error is kpathsea:make_tex: Invalid filename FontAwesome5ProSolid:script=latn;, contains ':'
Specifically, you want the fontawesome5 package (not just ,fontawesome
). Note I load it with the pro option, which then relies on the system fonts rather than those bundled with the package. Happy to provide some TTFs if you want them
Font files: fa5.zip
I don't see this any different to publishing the compiled PDF. Obviously there are boring legal implications if you e.g. go on to use these fonts commercially without your own license :stuck_out_tongue:
Thanks for the fonts, though I think you want to remove the attachment again just be sure you don't have any legal implications hanging around :) I had not realised that front awesome pro is not free.
It still doesn't really work for me when I install those (same error) - but even if it did, I don't see any easy way to use non-public fonts in a Travis build? I mean, you could store a secret environment variable which gives access to some non-public storage, but I don't think this is easy. I also don't think it is a good idea to commit those fonts to a private git repository like this, just in case it ever gets public (maybe if your pro account expires or something).
@PHPirates It's an interesting one, but as I mentioned, you'd be redistributing these font files in the same way if you used them on a web-page or inside a document. The browser/document viewer needs to render them somehow, right? If folks use your git repo/document/website to extract the fonts out and violate the license, I see that as their problem. Of course, I'm not doing a law degree :laughing:
In terms of Travis support, the package doesn't need to take the [pro]
option. If it's omitted, it'll use the free fonts shipped with the package without requiring system installation. So yeah, supporting random custom fonts is probably out of scope for your work (Proxima Nova is another problem we haven't discussed) but I think that's just a consequence of me being awkward with how I put the document together :stuck_out_tongue:. I can't see many other projects doing the same.
Hm yeah that makes sense, hadn't thought about that. In the meantime I made the other fonts compile (I didn't really check if they actually appeared in the document, I'm not really good in recognising fonts) on Travis: https://github.com/PHPirates/travis-ci-latex-pdf#using-custom-fonts - it's pretty simple once you remember to install luaotfload when using lualatex...
I know, but you use pro icons so I still want to be able to compile those :) I should try on my Ubuntu system though, maybe it's just my Arch that doesn't want to recognise the pro font.
For reference, I think brackets-curly seems a pro icon, so:
\documentclass{article}
\usepackage[pro]{fontawesome5}
\begin{document}
\faIcon{brackets-curly}
\end{document}
Hey, I was using your document as a test document, but when I tried to compile it (using xelatex and shell escape) your use of these
\PYGidseven
commands confused me, are those internal minted commands? I don't know enough of pygments :/ They get meUndefined control sequence. ...ED\PYGidsevenZus {}META}\PYGidseven
I did find that
idseven
is the pygments style. I found https://github.com/gpoore/minted/issues/230 but it didn't get any clearer.I can reproduce the problem when I compile the following
[Edit] I hope it's no problem for your supervisor that I am trying to compile it... I won't distribute the result, of course.