Multifolder Documents

[JensAc]

Hi @astoff , I am really interested in using digestif in combination with eglot and tikz-mode. Recently, I have forked tikz.el, and made some changes in order to fit it to my typical workflow. You can find my tikz.el here. However, when I open the example block diagram in Emacs, and add the first line

% !TeX root = ../templates/tikz-preamble-template.tex

I do not get completions for e.g. tikz's node options. Am I missing something? From #19, I know that you are also using and developing digestif in combination with elgot. Would be great, if you could help me with ideas what might be going wrong.

[astoff]

Are you sure this is related to multifolder documents? In other words, if you just have a single file loading tikz, do you get the completions you expect? Conversely, are you not getting completion for cross-references and basic tikz commands in the situation above?

There's one extra issue with tikz key completion: the keys can contain spaces, and Eglot/Emacs doesn't think that completing text with spaces in it is a thing. In my setup, I only see completion when I try to complete the first word. So, for instance, line<tab> suggests line width, but line w<tab> doesn't work.

[JensAc]

Thank you very much for the prompt response. I really appreciate it :+1: . I not entirely sure, but yes: A single files works perfectly fine. Even if I simply add

\usepackage{tikz}

to the example, I get corrections and eldoc hints. As soon as I remove the usepackage-command and rely on the parent document, I only get e.g. eldoc hints, when point is at e.g. \begin{environment}. When I put both, child and parent document, into a single folder, and really \input{.} the child document (which I do not want to do for tikz.el), I get completion and eldoc hints. That is why I supposed that this relates to my folder structure.

I don't care for the extra issue. Apparently this is out of scope and having eldoc and basic completion working would be great for tikz.el.

[astoff]

Okay, I understood what is causing the problem, and will try to fix it soon. In the meanwhile you can just use an absolute file name in the root directive.

By the way, since you are interested in tikz integration, it would be really cool we if we could ship a version of the PGF manual so it can be shown in eldoc. We would have to convert it to markdown, like the “Impatient” book.

Another idea would be to have the PGF manual in https://devdocs.io/. If DevDocs shipped a cleaned-up version of the PGF manual, we could do a (somewhat lossy) conversion to markdown rather easily.

[JensAc]

Alright, thank you very much. I'll take a look into the integration of the PGF manual. I hope that I can find some time for this in the next weeks.

[JensAc]

One further question in order to understand, whether I get it correctly: Converting the PGF manual into markdown or porting it to devdocs.io is mainly a manual process or am I mistaken? Then, it would also require manual maintenance with very high effort, as the documents are non diffable.
However, it may make sense to convert only a fraction with the most basic commands.

[astoff]

Hi Jens, I pushed a fix to this issue, if you get the chance to test, let me know if it works for you.

As to the conversion, one would have to write some scrapper or converter to a plain text format, since the PGF manual aims mostly at the PDF version. I wasn't thinking of it as a totally manual process, but rather some script that converts the subset of LaTeX used by the PGF manual to text.

It is possible to write new tags files completely or mostly by hand. For smaller packages, it's not an unreasonable thing to do.

[JensAc]

Just tested it, and for me the behavior is still the same. Everything works fine, when the files are located in the same directory, but I do not get eldoc and completions working, when the files are distributed over several directories. I also tested with absolute file paths in the % !Tex root = ... magic and also the input{.} statement in the parent document. That did not play the trick either.

I installed digestif via the wrapper script and the repository at ~/digestif is at your latest commit. I am sorry, but I think at the moment I am of little help, when it comes to the actual implementation, as I am neither familiar with lua nor with lsp.

[astoff]

That's strange, since I could reproduce the issue and it's fixed for me now. One sanity check you can make if you want is, from the root file, can you complete \label names defined in the \inputed ones? Also, I think \input requires that you add the .tex extension, and \include requires that you don't add it.

[JensAc]

Ok, than I am most probably doing something wrong. Did you reproduce with the folder structure in of the tikz.el repo or some other mwe? In the second case, could you point me to your test environment so that I can double check what I might be doing wrong?

I did the \label completion test in the root file and it works perfectly fine.

[astoff]

If you prepare a ready to use MWE (which I believe the linked repo isn't), then I'll be happy to test.

But note: if the root file doesn't refer to the current file through an \input or chain thereof, then the root directive is ignored.

[JensAc]

Yes, you are right. Apologies, I did not provide a real MWE, but had some local changes. I prepared a separate branch

git clone git@github.com:JensAc/tikz.git
cd tikz
git checkout MWEDigestifIssue
cd templates
pdflatex tikz-preamble-template.tex

should result in an output document. However, opening the child document in tikz/examples/block_diagram.tex is exactly what I was testing with and what was making me observe the described behavior.

I understood already from your previous posts that the root file needs real \inputs. At the end of the day, that will be no issue, since tikz.el is agnostic to this, and one could set the root file as tikz-preamble-template in a file-local-variable.

[astoff]

Ok, there's another issue to fix here, indeed. However, if the root document is not at the root directory of the project, it's not even clear what the expected behavior is.

Eglot calls Digestif from the tikz directory, since that's you project root. But if you do pdflatex templates/tikz-preamble-template.tex from the project root directory, your files will not compile. If you place you root document at the project root directory, there's not possible confusion.

[JensAc]

Hi @astoff ,

As to the conversion, one would have to write some scrapper or converter to a plain text format, since the PGF manual aims mostly at the PDF version. I wasn't thinking of it as a totally manual process, but rather some script that converts the subset of LaTeX used by the PGF manual to text.

I checked the repo once again this weekend, and I am ready for another educated guess how to proceed....You have written a script already, which generates a tags file for PGF and TikZ. However, the actual docstring is missing, as one would need to get rid of the TeX-macros found in the details of the manual? Then, the next step would be to check, whether once can eliminate all disturbing TeX code from the docstring automatically via PEGs or similar? Moreover, an appropriate Markdown formatting should be done?

I think, I would give this approach a try. What do you think? Did you already try and were left without something useful at the end of the day? I just want to make sure that I am not running down a rabbit hole, which some else has already explored, before I really start on that :wink: .

[astoff]

Hi Jens,

However, the actual docstring is missing, as one would need to get rid of the TeX-macros found in the details of the manual? Then, the next step would be to check, whether once can eliminate all disturbing TeX code from the docstring automatically via PEGs or similar? Moreover, an appropriate Markdown formatting should be done?

That's absolutely right. One can more or less easily extend that script to capture the body of the interesting_envs. The difficult part is to convert the LaTeX markup to some reasonable Markdown. I'd suggest using pandoc for that.

I must confess that I've tried this in the past and wasn't super happy with the result. But since then no better alternative emerged (for one thing, I just tested and the PGF manual doesn't look very good when compiled with tex4ht), so we could revisit the idea.

[astoff]

I've added the script that filtered text through pandoc here. I hope it still works, but it might need some adaptations.