Closed flying-sheep closed 4 years ago
Nice, I'll review this shortly.
There's something in this script that's not quite deterministic, so different runs give seemingly equivalent but different outputs (hence 61k additions, 44k deletions). If you could find out why, it would be really great.
make the language server aware of the section hierarchy
Well, for which functionality would you like to use this? As of yet, nothing uses the section level information.
support
\start<section>[title={...}]
versions
I'm not sure yet this is the ideal way to deal with startstop stuff, but in principle \startsomething
is governed by the environments.something
entry of the data file.
The script is deterministic. The diff algorithm just isn’t very good, and my change to include all alternatives added a lot of commands:
$ git diff --stat master -- context.tags
data/context.tags | 106618 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-------------------------------------------------------------------------------
1 file changed, 62216 insertions(+), 44402 deletions(-)
$ git diff --stat --diff-algorithm=minimal master -- context.tags
data/context.tags | 17966 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 17890 insertions(+), 76 deletions(-)
ConTeXt’s “alternatives” are instances of commands with some parameters pre-filled. E.g. all headings are just instances of \section
with different levels, different styles, and some with no numbering.
They could of course be represented more minimally in the data file if you want.
I’m cheating, as I actually get the following error when running the script. Any idea why my addition to the utils
file doesn’t work?
lua: ../scripts/extract-context.lua:162: attempt to call a nil value (field 'has_value')
Am I using the wrong digestif.utils
because I also have digestif installed system wide?
As of yet, nothing uses the section level information.
We should use the symbol hierarchy to have an outline for navigation.
1. The script is deterministic.
Thanks for checking!
2. Am I using the wrong
digestif.utils
because I also have digestif installed system wide?
Probably. The order in which modules are loaded in Lua also seems a bit weird to me. In this specific case, I think has_value
is specialized enough that it could stay inside the extract script.
3. We should use the symbol hierarchy to have an outline for navigation.
Yes, that's probably the next most interesting feature to add.
Are you interested in cross-reference and bibliography support? For this, it would be necessary to add action
fields with value "label"
, "ref"
, "cite"
, "bibitem"
to the commands that behave like the corresponding LaTeX commands. This is a lower-hanging fruit, since we can reuse most of the existing code for LaTeX.
Well, I also don’t have much time, as I rather spend time on my thesis than this at the moment, so figuring this out isn’t my first priority, sorry. Maybe some hours at the weekend.
But here’s all I know to make it easier:
% section headers, \command style
\chapter[preface]{Dear Reader}
% section headers, \startenvironment style
\startsection[reference=methods, title={Methods}]
% direct label creation
\pagereference[reference]
\textreference[reference]{text}
\in{chapter}[preface] % chapter 2.2
\at{page}[preface] % page 24
\about[preface] % “Dear Reader”
% after the following, \insec[reference]/\atsec[ can be used
\definereferenceformat[insec][text=section]
Standard or named
\usebtxdataset[filename.bib]
% or
\usebtxdataset[standard][filename.bib]
% or
\definebtxdataset[name]
\usebtxdataset[name][filename.bib]
Besides a .bib file, there are the possibilities buffer, XML, Lua (see here)
\startbuffer[bufname]
...
\stopbuffer
\usebtxdataset[...][bufname.buffer]
% deprecated
\cite{key}
\nocite{key}
%supported
\cite[scheme][key]
\nocite[key]
\citation[scheme] [key]
\nocitation[key] %or \usecitation[key]
% people also do things like
\def\inlinecite{\cite[authoryears]}
Is the stuff concerning sectioning ready? In this case I would like to merge! The other types of commands can be done later (by either of us).
Not yet, I still have to
\start<sectionkind>{title=...}
form, and then/edit. Tests are in haskell. I can’t (really) code in haskell and have no idea how to set it up in Arch and install the necessary libs…
So, concerning \startsection
and similar forms, the procedure is to add an entry environments.section
to the tags file. This entry can be identical to commands.section
. Alternatively, "links" can be used, which would look like this:
environments = {
section = "$DIGESTIFDATA/context/commands/section"
subsection = "$DIGESTIFDATA/context/commands/subsection"
}
I'd be happy(er) with one PR just to set up the data files, and leave further work that actually do something with the information (including the necessary tests) for future PRs.
Once the data files are there, I personally would work out the cross-references and bibliography part.
PS: I just noticed that the startstop forms for section have different arguments, so linking can't be used in this case.
Okay, then this should be almost ready, as e.g. environments.chapter
already has action
and section_level
set correctly.
One question: I could link section
instances to their main command, if it wasn’t for section_level
. Any way around that?
Other thoughts:
For the keys to use, check this. I think the fallback sequence for what to use as LSP DocumentSymbol
should be bookmark
→ list
→ title
(as PDF bookmarks are used and displayed similarly as DocumentSymbol
s), and reference
can be used for supplying, well, reference targets.
chapter = {
action = "section",
arguments = {
{
delimiters = "$DIGESTIFDATA/context/data/brackets",
keys = {
bookmark = { meta = "text" },
list = { meta = "text" },
marking = { meta = "text" },
ownnumber = { meta = "text" },
reference = { meta = "reference" },
title = { meta = "text" }
},
list = true,
meta = "assignments",
optional = true
},
{
delimiters = "$DIGESTIFDATA/context/data/brackets",
keys = { ["cd:key"] = { meta = "value" } },
list = true,
meta = "assignments",
optional = true
}
},
section_level = 2
},
For tests I came up with this so far, but I think that’s trivial:
syms_or_infs <- getDocumentSymbols doc
liftIO $ case syms_or_infs of
Right infs -> fail (printf "Expected [DocumentSymbol] not %s" infs)
Left syms -> syms `shouldBe` [...]
One question: I could link
section
instances to their main command, if it wasn’t forsection_level
. Any way around that?
No. What could be done is linking e.g. commands.subsection.arguments[1].keys
to commands.section.arguments[1].keys
. This in fact applies to all commands that inherit key-value arguments from somewhere else without adding anything new. But that's just an optimization.
what to use as LSP
DocumentSymbol
I thought it would be the reference
! But I guess this depends on what the DocumentSymbol
interface will look like on the editor. What is your expectation? Do you want to see the document sections in a tree structure?
By the way, you pointed out to the wiki, so let me ask: Is there a way we could get a docstring for every command and keyword without hard work like parsing wiki markup or HTML? I couldn't find any.
Another cool thing would be to have a PDF bookmark to the place each command is described in the manual, like we have already for PGF/TikZ. When I checked, the manual had no permanent-looking bookmarks.
What is your expectation? Do you want to see the document sections in a tree structure?
DocumentSymbol is for TOCs: Every editor represent them as a tree of collapsible headers that you can click to jump to a thing. I use them mostly to jump around in my document or code. About the reference
:
detail
: String | null
More detail for this symbol, e.g the signature of a function.
I think adding it as DocumentSymbol.detail
would make a lot of sense!
Is there a way we could get a docstring for every command and keyword without hard work like parsing wiki markup or HTML?
Nope, I asked them and they said:
No, there isn’t. It could be in the interface files if someone would put in the work.
A problem I see with the documentSymbol
request is that, in the LSP spec, a "document" is a single file. So it doesn't seem that LSP allows to show the outline of an entire book if it's split into several files.
There is also a workspace/symbol request, but this one is for a non-hierarchical list of things, so it's also not ideal for this use-case.
Anyway, the way to proceed here is to come up with the best document-outline API for TeX, and then map it onto the LSP spec in the best possible way, even if that's suboptimal.
I don’t see a problem. Yes, it’s per-file. There’s a clear hierarchy defined for LaTeX and ConTeXt each, which can be parsed out of each file.
The workspace/symbol request is of seems useful as a project-wide “go to header/figure” search
@flying-sheep Do you think this can be merged as is now, or with just some quick adjustments? I think we should leave the document outline functions for later.
Sure, I mean it doesn’t do much currently, but it certainly adds correct metadata
\part{...}
, …) asaction = "section"
withsection_level
using extract-context.luaManuscriptLatex
hasaction = "section"
but usesinit_callbacks.heading
?!)\start<section>[title={...}]
versions (how? are\start<thing>
commands specifically supported somehow or do I register a\start
variant of every section command manually?)