Cell Specifiers: Caret Parsed as Superscript

[tajmone]

When tables contain cell delimiters containing cell specifiers with carets to override default cell alignments (e.g. .^|), the caret is being parsed as superscript quote markup.

We need to capture table blocks (opening and closing |=== delimiter) and then ensure we properly match cell delimiters, with or without special characters, thus preventing the syntax from breaking up.

Currently the syntax doesn't cover table blocks (the original didn't, apparently), so we'll have to come up with some solution.

Example:

[cols="<m,2*<d"]
|===============
3+^| header spanning three cells

3*| repeating cell

.^| v-align center
>| h-align right
a| ADoc cell +
ADoc cell +
ADoc cell +
ADoc cell
|===============

in the above example all carets are being parsed as superscript.

References

• Asciidoctor User Manual:
  • §23. Tables
    • §23.3. Cell Formatting

[tajmone]

Table Scopes

Since the current syntax doesn't support Tables, we'll need to find a suitable scope naming.

I've checked the way Markdown is being scoped, but it doesn't provide any scoping for tables; so I searched various resources on the web and found two useful scope examples in Brian Reilly's Sublime Syntax Dashboard, under Scopes » Markup:

markup.other.table
markup.other.table.cell

but I'll probably have to expand on these two a bit, for we need to enclose the whole table in its own scope (possibly, a meta. scope), and account for the opening and closing delimiters.

Also, the above markup.other.table.cell is rather generic, for there are multiple cells in a table, and we need to think if and how we're going to handle them — especially since AsciiDoc can use columns and cells specifiers to assign special styling to table columns/cells (e.g. m for monospace, and a for AsciiDoc cell types), which can lead to quite different supported elements in each cell.

I'm not sure how deep we can (or want) to go into these type of distinctions, because it would be quite difficult to handle all the possible cases (not to mention the multiple ways to attain them) via ST3's RegEx-based syntax — there are clearly limits on how AsciiDoc might be supported without an LSP Language Server, but still ... it's worth considering all possibilities, just to have a clear picture.

[tajmone]

Done! Ready to Merge from Its Dev Branch

@polyglot-jones, I've working on implementing these table fixes, and you can see them in the dev-table-delimiters branch.

Before merging into master, it's worth pointing out that right now table contents are not being highlighted, except for comments and table delimiters.

So, we should decide whether to merge or not this branch, since this will the affect our daily production work.

IMO, we should merge it as it, because cell delimiters operators are one of the major causes of documents breaking up. Even though table contents won't be highlighted, that shouldn't be a huge deal, since most tables have simple contents anyhow, and this is a temporary problem only.

I strongly recommend caution before starting to add extra context inside tables, because until the root elements are not disentangled it's going to be risky to start and include: them at random.

I would rather use un-highlighted tables for a while, until we manage to fix the most common breaking patterns in the syntax (e.g. I keep noticing that some inline styles break Example blocks, for some reason).

ST4 has introduced a new feature which also traces the contexts that lead to a scoping match, when inspecting highlighted tokens (via CtrlShiftAltP), which is kind of useful (even though is seems rather buggy, for it's reporting named contexts as "anonymous" and some reference number).

So Craig, what's your view on merging the tables as implemented in the dev-table-delimiters branch?

Could you checkout the branch locally and play around with it, to see how it feels in production documents. Also look at the Table tests in Tests/table/.

[polyglot-jones]

So Craig, what's your view on merging the tables as implemented in the dev-table-delimiters branch?

I'm in total agreement on un-highlighted tables being better than broken highlights.

Could you checkout the branch locally and play around with it, to see how it feels in production documents. Also look at the Table tests in Tests/table/.

Will do.

[tajmone]

Great! Than I'll wait to merge until you've had a chance to play around with it (I dind't manage to, just a few tests here and there).