Support arbitrary content for entries

[lukasjuhrich]

First up, thanks to the maintainers for providing something as useful as an indexing library. I currently use it for my master's thesis.

Upshot: I cannot do something like #index[$cal(T)_n$-set].

Currently, the index normalizes every entry to text using the as-text function (and as far as I can see, uses this to display things, instead of the original content). That unnecessarily restricts the use of this index, because the only reason to demand a string is sorting of entries and grouping of references.

I suggest an entry should be a dictionary (display: …, key: …) with the following implicit conversions:

• strings will be converted via s => (display: s, key: s)
• content will be converted via c => (display: c, key: as-text(c))
• duples will be converted via (d, k) => (display: d, key: k)
• dicts will be passed as-is.

This would allow for something like

#index([main entry], ($cal(T)_n$-set, "Tn-set"))

or even to arbitrarily manipulate the sort order inside an initial.

When that exists, it might be a good idea to make the panic in as-text refer to this explicit option as a workaround.

While it would also be valuable to make as-text handle more cases on its own, in my opinion this is an orthogonal issue.

[RolfBremer]

That is a good idea! We will experiment with it and see how it may conflict with the formatting of the index page and how to possible prevent it. If you have some real world samples you want to use, I would like to use them for testing.

[RolfBremer]

Ok, we have implemented support for rich display content in the index page entry. Now it is possible to have mathematical expressions in the index. Several samples are in the sample-usage.typ document. Please have a look at it.

[lukasjuhrich]

Hi, I really appreciate the reaction and even implementation! (I got the notification for the v0.4.0 release) Just a heads up, since I am in the middle of a thesis (and some content is still missing), my reaction will be delayed until the polishing phase, which I should get around to in a few days.

[lukasjuhrich]

The polishing phase was a little tighter than anticipated, so I'm reporting right now after handing things in. Thanks for your patience so far.

I've checked fc7535e0f217452da759665057db40483f1a9f8f with my thesis (https://gitlab.mn.tu-dresden.de/s3540841/ma-thesis, tag final-tud), and nothing that I've used so far seems to break.

However, when replacing #index(initial: (letter: "t"))[$#T_n$-set] by #index(display: [$#T_n$-set], "Tn-set"), the formula does not seem to get rendered:

This is also the case in the sample-usage.typ render (see last page).

The reason is what I forgot about, namely the entry-casing. What is rendered is not display, but rather apply-entry-casing(display), which forces the entry through as-text: https://github.com/RolfBremer/in-dexter/blob/fc7535e0f217452da759665057db40483f1a9f8f/in-dexter.typ#L186-L193

I'm not quite sure how one would easily fix that without replacing the string coercion.

One of the two reasons for the existence of as-text – being able to sort – is now obsolete due to key. The remaining one, being able to apply casing, could now be achieved with something less invasive than a coercion. Sure, the casing function expects a string, but that does not mean we have to convert the whole entry to a string, instead we could just apply it to the first visible element of the given content tree (i.e. if the first element is a space, ignore it; otherwise, if the first element is an equation, don't apply the casing, and if it is a string, apply it).

My mind immediately goes to a show rule. After some tinkering, I managed to get the following thing to work:

#let apply-entry-casing(display, entry-casing) = {
    let applied = state("applied", false)
    applied.update(false)
    show text: it => context {
        let t = it.text
        let transformed = entry-casing(t)
        if t == transformed or applied.get() == true {
            it
        } else {
            let t2 = entry-casing(transformed)
            if t2 != transformed {
                panic("entry-casing not idempotent:" + repr(t)
                    + "~>" + repr(transformed) 
                    + "~>" + repr(t2))
            }
            applied.update(true)
            transformed
        }
    }
    display
}

There's a bunch of things happening here.

• First, we need to return it and not it.text if nothing has changed. The reason is that it.text is a string, so what we actually produce is a new text node, and not the original one, causing infinite recursion.
• Second, I added an idempotence test, because while fiddling I broke the first-letter-up function, causing infinite rewrites.
• Third, I added a way to apply this show rule to only the first matching element by using state. Otherwise, in #index(display: $(n, k)-representable$, "nkrepresentable"), both the $n$, the $k$, and the representable get uppercased.

The result with casing is something like this:

So the last thing one would need to fix is to allow users to set soomething like apply-casing: false as an escape-hatch, to fix indices starting with a formula.

While rewrites can be quite expensive, execution time is not impacted too much, because the show rule is only applied to index contents.

[RolfBremer]

We used your code in the just committed 0.4.3 version, and also implemented the apply-casing parameter to index. Currently, if there are multiple entries with the same key, but different apply-casingsettings, the first one wins. Not sure if that is the best solution.

[hurzl]

It would be nice to have some automatic sorting of umlauts etc. (ä as ae, ö as oe, ü as ue, ß as s, æ as ae, etc.) or a simple way to define these conversions.