The X<> code syntax for creating 2 or more entries is almost not used and probably should go away

Altai-man commented 1 year ago

Problem or new feature

The X<> formatting code allows to index 2 and more entries under a single code, separated by a semicolon.

However, this demands more complex code for the renderer which has to either render them as a single URL (wrong, how it was made up to now) or render a number of URLs in all the places where this is happening.

In fact, there are not so many places where the syntax is used, in docs with thousands of references we have:

➜  doc git:(master) rg 'X<(.*?);(.*?)>'
doc/Language/packages.pod6
201:=head2 X<Direct lookup|Syntax,::($c).m;Syntax,A."$m"()>

doc/Language/pod.pod6
784:A X<hash|B<hashes, definition of; associative arrays>>

doc/Language/quoting.pod6
19:=head2 X<Literal strings: Q|Syntax,Q;Syntax,｢ ｣>
132:=head2 X<Escaping: q|Syntax,q;Syntax,' '>
171:=head2 X<Interpolation: qq|Syntax,qq;Syntax," ">
454:=head2 X<<<Word quoting with interpolation and quote protection: « »|Syntax,<< >>;Syntax,« »>>>

doc/Language/typesystem.pod6
698:X<|Reference,Enumeration;Reference,Enums;Syntax,enum>

doc/Language/list.pod6
563:=head2 X<Typing|Language,typed array;Syntax,[ ] (typed array)>

doc/Language/objects.pod6
168:=head2 X<Attributes|Language,Attribute;Other languages,Property;Other languages,Member;Other languages,Slot>

doc/Language/syntax.pod6
875:=head3 X<C<Package>, C<Module>, C<Class>, C<Role>, and C<Grammar> declaration|Syntax,unit;Syntax,module;Syntax,package>

doc/Language/subscripts.pod6
375:X<|Syntax,flattening ;>

doc/Language/classtut.pod6
184:=head1 X<Starting with class|Tutorial,class;Tutorial,state;Tutorial,has>
193:X<|Tutorial,type object;Tutorial,DEFINITE>
466:=head2 X<Private methods|Tutorial,FQN;Tutorial,Private methods>

doc/Language/hashmap.pod6
279:=head2 X<Non-string keys (object hash)|Language,non-string keys;Language,object hash;Syntax,:{}>

doc/Language/control.pod6
367:=head1 X<C<with orwith without>|Control flow,with;Control flow,orwith;Control flow,without>

doc/Language/operators.pod6
433:=head2 X<Reversed operators|Metaoperators,R;Metaoperators,reverse metaoperator>
442:=head2 X<<<Hyper operators|Operators,<<;Operators,>>;Operators,«;Operators,»;Operators,»=«;Operators,«=»>>>
620:=head2 X<Sequential operators|Metaoperators,S;Metaoperators,sequential metaoperator>
641:The X<quote-words|Terms,qw;Terms,quote-words> construct breaks up the contents on whitespace
665:L<Block|/type/Block> or L<Hash|/type/Hash> constructor. X<|Terms,block constructor;Terms,hash constructor>
706:@container, a.k.a. "X<array indexing operator|Postcircumfix operators,array indexing operator;Postcircumfix operators,array subscript operator>".
728:%container, a.k.a. "X<hash indexing operator|Postcircumfix operators,hash indexing operator;Postcircumfix operators,hash subscript operator>".

doc/Language/regexes.pod6
28:=head1 X<Lexical conventions|Syntax,/ /;Syntax,rx;Syntax,m>
406:=head3 X<C<\n> and C<\N>|Regexes,\n;Regexes,\N>
421:=head3 X<C<\t> and C<\T>|Regexes,\t;Regexes,\T>
429:=head3 X<C<\h> and C<\H>|Regexes,\h;Regexes,\H>
446:=head3 X<C<\v> and C<\V>|Regexes,\v;Regexes,\V>
465:=head3 X<C<\s> and C<\S>|Regexes,\s;Regexes,\S>
473:=head3 X<C<\d> and C<\D>|Regexes,\d;Regexes,\D>
509:=head3 X<C<\w> and C<\W>|Regexes,\w;Regexes,\W>
525:=head3 X<C<\c> and C<\C>|Regexes,\c;Regexes,\C>
548:=head3 X<C<\x> and C<\X>|Regexes,\x;Regexes,\X>
862:=head2 X<Modified quantifier: C<%>, C<%%>|Regexes,%;Regexes,%%>
1098:=head2 X<Start of string and end of string|Regexes,^;Regexes,$>
1141:=head2 X<Start of line and end of line|Regexes,^^;Regexes,$$>
1201:X<|Regexes,«;Regexes,»>
1843:=head1 X<Tilde for nesting structures|Regexes,tilde;Regexes,~>
2314:=head3 X<Ignorecase|Adverbs,:ignorecase;Adverbs,:i>
2322:=head3 X<Ignoremark|Adverbs,:ignoremark;Adverbs,:m>
2334:=head3 X<Ratchet|Adverbs,:ratchet;Adverbs,:r>
2374:=head3 X<Sigspace|Adverbs,:sigspace;Adverbs,:s>
2502:=head3 X<Positional adverbs|Adverbs,:1st;Adverbs,:2nd;Adverbs,:3rd;Adverbs,:nth>
2526:=head3 X<Continue|Adverbs,:continue;Adverbs,:c>
2545:=head3 X<Exhaustive|Adverbs,:exhaustive;Adverbs,:ex>
2571:=head3 X<Global|Adverbs,:global;Adverbs,:g>
2586:=head3 X<Pos|Adverbs,:pos;Adverbs,:p>
2604:=head3 X<Overlap|Adverbs,:overlap;Adverbs,:ov>
2631:=head3 X<Samecase|Adverbs,:samecase;Adverbs,:ii>
2647:=head3 X<Samemark|Adverbs,:samemark;Adverbs,:mm>
2659:=head3 X<Samespace|Adverbs,:samespace;Adverbs,:ss>

doc/Language/glossary.pod6
163:=head1 X<ASCII operator|Language,ASCII operator;Language,Texas operator>
244:=head1 X<Colon pair and colon list|Reference,Colon Pair;Reference,Colon List>
445:=head2 X<Compilation unit or I<compunit>|Reference,compunit (glossary);Reference,compilation unit>
674:=head1 X<Multi-dispatch|Reference,Multi-Dispatch;Reference,MMD>
1011:=head1 X<Tight and loose precedence|Reference,Tight;Reference,Loose>

doc/Language/functions.pod6
142:=head3 X<Automatic signatures|Variables,@_;Variables,%_>
888:=head2 X<Precedence|Traits,is tighter;Traits,is equiv;Traits,is looser>

writing-docs/INDEXING.md
15:    X<text|category1,item1;category2,item2;...>
34:    X<Automatic signatures|Variables,@_;Variables,%_>
35:    X<Typing|Language,typed array;Syntax,[ ] (typed array)>
36:    X<Attributes|Language,Attribute;Other languages,Property;Other languages,Member;Other languages,Slot>

so those can be manually replaced with the 1 code == 1 entry form.

Suggestions

[ ] Write a test detecting the case
[ ] Update the source code
[ ] Bonus points for understanding where semicolon was used on purpose, and ensure it is escaped: say we have this:
```
➜  doc git:(master) rg 'X<(.*?);>'
doc/Language/subscripts.pod6
375:X<|Syntax,flattening ;>
```

which is rendered as just flattening, which is wrong, so resolving such cases would be great too.

finanalyst commented 1 year ago

The problem is with the Renderer, not with the specification. There are many problems with Pod::To::HTML because it is old. For example, it does not implement P<>, nor does it implement =Image (which is a custom block), nor does it implement =item as anything but a bullet list. Just because Pod::To::HTML is wrong, does not mean the POD6 (Rakudoc) specification is bad or should go away

Altai-man commented 1 year ago

The problem is with the Renderer, not with the specification.

I am not calling out the specification itself, I think I never did suggestions to change how Rakudoc language works (if I did that'd be a mistake). The sources we have in this repo use some subset of Rakudoc and because most of the writers should not be absolute experts in this admittedly obscure language, all the used features should be dead simple with plenty of safety nets around.

Otherwise people tend to misuse syntax without knowing and we have bugs such as the case I named:

375:X<|Syntax,flattening ;>

The intention was apparently to get flattening ; as the entry key, but because of misunderstanding the spec it's indexed as just flattening which means a different thing.

So the suggestion here is to, at least for a bit, streamline the subset of syntax this particular repo uses. What Pod::To::HTML can or does has nothing to do with that, it does not matter what tooling you use if the writers misunderstood the spec and due to that do mistakes in the sources.

patrickbkr commented 1 year ago

I agree with the suggestions in the OP.

2colours commented 11 months ago

so those can be manually replaced with the 1 code == 1 entry form.

So what would the "1 code == 1 entry form" look like?

Altai-man commented 11 months ago

For example, instead of

X<Direct lookup|Syntax,::($c).m;Syntax,A."$m"()>

We will have:

Direct lookup X<Syntax,::($c).m> X<Syntax,A."$m"()>

I barely remember the syntax by now, to be honest, but that should give you an idea.

finanalyst commented 11 months ago

@Altai-man The syntax as proposed is quite useful and can be extended to other markup instructions. Basically, the text in the 'meta' section of the markup instruction can consist of a sequence of strings, or a sequence of a sequence of strings. The text is split on ';' and then on ','.

This syntax has been adopted in RakuDoc version 2, please look at Raku/RakuDoc-GAMMA.

The improper use of X<> in the documentation suite is another issue.

Altai-man commented 11 months ago

The syntax as proposed is quite useful and can be extended to other markup instructions

Maybe, to me it looks like yet another abbreviation to golf a symbol or two.

The improper use of X<> in the documentation suite is another issue.

Yes, that's exactly what the ticket is about. :)

finanalyst commented 11 months ago

Yes, that's exactly what the ticket is about. :) Then I am definitely in agreement. X<> was over-used in the documentation suite for the wrong reasons, and under-used for the right reasons.

coke commented 1 month ago

✗ ack 'X<' doc | grep ';' | wc -l
      67

raiph commented 1 month ago

@coke

So, having read this issue it looks like:

2 years ago there were 62 doc lines in 16 doc files that contained at least one X<...;...>. Now there are roughly the same number of lines in roughly the same number of files.
There's consensus about processing them precisely as @Altai-man suggested in his issue opening comment.

Agreed?

Raku / doc

The X<> code syntax for creating 2 or more entries is almost not used and probably should go away #4099

Problem or new feature

Suggestions