In addition to being an extreme amount of work, a lot of users will probably have strong reactions to major syntax changes. Still, it's a good time to do it if you're going to.

Markup

I think SugarCube should adopt github flavored markdown, or at least most of it to the extent possible. I doubt this is particularly controversial.

Macro Syntax (general)

Double angle brackets are a bit awkward to type and read, though they work and you do get used to them. Ideas for potential replacements:

Harlowe-esque: (macro: args...)
BBCode style: [macro: args]
Most grouping-style punctuation could conceivably work, grouping meaning it uses two separate characters for opening and closing, so braces could work too. Single arrow brackets are certainly unwise given the potential confusion with HTML. Using parens or brackets will probably require an additional tell in the macro like Harlowe's colon.

Arguments

A space separated list is fine, but it introduces problems parsing expressions, necessitating the use of backticks to indicate when the result of an expression is to be passed as an argument. Harlowe uses commas to get around this (mostly). Separating arguments like this may be the lesser of two evils compared to a special syntax for expressions.

Arguments could also be passed in other ways, like in this BBCode-ish syntax:

[link passage="some passage" text="Click me!"]...[/link]

This doesn't really solve the issue of expressions directly, though it may still be possible compared to space-separated arguments. It does allow arguments to be passed in any order, however, at the cost of more typing.

Containers

SugarCube could adopt blocks (or "hooks") rather than opening and closing macros as containers. Doing this may actually open up even more possibilities for macro syntax as well, since they will be associated with elements by means other than a closing tag. Of course, this has the side effect of making child tags of some kinds quite a lot more complicated to implement as near as I can reason about it.

Markup

I think SugarCube should adopt github flavored markdown […]

Some form of Markdown would be nice.

Macro Syntax (general)

I'd rather not use a Harlowe-esque parenthesis macro syntax for the simple reason of not wanting users to expect things to work the same as Harlowe.

The same could be said for a square bracket syntax now that Chapbook is using it, though I'm less firm about that currently as I see Chapbook being more of a niche player than Harlowe is.

Block-style syntaxes that I've what-if'd before:

(macro: …){ … } → Harlowe-esque for the primary macro body, so boo
[macro: …]( … ) → Vaguely Markdown-esque as it looks like the its link syntax
[macro: …]{ … } → I have to admit, I do love me some curly braced blocks
{macro: …}( … ) → No format is using curly braces as its primary macro syntax metacharacter
{macro: …}[ … ] → ditto

Arguments

Harlowe uses commas to get around this (mostly).

Harlowe gets around that in its argument-type macros by (a) largely not allowing JavaScript expressions and (b) having a severely restricted syntax besides. Commas aren't a particularly great help there with parsing—not saying they're no help, just that the simpler situation in general makes the most difference.

That said, and beyond any potential they may/may not have in parsing, comma-separated argument lists are probably nicer for users as a more explicit way to delineate arguments.

Arguments could also be passed in other ways, like in this BBCode-ish syntax: […]

HTML content attribute-style named arguments do not give me the warm fuzzies. Some macros do have named arguments by sheer necessity—e.g., <<audio>>—but with or without a syntax change I'd prefer to keep the current name value syntax there. E.g.,

[audio: trackId, volume 1, loop true, play]

Of course, named arguments could be done away with altogether if they were converted to subordinate macros. E.g.,

/* Indented */
[audio: trackId]{
    [volume: 1]
    [loop: true]
    [play:]
}

/* Compact */
[audio: trackId]{ [volume: 1] [loop: true] [play:] }

Just me playing what-if.

Containers

SugarCube could adopt blocks ~~(or "hooks")~~ rather than opening and closing macros as containers.

If I were to change the syntax, blocks would almost be a given.

Doing this may actually open up even more possibilities for macro syntax as well, since they will be associated with elements by means other than a closing tag.

I'm curious about your thinking here.

Of course, this has the side effect of making child tags of some kinds quite a lot more complicated to implement as near as I can reason about it.

Well. With a block-style, rather than container-style, syntax, I think most child tags would pretty much need to become separate subordinate macros—some would definitely need to become so. For the most part that shouldn't be a huge deal. The only real snag would be child tags with the same name of differing parents, which is easily doable in the current syntax, but could become a nightmare in a block-style syntax.

Examples

NOTE: The following examples are not an endorsement of any particular syntax.

For child tags that are better as follow-ons, they'll simply need some must-follow logic. E.g.,

[if: condition]{ content… }
[elseif: condition]{ content… } → Must follow [if:] or [elseif:]
[else:]{ content… }             → Must follow [if:] or [elseif:]

For child tags that are still better as children, they'd probably still become separate subordinate macros, but with must-have-parent logic—parents would also likely have only-children logic. E.g.,

[switch: expression]{
    [case: valueList…]{ content… }
    [default:]{ content… }
}

[createplaylist: listId]{
    [track: trackId, actionList…]
}

Some macros within the standard macro library already work that way: <<for>> + <<break>>/<<continue>> and <<repeat>> + <<stop>>.

Bonus: Such a syntax could allow optional blocks easier than the current syntax does. E.g.,

/* Valid, no on change block */
[textbox: receiver, defaultValue, passageName, autofocus]

/* Also valid, with an optional on change block */
[textbox: receiver, defaultValue, passageName, autofocus]{
    on change content…
}

Concern: The real concern, as noted above, are macros with child tags of the same name. E.g.,

[a:]{ [c:] }

[b:]{ [c:] }

Or a real example from the standard macro library:

[createaudiogroup:]{
    [track: trackId]
}

[createplaylist:]{
    [track: trackId, actionList…]
}

This isn't impossible to solve, but it does make macro registration and handler dispatch more complicated.

general syntax

My preference is [macro: ... ]{ ... }, though really I think all are okay. I suggested the Harlowe syntax because familiarity has benefits too, though I agree that the confusion could very well outweigh any positives.

commas, subordinate macros and named arguments

That's true about Harlowe's limitations. Still, I agree that comma-separated arguments are easier to manage for users.

Named arguments has benefits, but it's mostly just spit balling. I don't hate the idea of subordinate macros when named arguments are necessary, though.

"Doing this may actually open up even more possibilities for macro syntax as well, since they will be associated with elements by means other than a closing tag."

Hmm. I don't remember what I was thinking. I might have been saying "if we do drop the container macro system in favor of blocks, that makes chaining the syntax easier." not 100% sure though, sorry. If it was a good idea or point I'm sure I wouldn't have forgotten it though, lol.

Well. With a block-style, rather than container-style, syntax, I think most child tags would pretty much need to become separate subordinate macros—some would definitely need to become so. For the most part that shouldn't be a huge deal. The only real snag would be child tags with the same name of differing parents, which is easily doable in the current syntax, but could become a nightmare in a block-style syntax.

Yeah that's what I meant. There'll need to be at least two "classes" of subordinate macro for children and follow-on macros, and there may be more strictness with names. Maybe I over sold it by saying "a lot" more complicated, but the simplicity of the current container-child system is a valid benefit it has over blocks.

the examples

I am a big fan of this syntax, seeing it in action. Being able to have optional blocks is kinda awesome too.

but it does make macro registration and handler dispatch more complicated.

One possible solution is to reference the 'child' macros (eg. [track: trackId] and [track: trackId, actionList…] as something other than 'macro', for instance call them a macro 'extension'.

Macro.add('createaudiogroup', {
    ...parent macro definition...
});

Macro.extend('createaudiogroup', 'track', {
    ...child macro definition...
});

Macro.add('createplaylist', {
    ...parent macro definition...
});

Macro.extend('createplaylist', 'track', {
    ...child macro definition...
});

I am not privy to every language in the world but of the languages that use a Latin-based alphabet, virtually none that I know of use curly braces with any regularity. On the other hand parenthesis are very common and square brackets not uncommon. Angle brackets (which are basically abused great and less than signs) are used but as a bracket most commonly only with things derived from SGML like HTML, XML, etc.

Since we expect inline language text and even some HTML tags, I would prefer overloading those in non-intuitive ways. I like the author closing macro concept akin to Harlowe hooks, etc., however, since we already have begin and end tags for inline HTML, perhaps we should continue in motif. I can understand wanting to get away from <<>> syntax but scv3 could continue to use double character markers without having to use angle brackets or even using the same characters. C language comments /* */ are an example. The pipe bar | rare in non-code so we could adopt that.

I like named parameters but I agree they need to be denoted in another way besides space-separated. Either use some quoting demarcation or some other separator like the aforementioned comma , . In situations where you want a long list of similar arguments, I recommend encapsulation, passing the long separated list as the argument of a named parameter.

Perhaps a syntax like:

{|macrotag|paramname1="x"|paramname2="y"|paramnamen="1,2,3"|}contenttext{|endmacrotag|}

Of could it might look nicer with a little optional space:

{|macrotag |paramname1="x" |paramname2="y" |paramnamen="1,2,3" |}contenttext{|endmacrotag|}

If you prefer we could open and close with double curly braces. With the pipe bar this makes it similar to Mediawiki template markup (used on Wikipedia).

If we really like the autoclosing Harlowe hooks concept, content could be gotten rid of by putting it into an argument. Named parameters make this considerably less objectionable (not to mention removing argument ordering dependence).

Of course something like this might be interesting too (I think they lifts that from C#): https://docs.python.org/3/library/string.html#formatstrings

I think I've made up my mind about some of the syntax changes I plan to implement in v3—the scope of which, or lack thereof, may be disappointing to some.

Markup

Styling markup

I believe that I am going to switch to something Markdown-flavored. I may still support the old styling somehow for compatibilities sake.

Link & image markup

The components of the link and image markup will be changed to: plaintext or template string.

Some link examples:

[[passageName]]
[[linkText|passageName]]

[[`passageName`]]
[[`linkText`|`passageName`]]

Template strings may, of course, contain placeholders.

Switching to template strings from raw expressions will resolve a few different issues.

I've not made up my mind about the setter component, but I'm leaning heavily towards dropping it from both markups. Both because it's unnessary and for compatibility with other modern formats.

Macros

Macro changes will likely be minimal, but will resolve current ambiguities and make it much easier to add new arguments in the future.

Expression-style macros

Expression-style macros should largely remain as-is.

Argument-style macros

Arguments

Argument-style macros will have two types of arguments: required positional and optional named (optionally with a required sub-argument). Optional arguments may no longer be positional.

Removing the ambiguity of optional positional arguments allows the addition of new (named) arguments both immediately and in the future.

Some examples:

<<link
    /* Required positional. */
    requiredLinkText | [[link_markup]]

    /* Optional named. */
    go passageName
    id id
    class classes
    one
>> …

<<radiobutton
    /* Required positional. */
    receiverName
    checkedValue

    /* Optional named. */
    group groupId
    autocheck | checked
>>

<<textbox
    /* Required positional. */
    receiverName

    /* Optional named. */
    default text | placeholder text
    go passageName
    autofocus
>>

Argument Types

Argument-style macro arguments will accept the following types:

${expression} → Replaces backquote expression
template string → Was backquote expression
double & single quoted string
bare string
bigint
number
boolean
null
undefined

Anything else, symbols & objects, may be handled via the expression syntax (${…}).

I am not a fan SugarCube's use of double angle bracket usage but I am pleased to hear that progress has been made. I like the ${expr} syntax. It feels sort of Bourne shell like (while also not being so far away from jQuery, etc.).

Styling Markup

I would love to see markdown, as I'm sure you know. I actually think you should use the opportunity of moving to SC3 to just get rid of stuff. I don't think it'll hurt to support some old SC markup, but if you can't support all of the old markup (maybe you can support it all, I haven't looked closely, but I think at least some stuff will need dropped) supporting any of it could cause even more confusion. Like if a user is accidentally looking at SC2 docs while using SC3, some markup or code working may just be more confusing. It's not a hill I'd die on, and compatibility is good, but there's no need to support old syntax in my opinion.

Link & image markup

I think moving to plain text & template strings is smart and better than the somewhat strange behavior we have now.

I support getting rid of setters. I get why they exist, but I think they were always fiddly and unnecessary. Every alternative to setters creates easier, more maintainable, and less error-prone code.

Macro arguments

Argument-style macros will have two types of arguments: required positional and optional named (optionally with a required sub-argument). Optional arguments may no longer be positional.

I like this change. I'm curious about if users try weird things, like if they have $linkAction = "go", $linkActionData = "passage" could they do something like <<link "Click me!" ${$linkAction} ${$linkActionData}>>? In other words can they dynamically provide argument names?

I also think commas to separate arguments or name-argument pairs might be good,

<<link "Click me!", go "passage name", id "my-link">>

In most of your examples you split them off onto newlines, but all sandwiched together, it's a little less readable:

<<link "Click me!" go "passage name"  id "my-link">>

It's just not as clear at a glance I guess what's associated with what as it could be. Another option is using equals

<<link "Click me!" go="passage name"  id="my-link">>

Or colons

<<link "Click me!" go: "passage name"  id: "my-link">>

You could also optionally allow that syntax but not require it, but maybe that's unwise. I guess I feel like I just want something a little clearer in connecting the names to their arguments. Someone is gonna look at this:

<<link "Click me!" go "passage name"  id "my-link" one>>

And write this:

<<link "Click me!" go "passage name"  id one "my-link">>

At some point. And I don't even know if that would/should error since "one" is acceptable as a bare string.

Anyway, overall I think everything here is smart. Good changes, I look forward to seeing more.

About the double angle brackets

I just want to say I still don't like them, but I also get the reasons for sticking to them. Same with start/end tags instead of block syntax. I would have preferred some change here, I think there are a lot of valid reasons to stick to them.

If there is a drastic change in the markup between scv2 and scv3 (which seems appropriate) and there is concern about portability and backwards compatibility, an alternative traditional style of markup could be made an add-on an/or a (default disabled) configuration option. This would allow for alternative markups including native scv3, a more backwards compatible scv2 style and perhaps even an API to allow people to develop others (beyond templates and macros).

The real power of any game engine like twee/twine story formats are its functionality and programming API. For twee/twine story formats this is most appropriately presented as a JavaScript API (as with most browser-based game engines). That said, it is also useful to have one or more markup formats to aid in authoring simplistic stories/games.

The most flexibility would be to have an API that completely controls the available markup. Games that wanted a completely custom markup could develop and load their own in the JavaScript startup "passages", etc. With wide enough functionality and an API to control the markup, a single story format could be used to support emulations of any other story format. Of course they would not be compatible from a JavaScript API level but otherwise could be made to work essentially the same. For example, it might be possible to one day emulate Harlowe in SugarCube, etc. Of course this begs the question "Is that useful/worthwhile?" (perhaps not). But the point is, the key to such power is flexibility. For this reason I would like to see the markup completely modularized.

You can basically already add custom syntax very easily. And it's not like power users couldn't fork the engine to completely replace all the syntax. But encouraging users to create their own syntax is probably not a great idea. If everyone is using their own customized syntax for no reason other than personal aesthetics, that makes helping other people, sharing code, or even just collaborating a chore.

I think there are limits to how much customization is even useful, let alone practical.

Macro arguments

Argument-style macros will have two types of arguments: required positional and optional named (optionally with a required sub-argument). Optional arguments may no longer be positional.

While I don't expect those who are used to the syntax of command line arguments passage to have any issues with the new proposed structure, I can see some potential for confusion/error for those with less experience. eg. what potential "why doesn't my code work" questions we may recieve.

What you are basically suggesting is the passing of an Generic Object containing the Name/Key value pairs to the macro, using "suger" to remove the normally required syntax.

eg.

/* Using sugar */
<<link "Library" go "Library" id "library" class "library" one>>

/* Without sugar */
<<link "Library" {go: "Library", id: "library", class: "library"} one>>
or
<<link "Library" {go: "Library", id: "library", class: "library", one: true}>>

And amount the many ways I can see the less experienced Authors getting the above structure wrong, I can also see them trying to use the same "sugar" when doing things like...

<<set $player to id "player" name "Jane" age 21>>

Then there is the question about how someone defines the above argument structure within their own custom widgets & macros, and the potential to also get that muddled.

tmedwards / sugarcube-3

Syntax Changes #15

Markup

Macro Syntax (general)

Arguments

Containers

Markup

Macro Syntax (general)

Arguments

Containers

Examples

general syntax

commas, subordinate macros and named arguments

the examples

Markup

Styling markup

Link & image markup

Macros

Expression-style macros

Argument-style macros

Arguments

Argument Types

Styling Markup

Link & image markup

Macro arguments

About the double angle brackets

Macro arguments