Return optionals as part of a Cucumber Expression match

[aslakhellesoy]

🤔 What's the problem you're trying to solve?

I want to provide syntax highlighting (LSP semantic tokens) for the parts of a Gherkin step that is optional in the corresponding Cucumber Expression.

✨ What's your proposed solution?

Use named capture groups for optionals, as explained in https://github.com/cucumber/language-service/issues/57

The return type of match would change from Argument[] | null to Match | null:

export type Match = {
  arguments: readonly Argument[]
  optionals: readonly Optional[]
}

export type Optional = {
  group: Group
}

⛏ Have you considered any alternatives or workarounds?

If we want to support this, I can't think of any other option.

[mpkorstanje]

Currently you can not assign special meaning to groups because every possible regex generated by a Cucumber expression is also a regular valid regular regex that may be provided by a user.

For example the regex:

A(?<Special-String> non-optional) pattern

Would be capturing when used with Cucumber, but non capturing in the Language Server. And while it is unlikely a user may have guessed Special-String, it is an indication that this abstraction is leaking.

Now I'm not opposed to adding named group support to the tree regex. These names could be used when generating step definitions based of regular regexes for example.

However the proposed Special-String should not be used in production code. So consider generating a different regex patterns when transforming the AST to a Cucumber expression while using the langue server.

This may require some refactoring.

[aslakhellesoy]

I wrote in https://github.com/cucumber/language-service/issues/57 that we could change TreeRegexp. After thinking about it a bit more, I think the correct place to do this is in rewriteOptional: https://github.com/cucumber/cucumber-expressions/blob/2841a19bc90d6c02b571a3670f9b89583bda5f79/javascript/src/CucumberExpression.ts#L72

We could pass a makeOptionalGroup: (regex: string, count: number) => string constructor argument to CucumberExpression (defaulting to this:

(regex, count) => `(?:${regex})?`

That way, the language server could pass its own:

(regex, count) => `(?<optional-${count}>${regex})`

[mpkorstanje]

That interface is very language server specific.

For example:

• count is not a concept that occurs in cucumber expressions. It exists only for the language server.
• This only and specifically replaces optionals.

Ideally we define the Cucumber expression API in its own domain terms.

So I think some refactoring is in order.

[mpkorstanje]

It does come to mind that currently we also assume that all capture groups are parameters. If you write optionals as capturing then match won't work anymore.

Maybe you can extract the code that rewrites the AST, add support for named groups to tree-regex and then use those in a class that is not CucumberExpression.