Where is the list of YW comment "keywords?"

[olyerickson]

Perhaps I've missed this; where can I find a complete list of the YW comments and their definitions?

So far I've only been able to find keywords from the examples in the prototypes README.

Thanks!

[csjx]

@olyerickson This is definitely a work in progress, and we'll need to touch base with @tmcphillips to see if he has updated the syntax that we've worked on to formalize the YesWorkflow syntax early on. The ticket for that is https://github.com/DataONEorg/sem-prov-design/issues/54, and the draft document to produce an EBNF specification for the YW syntax is: https://docs.google.com/document/d/16320aNsDgvanDVsjkm8Kj0p9afVAocril8dS8_mtYxc/edit .

Tim, will you review the EBNF spec table, and determine if you've added or removed any YW keywords? If so, I think it's most appropriate to update the document and get comments from the YesWorkflow and DataONE Semantics/Provenance groups on any changes. I think we should also make an effort to tag and release this EBNF spec for any tagged and released versions of the YW implementation projects.

John, thanks for pointing this out.

Cheers, Chris

[csjx]

Aside from the formal EBNF spec document, we also need a syntax guide based on the EBNF spec for people to easily refer to. That's really what John needs here, but I think the EBNF draft is all we have so far. Tim, I realized that you added @url, so that needs to be added to the spec document.

[tmcphillips]

It sounds like there are three things we need here:

1. Comprehensive user documentation of the YW syntax, for example on a GitHub wiki page for now. We'll need to distinguish between stable keywords/features on the one hand, versus experimental keywords and features on the other.
2. Formal EBNF specification of the language, initially on a wiki page.
3. Use of ANTLR, say, to generate the parser for YesWorkflow. Besides other advantages we'd know that the EBNF spec is correct. We (and others) also would be able to generate a parser useable from Python this way.

[tmcphillips]

The current YesWorkflow implementation extracts YW annotations from source code in two stages. First it extracts all of the comments that appear in the code. Then it parses the YW annotations found in those comments. The first step is language dependent, while the second is language independent. The goal is to allow YW annotations to be detected in any comment recognized by each language even when there are several different ways of indicating or delimiting comments in a particular language.

We can approach the EBNF specification and parser generation the same way to make these clearer and to keep the emphasis on the more interesting, language-independent aspects of the YW syntax.

[tmcphillips]

I've started a new draft of an EBNF specification of the YW annotation language, starting from the earlier specification. The grammar assumes that the input comprises just YW comment text that has been extracted from a source file in a previous processing step, so there are no references in the grammar to the code that is annotated, to comment delimiters, etc. It uses the ANTLR4 variant of EBNF so that eventually we can generate parsers from it.

The draft is still somewhere between pseudo-code and actual EBNF, hasn't been tested, and likely is full of errors. But it already may serve as a useful reference to YW. I'll post an update hear once I verify I have a version that ANTLR4 can generate parsers from.

[tmcphillips]

The EBNF specification for the YW grammar now can be used by ANTLR4 to generate a parser. The parse trees I've generated with the parser so far for simple YW examples appear correct, although the testing is not extensive yet, and the PATH_TEMPLATE lexer rule is incomplete.