[Spec] <declaration> production seems to be missing some types of modifiers

[kaby76]

This code can be compiled by the Dart compiler, but it can't be parse with the given rules in the Spec:

abstract class StreamController {
    final int i1 = 0;
    abstract int i2; // Can't parse
    abstract final int i3;  // Can't parse
}

It fails to parse the declaration for i2 and i3 because of the modifier "abstract".

From the Spec, the rule for declaration allows for a number of cases, but does not allow for a declaration that begins with abstract.

Note, for a class member declaration, the parser itself seems to recognize three groups of modifiers followed by a [list of modifiers](), roughly as ('external'|'augment'|'abstract')? ('static'|'covariant')? ('final'|'var'|'const'|('late' 'final'?)? classMemberModifiers?; classMemberModifiers : modifiers; modifiers : (abstract|augment|const|covariant|external|final|late|required|static|var)*. Anyway, the CFG in the Spec needs alts to allow abstract members of a class

[eernstg]

External variables (top-level, static, instance) and abstract instance variables were added with null safety (that is, Dart 2.12). The specification updates about null safety are currently being reviewed, so that's the reason why they are not included in the current version of the language specification.

[eernstg]

Note that the Dart language specification is being updated based on so-called feature specifications. A feature specification is a document which is similar in style to the language specification (some of them are more formal/precise, others a bit less), and it is used to hold the information about the intended syntax and semantics of an upcoming language feature. Check out this directory to find some feature specifications. In particular, the null safety feature specification is here, and the feature specification about abstract and external fields is here.

[kaby76]

Sorry, I didn't understand your development process. Thanks for explaining.

From experimentation, I gathered the "2.15-dev" spec is a backward-looking document. Yes, it's work-in-progress, and it doesn't yet reflect the state of the compiler at tag "2.15.1".

I extended (fudged) enough of the CFG scraped from the spec to construct an Antlr4 parser that recognizes all but a couple of files from a significant subset of the sdk. More importantly, I can now parse the compiler source itself. I will be using that to bootstrap the CFG directly from the source code.

[eernstg]

Right, the version number 2.15 should actually be changed to 2.13 at this point, because it's taken a long time to work on the null safety updates, and we're actually more likely to release a 2.13 spec at this point.

I extended (fudged) enough of the CFG scraped from the spec to construct an Antlr4 parser

Sounds good! I was wondering why you weren't just using Dart.g. If you do find faults in there we should just fix them, and that grammar isn't tied to the same level of scrutiny as the language specification text (so we can move faster, and we can make the choice to model a language version which is basically as recent as we'd want, and it's already close to that).

[kaby76]

Well, I probably could have just used Dart.g, but there was already an outstanding problem with a forked copy of the grammar in grammars-v4 from some years ago, so I wanted to update the copy in grammars-v4. I ported the current Dart.g to "target agnostic format" in order to generated a parser for other targets. But, when it was tested against the issue raised, it didn't fix the problem. I didn't check to see that it works well enough against the Dart compiler source code--but it does! Anyway, that's how I ended up writing code to scrape the grammar from the spec. Thank you for having the spec in .tex format. I have been working on a ML program that scrapes grammars from the pdf's themselves, but it requires an enormous library for training--which is a problem in itself. Since almost all compilers use a hand-written parser, I've been thinking a lot of how to scrape the grammar from an implementation. Dart will be the first.