search

protocolbuffers / protobuf-grammar

This repository contains grammars for parsing Protocol Buffers's .proto files
BSD 3-Clause "New" or "Revised" License
25 stars 12 forks source link

Add context free proto file grammar #2

Open ObsidianMinor opened 4 years ago

ObsidianMinor commented 4 years ago

This repository is looking lacking, so I thought I'd get it started. I also plan to write specs for proto2 and proto3, but in the meantime I decided to write this context free grammar to document a grammar that works for every proto file regardless of whether the syntax is valid in context (like incorrect syntax versions or invalid values for certain options)

derekperkins commented 4 years ago

Was there a reason this never got merged?

teijeong commented 4 years ago

+1, Was there a reason this never got merged?

Logofile commented 2 years ago

ObsidianMinor, we were going to turn down this repository when we discovered this outstanding PR. Sorry for the losing track of this!

Would you mind redirecting the location for the file to https://github.com/protocolbuffers/protobuf/blob/master/docs/grammar.ebnf? I'll work to get the doc reviewed by our engineering team in the meantime, in case any changes are needed before we can accept the PR.

jhump commented 2 years ago

FWIW, here's another alternative that describes the language, also in EBNF. https://github.com/jhump/protocompile/blob/master/grammar/README.md

Unlike this document, it separates lexical analysis from the rest of the grammar productions in order to describe the nuance in tokenization relating to handling of whitespace and comments. It also has a slightly different way to interpret numeric literals, in an attempt to codify some of protoc's behavior. For example, in the face of "1to1000", the grammar in this PR suggests a tokenization of "1", "to", "1000", but protoc considers this a syntax error.

I have high confidence in the grammar as it is based on a yacc grammar that powers what I think might be the closest implementation to protoc itself.

Logofile commented 2 years ago

Joshua, if you'd like to submit a PR for the same target location (so we can get all of the CLAs and such covered), we'd love to take a look at adding it to our docs. I'm still coming up-to-speed on EBNF, so I'll need to get some SWE eyes on it once the PR comes in if you do submit one.