Text Based Protocol Buffer Support

thejustinwalsh commented 4 years ago

The text based protocol buffer format does not highlight. It seems there was at least recognition of the elusive text based protocol buffer format back in https://github.com/github/linguist/issues/3816 when it was open.

I am unable to find any formal documentation on the format itself in the Protocol Buffer specification. However, there are methods within the Protocol Buffer API that handle encoding and decoding to and from the text based format.

You can use protoc --encode and protoc --decode on the command line to test.

Here is great example of the output format: https://github.com/google/nvidia_libs_test/blob/master/cudnn_benchmarks.textproto

Some additional information: https://gist.github.com/henridf/704c1c812f04a502c1c26f77a739090b

Currently I can mark the type as JSON5 and have somewhat acceptable highlighting based on the keys and values, however the format has a few deviations from what I can tell...

keys do not need to be wrapped in an object, they can exist at the top level
Comments are denoted by a #
Commas are not used to denote an entry in an object, newlines are used in their place.
Commas are used in array entries
Objects do not use a colon for it's key name when defined. i.e object { ... } vs object-key: { ... }

Preliminary Steps

Include a text file in your repository including a text encoded protocol buffer message with the extension .proto or mark the file extension in .gitattributes with linguist-language=Protocol-Buffer.

Please confirm you have...

[✔️] reviewed How Linguist Works,
[✔️] reviewed the Troubleshooting docs,
[✔️] considered implementing an override,
[✔️] verified an issue has not already been logged for your issue (linguist issues).

Problem Description

I am not necissarily advocating that the Protocol-Buffer type support this format under the entry for proto files, but rather advocating that there is a human readable, syntax highlight-able format that is emitted and would make it easier to read and diff were it supported.

Protocol-Buffer-Text, textproto, or something of the like seems like the likely candidate for detection.

URL of the affected repository:

https://github.com/ts-defold/tsd-template

Last modified on:

Expected language:

n/a

Detected language:

n/a

lildude commented 4 years ago

Looks like these files are quite common on GitHub (most files appear to be vendored and generated from a quick look) so we'd certainly welcome a PR adding support.

thejustinwalsh commented 3 years ago

Is there a guideline for what the repository of the grammear should look like? The contributing guidelines seem to suggest you just point it at a repo that contains a textmate grammar, and it takes it from there?

From the list of already included ones I see lots of Atom plugins, however my grammar I would like to add in the PR is a VS Code plugin. (https://github.com/thejustinwalsh/textproto-grammar). Just wanting to verify that I am setting this up correctly.

lildude commented 3 years ago

Just wanting to verify that I am setting this up correctly.

You are. VSCode grammars are TextMate compatible and you'll see we have plenty other VSCode grammars included.

You can also test your grammar to see how it'll appear on GitHub using Lightshow at https://github-lightshow.herokuapp.com/

hickford commented 3 years ago

Adding textproto support might now be possible using https://github.com/thejustinwalsh/textproto-grammar . Thanks @thejustinwalsh !

Example of this working https://github-lightshow.herokuapp.com/?utf8=%E2%9C%93&scope=from-url&grammar_format=auto&grammar_url=https%3A%2F%2Fgithub.com%2Fthejustinwalsh%2Ftextproto-grammar%2Fblob%2Fmain%2Fsyntaxes%2Ftextproto.tmLanguage.json&grammar_text=&code_source=from-url&code_url=https%3A%2F%2Fgithub.com%2Fgoogle%2Fnvidia_libs_test%2Fblob%2Fmaster%2Fcudnn_benchmarks.textproto&code=

thejustinwalsh commented 3 years ago

@hickford Yes, I was working out some edge case alternative syntax styles and wanting to contribute, but dropped the ball while I was switching jobs. I think this is just on me to finish up the guidelines and push in the PR.

hickford commented 3 years ago

Brilliant! Oops I missed that you opened this issue; obviously you are aware of your own project.

thejustinwalsh commented 3 years ago

No worries. Thanks for the reminder to get this pushed!

github-linguist / linguist