search

microsoft / Kusto-Query-Language

Kusto Query Language is a simple and productive language for querying Big Data.
Apache License 2.0
510 stars 97 forks source link

Grammar File #56

Closed JasonKeirstead closed 1 year ago

JasonKeirstead commented 2 years ago

Is there any official Microsoft-provided Grammar file for Kusto? I have been unable to locate one. There are various third-party / unofficial ones bouncing around Github for various parsers, but it would be much simpler to support Kusto with an official and known accurate reference grammar file.

mattwar commented 2 years ago

Are you still interested in a grammar file? I don't have one to share, but I am interested in the reason you might need one.

mavam commented 2 years ago

Same here, I'd love to see an official language spec to target Kusto as backend.

On a related note: for those asking here, https://substrait.io/ might be worthwhile looking into.

JasonKeirstead commented 2 years ago

@mattwar Yes we would still be interested in one, so that one can create their own parser.

mattwar commented 2 years ago

@JasonKeirstead, you realize this repo is a Kusto parser right? :-) I put the source here primarily so you would not need to write your own parser.

We do have an ANTLR3 grammar file we still use for the server, that eventually this parser will replace. However, that file is interlaced with code that is not public, so I'm not able to share it and I don't know how useful to you it would be in that form.

mattwar commented 2 years ago

Same here, I'd love to see an official language spec to target Kusto as backend.

@mavam, what does it mean to you to 'target Kusto as backend?'. How is the language specification going to help you do that?

JasonKeirstead commented 2 years ago

@mattwar Right. If one was targeting a C# runtime then obviously they would consume this parser. But if you need to build a Kusto parser in a different language or runtime, a grammar file is invaluable. Usually for a language, the grammar file is the normative reference vs a parser since parsers can be generated from grammars.

It's probably obvious to read between the lines - we are looking at a Kusto parser in another language.

mattwar commented 2 years ago

@JasonKeirstead, I'll see if I can generate ANTLR4 files from the ANTLR3 files, so it will have the same grammar but no code.

kashwy commented 2 years ago

@mattwar , same here, I'd love to see grammar files . have you generated the ANTLR4 files?

Thanks

mavam commented 2 years ago

@mavam, what does it mean to you to 'target Kusto as backend?'. How is the language specification going to help you do that?

@mattwar if I have a Kusto query, why not run it on other data storage systems outside of Azure? In principle, I can also use Kusto to query a local text file. It just need to write a tool (= requires grammar) that takes a Kusto query and translates it into processing instructions. If the grammar was exposed, 3rd-party tooling can extend the reach of Kusto much more easily.

ledbit commented 2 years ago

@mattwar +1 on the grammar files. Thanks!

mattwar commented 1 year ago

I was able to generate ANTLR4 versions of the grammar, but I had to do a bunch of hand modifications to get it to work. This would not allow it to be automated when changes are made to the original file.

JasonKeirstead commented 1 year ago

@mattwar That'd great news! Any way to share the work in progress? Maybe we can collaborate somehow?

mattwar commented 1 year ago

I have a discussion about this with the team. The decision was made not to release a separate grammar file. As it stands now, the code in this repo is the only published version of the language grammar.

jodybrownell commented 1 year ago

Is there a reason the grammar is not going to be published? Can one be contributed, and community maintained, or will it get rejected?

avneraa commented 1 year ago

Because we do not use Antlr internally for the grammar, as Matt said the code is the grammar so maintaining an external file will require a large investment with not much benefit.

mattwar commented 1 year ago

I'm closing this issue. Feel free to open a discussion if you want to continue discussion this topic.

arpieb commented 5 months ago

IMHO a language without a grammar, locked to a single programming language, is pretty much useless to the OSS community as a whole. There are plenty of cases where one would want to build a parser and not be required to use a specific platform to do so. Sad to see no public grammar will be made available.