search

shnewto / bnf

Parse BNF grammar definitions
MIT License
256 stars 22 forks source link

How to parse a certain text according to a bnf grammar and get a list of token #67

Closed pankajrlal closed 2 years ago

pankajrlal commented 3 years ago

The examples provided take a bnf grammar description and then are able to generate text that matches the grammar. How do I give the grammar a string compliant with bnf specification and then get a list of tokens out? This is something that pyparsing does.

shnewto commented 3 years ago

Hi @prmammoth that's a feature I've been thinking about implementing for awhile now, I do think it'd be a valuable addition to this library. Taken in steps, I think the most straight forward approach is to implement an Earley parser that would be able to validate that input is or is not valid for the given grammar, then use the result to generate a "parse forest" of possible paths to that input from the grammar 🤔

shnewto commented 3 years ago

I might take a swing at implementing this over the next week or two, if I have anything interesting to report I'll fill ya in here.

shnewto commented 3 years ago

Quick update, I haven't forgotten abt this. I've got a prototype working for grammars accepting/rejecting sentences but am still chipping at returning the parse trees that show how a sentence fits the grammar. More specifically, I have an initial Earley parser worked out, just haven't wrapped up the logic to take the result and create a parse forest from it.

In case anyones interested in what I mean by "I have an initial Earley parser worked out", I'm hacking around on it all over here https://github.com/shnewto/earley. Fwiw, I suspect the tests there would be more informative than the prototype code 🤷‍♂️

artman41 commented 2 years ago

@shnewto any update on this? Has the feature been abandoned?

shnewto commented 2 years ago

Hi @artman41 thanks for pinging, this isn't abandoned but it's also not implemented yet. I'm hoping to have some happy updates regarding it in the next couple weeks.

CrockAgile commented 2 years ago

just wanted to share that I've been poking at this feature recently. nothing to show yet, but it has not been forgotten!

shnewto commented 2 years ago

@prmammoth @artman41 we've merged the implementation of this feature and are drafting the release for later today! it was a long time coming 😄 check out the example in the readme and reach out or create another issue if you have any questions!

pankajrlal commented 2 years ago

Wonderful. Thanks a lot