Support for escaped quotes

dmfxyz commented 2 years ago

PR includes changes from #22, that one can be merged first to see true diff.

There is a problem with the ebnf module we use -- it auto escapes strings somewhere in its implementation. For example the string:

`{key "val\""}`

is re-escaped to

`{key "val\\""}`

within the ebnf module itself.

So to address this I implemented

introduce unsafe type (only supports escaped quotes for now)
Only allowing unsafe to exist within quote
parsing quote child by child, and replacing the re-escaped characters only if the child is of type unsafe

We could redefine the grammar to expect the double \\, but this would be an inaccurate grammar tailored to our specific parser.

Added new file based tests that are all passing. If we have a list of other characters that should be escaped, I can add them too.

nmushegian commented 2 years ago

This is a good find. Thank you, please file an issue in their repo and point here

dmfxyz commented 2 years ago

@nmushegian see here for more issue and example: https://github.com/lys-lang/node-ebnf/issues/41

nmushegian commented 2 years ago

This is too many changes in one PR

nmushegian / jams