Mini-LISP JS Parser - Symbol defined exactly as in the C tokenizer

[yossigil]

Should be easy, but critical. We need to be able to use weird symbols such as ~!, and 12. Crucial for assignments to extend mini lisp, e.g., with arithmetic.

[OfirMarkowitz1]

Done

On Wed, 28 Jul 2021 at 17:39, Yossi Gil @.***> wrote:

Should be easy, but critical. We need to be able to use weird symbols such as ~!, and 12. Crucial for assignments to extend mini lisp, e.g., with arithmetic.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/yossigil/safot/issues/430#issuecomment-888365365, or unsubscribe https://github.com/notifications/unsubscribe-auth/APCETIGMBFUOE4HGOTOWTILT2AJBTANCNFSM5BD7ZNOA .

[yossigil]

Next time, tag the commit with issue number, e.g., git comment -m "fixes #430" or if it just deals with the issue, something like git comment -m "more work on #430"

if you do so, you will save yourself the trouble of updating the issue; if you write fix or close this will also automatically close the issue. Also, no need to mention me, at least for now, since I try to follow everything. If I miss something, feel free to mention me.

[OfirMarkowitz1]

All of the issues I opened, beside the one involving the C project compilation error, is in the project's sprint, in the "Ready for review" column. I was meaning to close them after your review is over without rejections. I can close them immediately when I put them for review, if it's preferable. Thanks for the advice on git! It's very helpful.

On Wed, 28 Jul 2021 at 17:56, Yossi Gil @.***> wrote:

Next time, tag the commit with issue number, e.g., git comment -m "fixes #430" or if it just deals with the issue, something like git comment -m "more work on #430"

if you do so, you will save yourself the trouble of updating the issue; if you write fix or close this will also automatically close the issue.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/yossigil/safot/issues/430#issuecomment-888379209, or unsubscribe https://github.com/notifications/unsubscribe-auth/APCETIFKEUQQ5XQVK4434YDT2ALCVANCNFSM5BD7ZNOA .

[yossigil]

I checked the code. Not right still. Please do something like [^()\[\].;] There are six or so reserved characters used as tokens. Spaces are ignored. So are comments. I beg that you examine the tokenizer in C. It is all there, and would have saved all this interaction.

[yossigil]

Exactly means exactly

[yossigil]

Take a look at the PEGS file.

[OfirMarkowitz1]

Firstly, I did examine the Tokenizer h and cc files carefully before editing the pegjs file. Secondly, I think you made a mistake - you replaced the quote sign e.g. ', with semicolon, e.g. ; - correct me if I'm wrong. Now for the essence of the matter, the pattern [^()[].\'] doesn't work in PEG.js. That's why I used unicode ranges, the only thing that I found that seemed to work. ASCII didn't work, only unicode, i.e. [\u0021-\u0026]. If you check out the unicode table, you will find that this is equivalent to what you were suggesting, aside from the semicolon mistake.

Ofir

On Thu, 29 Jul 2021 at 09:21, Yossi Gil @.***> wrote:

Take a look at the PEGS file.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/yossigil/safot/issues/430#issuecomment-888836612, or unsubscribe https://github.com/notifications/unsubscribe-auth/APCETICSVXUY3UO662JL5YDT2DXOLANCNFSM5BD7ZNOA .

[OfirMarkowitz1]

I managed to shorten the unicode a bit by using [^...] as you suggested, by still needed to use unicode, and needed to added all the unprintable, whitespace and non-ASCII characters to the pattern in order for it to work.

[yossigil]

Please give reference to the commit you did

[OfirMarkowitz1]

84dc4a3

On Sun, 1 Aug 2021 at 14:16, Yossi Gil @.***> wrote:

Please give reference to the commit you did

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/yossigil/safot/issues/430#issuecomment-890500070, or unsubscribe https://github.com/notifications/unsubscribe-auth/APCETIEWQ3JOWADRCARCX4LT2UUKLANCNFSM5BD7ZNOA .

[OfirMarkowitz1]

I managed to improve this further. Turns out the problem was only because I have to add the whitespace, unprintable and non-ASCII characters to the Symbol regex. See commit - d48f25c

On Sun, 1 Aug 2021 at 14:24, Ofir Markowitz @.***> wrote:

84dc4a3

On Sun, 1 Aug 2021 at 14:16, Yossi Gil @.***> wrote:

Please give reference to the commit you did

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/yossigil/safot/issues/430#issuecomment-890500070, or unsubscribe https://github.com/notifications/unsubscribe-auth/APCETIEWQ3JOWADRCARCX4LT2UUKLANCNFSM5BD7ZNOA .

[OfirMarkowitz1]

It would be better to find a way for the "_" variable regex, e.g. the characters to ignore, to have precedence over the Symbol, such that we wouldn't need to add those characters to the Symbol regex. Simply putting the characters to ignore rule above the Symbol rule doesn't seem to do it (even though rules order makes difference in PEG grammars).

On Sun, 1 Aug 2021 at 14:32, Ofir Markowitz @.***> wrote:

I managed to improve this further. Turns out the problem was only because I have to add the whitespace, unprintable and non-ASCII characters to the Symbol regex. See commit - d48f25c

On Sun, 1 Aug 2021 at 14:24, Ofir Markowitz @.***> wrote:

84dc4a3

On Sun, 1 Aug 2021 at 14:16, Yossi Gil @.***> wrote:

Please give reference to the commit you did

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/yossigil/safot/issues/430#issuecomment-890500070, or unsubscribe https://github.com/notifications/unsubscribe-auth/APCETIEWQ3JOWADRCARCX4LT2UUKLANCNFSM5BD7ZNOA .

[yossigil]

Or, perhaps some very special Unicode classes/categories?