search

lcompilers / lpython

Python compiler
https://lpython.org/
Other
1.5k stars 158 forks source link

Request: Powerful use-case for macros #1972

Open rebcabin opened 1 year ago

rebcabin commented 1 year ago

Issue #1908 is a plea for CTR in LPython in the distant future. CTR enables macros. However, even a special-purpose, simplified macro system à la macropy would be immediately useful for cases like the following:

    assert a_char == '(', "a_char == '('"

With even very simple macros, this could be shortened to 

    assertm a_char == '('
certik commented 1 year ago

Or in this case we can simply improve our backend to print a much better error message for assert a_char == '(' that includes the string itself, line number and a stacktrace.

rebcabin commented 1 year ago

wouldn't that be a language extension, thus violating one of our tenets? Macros can be achieved in ordinary CPython!

Thirumalai-Shaktivel commented 1 year ago

Or in this case we can simply improve our backend to print a much better error message for assert a_char == '(' that includes the string itself, line number and a stacktrace.

Yup, currently we have the following:

$ lpython examples/expr2.py -g --debug-with-line-column
  File "examples/expr2.py", line 2
    assert t == 'c'
AssertionError

We can improve this or give it a try to use macros.

I opened Issue #2009, which makes this use-case for macros go away :) However, the discussion is still valuable, because it clarifies use cases for CTR, in general, from use cases for macros, in particular.

certik commented 1 year ago

The tenet is that if it runs in LPython, it will run in CPython and produce the same result. Our exceptions and failure modes do produce slightly different results (the stacktrace might look slightly differently, depending on how you compile it). I think we have some wiggle room there, and I thought a nicer assert error message would still fit the tenet. But I agree it's borderline and one can argue against doing that.

The issue with macropy is that last commit was 6 years ago and I worry it will take the same route as the goto, that we eventually removed, because a new CPython version broke it.

I feel macropy is actually extending Python, just like goto. It's extending Python with "hacks" that it was not meant to do. It works, until it gets broken by upstream. I would only commit to CPython as released upstream and its features.

We do support somethings like structs and unions, but we do it via ctypes, which are supported upstream, so I think that's fine.

So I would not do macros, since Python doesn't have macros, so it would be against the tenets. But making an error message from assert slightly nicer I think would be fine and within tenets. :)

rebcabin commented 1 year ago

I disagree with your last point (again, very long term). Macropy employed a standard feature of LPython, that the AST is exposed at import time. Yes, macropy is out of maintenance, but that's beside the point. Direct AST rewrites at import time are still part of the Python standard, sfaik.

I'm not saying we implement macropy the way we did goto. I am proposing that we embrace AST-rewriting as part of Python and integrate it with all the rest of LPython's unique symbolic capabilities: ASR, reflectively and SymPy / SymEngine integration. Macros are just one exploitation of AST rewriting. Rewriting is MUCH deeper and, again, very long term, should be a powerful differentiator of LPython -- without extending Python! -- from other compilers and implementations.

On Wed, Jun 21, 2023 at 7:48 AM Ondřej Čertík @.***> wrote:

The tenet is that if it runs in LPython, it will run in CPython and produce the same result. Our exceptions and failure modes do produce slightly different results (the stacktrace might look slightly differently, depending on how you compile it). I think we have some wiggle room there, and I thought a nicer assert error message would still fit the tenet.

The issue with macropy is that last commit was 6 years ago and I worry it will take the same route as the goto, that we eventually removed, because a new CPython version broke it.

I feel macropy is actually extending Python, just like goto. It's extending Python with "hacks" that it was not meant to do. It works, until it gets broken by upstream. I would only commit to CPython as released upstream and its features.

We do support somethings like structs and unions, but we do it via ctypes, which are supported upstream, so I think that's fine.

So I would not do macros, since Python doesn't have macros, so it would be against the tenets. But making an error message from assert slightly nicer I think would be fine and within tenets. :)

— Reply to this email directly, view it on GitHub https://github.com/lcompilers/lpython/issues/1972#issuecomment-1600981310, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABSRR2LWWIMFHGEWRJH6V3XMMCU7ANCNFSM6AAAAAAZNYBGBM . You are receiving this because you authored the thread.Message ID: @.***>

certik commented 1 year ago

Macros are powerful, but remember that AST changes from CPython version to version, and there are even some very slight differences with Python AST and LPython AST. So if we do it, we must tackle these challenges.

rebcabin commented 1 year ago

I totally agree! That's why macropy went out of maintenance! This is not a small job, and for the far future if ever.

I'm NOT selling macros, in particular. I AM selling CTR (Conditional Term Rewriting) as something we should take very seriously to propel LPython into the world's BEST symbolic programming language SOMEDAY :) Beyond Mathematica. My plea is in Issue #1908.

On Wed, Jun 21, 2023 at 12:07 PM Ondřej Čertík @.***> wrote:

Macros are powerful, but remember that AST changes from CPython version to version, and there are even some very slight differences with Python AST and LPython AST. So if we do it, we must tackle these challenges.

— Reply to this email directly, view it on GitHub https://github.com/lcompilers/lpython/issues/1972#issuecomment-1601457271, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABSRR2PVNJHJR7U5DR25OTXMNA7VANCNFSM6AAAAAAZNYBGBM . You are receiving this because you authored the thread.Message ID: @.***>

certik commented 1 year ago

Note that I thought #1908 is at the ASR level, while macropy is at the AST level. Rewriting ASR is powerful. The issue is that it's hard to do in pure Python mode without just calling LPython (but then you can't use debuggers). AST rewriting will work in both, but then it's tied to Python, unrelated to ASR (and unrelated to ASR->ASR passes).

rebcabin commented 1 year ago

yes, I wasn't all that clear about it. Rewriting applied to both AST and ASR, but it's MUCH more interesting at ASR :)

On Wed, Jun 21, 2023 at 1:48 PM Ondřej Čertík @.***> wrote:

Note that I thought #1908 https://github.com/lcompilers/lpython/issues/1908 is at the ASR level, while macropy is at the AST level. Rewriting ASR is powerful. The issue is that it's hard to do in pure Python mode without just calling LPython (but then you can't use debuggers). AST rewriting will work in both, but then it's tied to Python, unrelated to ASR.

— Reply to this email directly, view it on GitHub https://github.com/lcompilers/lpython/issues/1972#issuecomment-1601651834, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABSRRYISVK6JSGC4EWA5CDXMNMZDANCNFSM6AAAAAAZNYBGBM . You are receiving this because you authored the thread.Message ID: @.***>