tsoding / porth

It's like Forth but in Python
629 stars 50 forks source link

macros are TOO safe #74

Open drocha87 opened 3 years ago

drocha87 commented 3 years ago

Recently I was playing with a new keyword defined so we can check up front if a word is already defined, so we can for example avoid including a file twice or change the library behavior if some macro is already defined.

But for this happen I should have the ability to redefine a macro, something like this.

defined std.porth 0 = if
macro std.porth 1 end
...
end // defined std.porth

or

defined SOMETHING 0 = if   // if SOMETHING is not defined
    macro SOMETHING 69 end // define a default value in case of SOMETHING is undefined 
end

I don't know if this new keyword will be useful, but I think the ability to redefine a macro should be possible. What you guys think about it? Should macros behave like a constant expression?

IHateYourCode commented 3 years ago

I think it's the other way around, I'd say, being able to redefine macros should be the default and the actual question would be, how to not have the compiler stop on including the same file multiple times so you can safely include file with default macros. If you still want the advanced checking functionality for files with multiple different 'headers' I'd opt for a more intuetive syntax, something like

defmacro SOMETHING 69 end

or since you used a seperate keyword

default macro SOMETHING 69 end

in regards to default definition. For an actual definition checking keyword I'd suggest isDefined / isdefined but with a syntax more in line of what is currently already in place

std.porth isdefined if
   ...
end

which could be used to make a std macro like

macro ifndefined isdefined 0 = if end
macro ifndef ifndefined end

macro ifdefined isdefined if end
macro ifdef ifdefined end
std.porth ifdefined
   ...
end

if it weren't for #72

drocha87 commented 3 years ago

@IHateYourCode currently the lexer will try to expand the word as soon as it is found. So std.porth isdefined in your example will always fail before hit the isdefined keyword, since std.porth could be undefined.

But the point in this issue is not about introducing a new keyword or not, is just about if we should be able to redefine or not an already defined macro. If the author understands that it is worth to enable this behavior to redefine a macro, he or someone with a PR should just comment the following line:

-                if token.value in macros:
-                    print("%s:%d:%d: ERROR: redefinition of already existing macro `%s`" % (token.loc + (token.value, )), file=sys.stderr)
-                    print("%s:%d:%d: NOTE: the first definition is located here" % macros[token.value].loc, file=sys.stderr)
-                    exit(1)
+                # if token.value in macros:
+                #     print("%s:%d:%d: ERROR: redefinition of already existing macro `%s`" % (token.loc + (token.value, )), file=sys.stderr)
+                #     print("%s:%d:%d: NOTE: the first definition is located here" % macros[token.value].loc, file=sys.stderr)
+                #     exit(1)

Something like this.

IHateYourCode commented 3 years ago

The lexer doesn't look ahead at all .. O.O

Anyways, on the topic of macro redefinition, yes, one should be able to redefine them.

Being able to do so not only allows you to safely make sure they have been declared with default macros (without your compilation process stopping), but it also allows for postfixes / overwriting existing ones to for example enhance one library with the other, or in a simpler case, being able to use a lib without fearing to have certain macro names blocked by it.

drocha87 commented 3 years ago

The lexer doesn't look ahaed at all .. O.O

That's exactly what I said, and that's is the reason your example std.porth isdefined if ... end doesn't work. When the lexer tries to expand std.porth it could be undefined. But if you have something like defined in my example you can just pop the next token and check if it exists at compile time.

elif token.value == Keyword.DEFINED:
    token = rtokens.pop() # looking ahead for the next token
    if token.value in macros or token.value in INTRINSIC_NAMES or token.value in KEYWORD_NAMES:
        # word is defined as macro or intrinsic or keyword
    else:
        # word is not defined at all

This is exactly the same approach the keyword macro uses to check if the macro name is already defined.

But anyway, this is not the point discussed in this issue. Lets wait for the author point of view on macro redefinition.

IHateYourCode commented 3 years ago

That's exactly what I said, and that's is the reason your example std.porth isdefined if ... end doesn't work. When the lexer tries to expand std.porth it could be undefined.

I know, I was just acknowledging it with

The lexer doesn't look ahead at all .. O.O

Yeah, hopefully he sees this soon.

IHateYourCode commented 3 years ago

Isn't the compiler structured in a way where redefinition would also allow for removal of macros and thus allow for temporary macros? If this was the case, that's be fantastic, since that would basically be like having private functions, now I want it even more!

Domkeykong commented 2 years ago

there is a much more severe problem that is macros are defined in the compilation step that is before the program is executed so this wouldn't make any sense. you need a way to have something like preprocessor macros/keywords.