__debug__ enhancement and parse time constants

[nblzv]

In the spirit of the efforts being made by the faster-cpython team to improve performance, I think there is some low hanging fruit it terms of improving runtime performance by... not running code in the first place, since the fastest code is the one that never runs!

The lowest of all the hanging fruits would be turning debug into an int, allowing it to take on multiple values and as such allowing multiple debug levels to checked for. This would require minimal change to the codebase and command line option to support a number after the -O argument and the same number being stored for the .opt part of the .pyc files, i.e. -O3 would replace __debug__ with 3 instead of the current bool approach. This approach would allow developers to write as much debug code as they need, while also having it get compiled out completely when it's no needed, as opposed to the current approach of toggling if 0: blocks everywhere or commenting out the different tiers of debug code when not in use.

if __debug__:
    print("Debug mode")
if __debug__ >= 2:
    print("Verbose debug mode")
if __debug__ >= 3:
    print("Paranoid debug mode")

---

And while that would already bring a lot of value by itself with minimal changes, the natural extension to it is actual compile time constants, i.e. __debug__ but with different names, which would make things a lot more readable. This fruit is a few branches higher on the tree. One approach to implementing these compile time constants would be to add a const keyword which declares all the compile time constants used in the file, which would be used for syntax error purposes. And during compilation they would get turned into an actual constant similarly to __debug__, with the caveat that consts would evaluate to False if not explicitly defined. As to how they would be defined, might as well use the -O option again, i.e. -OFOO would define FOO as True during compilation, i.e.:

const FOO, BAZ

# Only one of the code paths should remain, based on whether -OFOO was passed when running python
if FOO:
    print("Foo defined")
else:
    print("Foo not defined")

# If not declared by -OBAZ, should be treated as False and the if should be compiled out completely
if BAZ:
    print("We got baz")

This change would also require changes to the .pyc files, either by storing a hash/all the compile time constants used in the file itself, or just generating a completely different .pyc with a hash of all the compile time constants in place of the .opt part of the filename.

---

And the natural extension to that would be supporting integer constants similar to the debug extension, or better yet and builtin type -- int, float or string (constants only of course). These again require a change to the command line option in the form of -OFOO=3, but if the previous two changes are implemented, shouldn't require too much compiler changes. \ \ All of these changes combined would more of less bring python on par with C's #if/#ifdef statements and allow for copious amounts of debug code to remain in the codebases with zero cost for the (optimized) runtime, while only incurring a small overhead for the startup (unless already cached).

[stevendaprano]

This is a lot of changes, and not very straight forward. Have you discussed this elsewhere to get community feedback? Such as the python-ideas mailing list or the Discuss?

This would involve:

• changes to the interpreter;
• changes to the byte-code and the compiler;
• changes to the command line switches;
• changes to the naming of .pyc files;
• introduction of a const keyword;
• changes to the execution model of Python to support constants;
• and possibly other changes as well.

I think you're going to need to write a PEP.

[nblzv]

It has not been discussed elsewhere as far as I've seen, and since I didn't know the channels/steps one needs to take in order to bring up an issue, I ended up posting an issue on here.

I am aware that the second and third stages would require actual work, but the first seems rather straightforward and would bring 80% of the whole feature for 20% of the effort. Of course it would be very nice to for the feature to be complete with const and str/float support too, but some janky workarounds could be made on the user side by abusing __debug__s value.

As far as changing __debug__ from a bool to an int and getting constant folded at compile time as such -- I don't think requires a whole PEP, for the rest I can see the case for it.

Edit: I just noticed that the constant folding in general seems to need some work:

>>> dis.dis("1 if __debug__ else 2")
  0           0 RESUME                   0

  1           2 LOAD_CONST               1 (1)
              4 RETURN_VALUE

# Should get compiled to the same, but doesn't
>>> dis.dis("1 if __debug__ == True else 2")
  0           0 RESUME                   0

  1           2 LOAD_CONST               0 (True)
              4 LOAD_CONST               0 (True)
              6 COMPARE_OP               2 (==)
             12 POP_JUMP_IF_FALSE        2 (to 18)
             14 LOAD_CONST               1 (1)
             16 RETURN_VALUE
        >>   18 LOAD_CONST               2 (2)
             20 RETURN_VALUE

Similarly == False also produces the comparison, as well as comparing to a 0/1 int literal.

[iritkatriel]

I'm not sure I see why this is a performance feature.

Performance matters only for the no-debug mode. If you want different debug modes, you can check which debug mode you are in inside the "if debug", where performance doesn't matter:

if __debug__:
    if FOO:
        print("Foo defined")
    else:
        print("Foo not defined")