pola-rs / polars

Dataframes powered by a multithreaded, vectorized query engine, written in Rust
https://docs.pola.rs
Other
27.22k stars 1.67k forks source link

feat(python): Start updating `BytecodeParser` for Python 3.13 #16304

Closed alexander-beedie closed 1 month ago

alexander-beedie commented 1 month ago

Looking to get ahead of this one early. Slightly painful to analyse in depth at the moment as the debugger promptly crashes on attach to the first 3.13 beta, but there's always the good old "print" statement, so...😎

Updates

Can't test with numpy translation yet, but these two small updates alone take the related tests from "almost 100% of them fail" to "only four fail", which is a good sign that the changes are going to be minor.

As the betas are released I'll make incremental adjustments so we don't have to scramble come October.

Example

from polars._utils.udfs import BytecodeParser

bp = BytecodeParser(
    function = lambda x: not (x > 1) or x == 2,
    map_target = "expr",
)
bp.dis()
# 2         RESUME                0
#           LOAD_FAST             0 (x)
#           LOAD_CONST            1 (1)
#           COMPARE_OP          148 (bool(>))    <<< new to 3.13; 'bool' wrapper
#           UNARY_NOT
#           COPY                  1
#           TO_BOOL                              <<< new opcode
#           POP_JUMP_IF_TRUE      5 (to L1)
#           POP_TOP
#           LOAD_FAST             0 (x)
#           LOAD_CONST            2 (2)
#           COMPARE_OP           72 (==)
#    L1:    RETURN_VALUE

bp.to_expression("x")
# ~(pl.col("x") > 1) | (pl.col("x") == 2)

Previously:

AssertionError: unrecognized opname
# Instruction(
#   opname='TO_BOOL', opcode=40, arg=None, argval=None, argrepr='', offset=14,
#   start_offset=14, starts_line=False, line_number=2, label=None,
#   positions=Positions(lineno=2, end_lineno=2, col_offset=15,
#   end_col_offset=36), cache_info=[('counter', 1, b'\x00\x00'),
#   ('version', 2, b'\x00\x00\x00\x00')],
# )
codecov[bot] commented 1 month ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 80.79%. Comparing base (6d48c11) to head (fc56d78). Report is 3 commits behind head on main.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #16304 +/- ## ========================================== + Coverage 80.78% 80.79% +0.01% ========================================== Files 1393 1393 Lines 179362 179363 +1 Branches 2921 2921 ========================================== + Hits 144894 144924 +30 + Misses 33965 33936 -29 Partials 503 503 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

ritchie46 commented 1 month ago

Early preparation. :ok_hand: