armoha / euddraft

System for pluginizing eudplib codes.
Other
29 stars 4 forks source link

Rewrite Python 3.11+ linetable calculator in Rust #127

Closed armoha closed 7 months ago

armoha commented 8 months ago

Calculating linetable format takes significant time in compilation for epScript-heavy project (~10% of total time): image

https://github.com/armoha/eudplib/blob/02bbbe6c04bcebcaa6e5a69c96031e53c36eb630/eudplib/epscript/linetable_calculator.py

Problem

https://github.com/armoha/eudplib/blob/02bbbe6c04bcebcaa6e5a69c96031e53c36eb630/eudplib/epscript/linetable_calculator.py#L91-L96

# eudplib/eudplib/epscript/linetable_calculator.py
def gen_new_opcode(
    code_object: types.CodeType,
    code_options: dict[str, Any],
    keys: list[str],
    ep_lineno_map: Callable[[int], int],
) -> types.CodeType:
    ep_firstlineno = ep_lineno_map(code_options["co_firstlineno"])
    linetable = assemble(
        get_instructions(code_object),  # <- disassembles code_object for every epScript files...
        ep_firstlineno,
        ep_lineno_map,
    )

We should not use dis.get_instructions(code_object), and instead need to deal with only linetable and epScript line number mapping.

구현 전략

  1. Rust한테 (1)pyc->eps 줄 번호 매핑과 (2) codeobj.linetable, codeobj.firstlineno 를 전달한다
  2. Rust에서 linetable을 디코드해서 codeobj 줄번호를 가져오고 eps줄번호로 대응시킨다
  3. 다시 linetable을 인코딩해서 파이썬한테 넘긴다
armoha commented 8 months ago

Rust에 __epspy__ 경로랑 codeobj.linetable만 전달하고, (Line ##) 읽고 epspy -> eps 라인 넘버 매핑 생성하는 것도 Rust 쪽에서 하는 게 나을 듯

armoha commented 8 months ago

https://github.com/PaddlePaddle/PaddleSOT/blob/develop/docs/compat/python311/co_linetable.md?plain=1 I think this is best documentation for co_linetable