Investigate ways to optimize the performance of `tja2fumen`

[vivaria]

tja2fumen is naturally a bit slow because it's written in Python. It takes around ~0.35s to convert a TJA. Here are some sources of slowdown + potential improvements:

• PyInstaller overhead
  • [ ] PyInstaller --onefile. Using --onedir causes a 50% speedup (0.34s -> 0.17s)
• Only about ~0.05s of that is tja2fumen itself, with the following bottlenecks:
  • [ ] 0.01s is spent on reading values from the CSV LUTs
  • [ ] 0.005s is spent on regex matching
  • [ ] 0.005s is spent writing values to .bin files in write_struct()

The first option seems like a good one to pursue for upstream packages. But, I'm not sure about whether it's worth it to pursue fixes for the latter issues.

(Going from 0.17s to 0.16s doesn't feel all that consequential? Even if someone converts the entirety of ESE (~2200 songs), they'll only go from 6m14s to 5m52s (-22s) That's somewhat significant, but the user will still have to wait ~6m either way.)

[vivaria]

PyInstaller overhead

Also worth looking into: Nuitka over Pyinstaller

[vivaria]

I tested mypyc by adding the following to pyproject.toml:

[build-system]
requires = ["setuptools >= 40.8.0", "wheel", "mypy"]

Then updating setup.py to the following:

from mypyc.build import mypycify

if __name__ == "__main__":
    setup(ext_modules=mypycify([
        'src/tja2fumen/__init__.py',
        'src/tja2fumen/classes.py',
        'src/tja2fumen/constants.py',
        'src/tja2fumen/converters.py',
        'src/tja2fumen/parsers.py',
        'src/tja2fumen/writers.py',
    ]))

---

I tested Nuitka by adding the following to pyproject.toml:

[build-system]
requires = ["setuptools>=42", "wheel", "nuitka", "toml"]
build-backend = "nuitka.distutils.Build"

# Data files are to be handled by setuptools and not Nuitka
[tool.setuptools.package-data]
tja2fumen = ['tja2fumen/hp_values.csv']

Then running the following command:

python -m nuitka \
       --onefile src/tja2fumen/__init__.py \
       --include-package=tja2fumen \
       --include-data-files=src/tja2fumen/hp_values.csv=tja2fumen/ \
       --clang

---

I have a feeling I'm doing something wrong, as neither was any faster than the typical Python + Pyinstaller route. I'll need to investigate in more detail.