py_binary launch performance problem on windows

[peakschris]

On windows, with rules_python as module and system interpreter and a simple script:

> timecmd bazel run //:myscript -- --help
command took 5.6s

> timecmd python myscript.py --help
command took 0.5s

Due to the 10x overhead, for such a regularly run tool, users are refusing to run via bazel. I'm guessing the overhead is unzipping, but am not sure. We would prefer to run without zipping if zipping has a 10x overhead.

Full output from bazel run:

INFO: Analyzed target //:myscript (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
Target //:myscript up-to-date:
  bazel-bin/myscript.zip
  bazel-bin/myscript.exe
INFO: Elapsed time: 0.494s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
INFO: Running command line: bazel-bin/myscript.exe --help
Usage: <snip>
command took 5.6s

Relevant options:

.bazelrc

startup --windows_enable_symlinks
common --noenable_runfiles
common --nolegacy_external_runfiles

Module.bazel

bazel_dep(name = "rules_python", version = "0.32.2")
# use system python for performance
register_toolchains("@bazel_tools//tools/python:autodetecting_toolchain")

🐞 bug report

Affected Rule

py_binary

Is this a regression?

No

🔬 Minimal Reproduction

I can create this if it would help

🌍 Your Environment

Operating System:

  
Windows
  

Output of bazel version:

  
bazel 7.2.1
  

Rules_python version:

  
0.32.2
  

Anything else relevant?

[aignas]

I am wondering if bazelbuild/bazel#22865 is related. Could you please try if nondblzmod is the same overhead?

Could you also check if precompiling py files to pyc helps (that is a new feature in 0.32.2)?

Other than that, #1653 could be also related.

[peakschris]

Ah, good thought! https://github.com/bazelbuild/bazel/issues/22865 was opened by me, so I have the fix. Sadly the fixed bazel doesn't resolve this - launch time still 5+s

Just to confirm that I have the fix, here are the files in my external dir:

I don't think that #1653 is related because I am using system interpreter, and I have disabled legacy_external_runfiles

I can't understand from the docs what I should change in my py_binary target to enable precompiling. Could you advise?

Thanks, Chris

[peakschris]

Ok, I updated from 0.32.2 to 0.33.2, and the precompiling option started working. But unfortunately no improvement in speed:

INFO: Elapsed time: 0.377s
bazel run //:myscript --@rules_python//python/config_settings:precompile=enabled -- --help
command took 5.61s

[peakschris]

Another datapoint: Using 7z to recursively extract the zip created by py_binary takes 0.64s

[aignas]

What is the performance if you launch the the script directly? I.e. bazel build + executing it instead of bazel run?