Non-optimized Python interpreters make Moulti slow

[MartyLake]

Hello, I really like the principle of that script! However, it is very slow on macOS. not sure why.

script used to time

Here is the one-liner I used:

time (ops=("one" "two"); source ./moulti-functions.bash; if moulti wait -d 1 -m 1; then for needle in "${ops[@]}"; do moulti_exec echo $needle; done; else for needle in "${ops[@]}"; do echo $needle; done; fi)

time with moulti init running in other term

2024-05-06T16:47:41,242301000+02:00
2024-05-06T16:47:41,960421000+02:00

real    0m1.630s
user    0m0.324s
sys 0m0.165s

time without moulti

Giving up.
one
two

real    0m0.178s
user    0m0.041s
sys 0m0.018s

Best,

[xavierog]

Hi,

Thanks for your report. The documentation states:

Moulti's performance has room for improvement.

... so I expected a report with such a title... but not with that exact content. The order of magnitude between the two scenarios is not exactly surprising considering the way moulti_exec works (multiple fork+exec of a Python program) and I get similar figures on my (Linux) machine: 40ms vs 400ms.

However, the absolute numbers you report are indeed suspicious. It feels like something is going wrong in terms of sleep / network timeout. I will investigate that.

[xavierog]

I forgot to ask: what version of Moulti are you using? Did you get troubles with moulti_duration and its use of date?

[MartyLake]

Yep, I had to use gdate instead date of macOS for the script to work.

[xavierog]

Let's focus on moulti wait -m 1 (without any Moulti instance running) as this command best reflects Moulti's minimal startup time. The idea here is to use hyperfine (available through brew) to get more specific figures.

Could you please run the commands below and report their output here?

hyperfine --warmup=2 --runs=400 --ignore-failure --shell=default 'python3 -c "import sys; print(sys.argv)" a b c'
hyperfine --warmup=2 --runs=400 --ignore-failure --shell=none    'moulti wait -m 1'

[MartyLake]

There you go:

/tmp$ hyperfine --warmup=2 --runs=400 --ignore-failure --shell=default 'python3 -c "import sys; print(sys.argv)" a b c'
Benchmark 1: python3 -c "import sys; print(sys.argv)" a b c
  Time (mean ± σ):      96.8 ms ±   1.6 ms    [User: 12.8 ms, System: 10.2 ms]
  Range (min … max):    91.1 ms … 105.9 ms    400 runs

/tmp$ hyperfine --warmup=2 --runs=400 --ignore-failure --shell=none    'moulti wait -m 1'
Benchmark 1: moulti wait -m 1
  Time (mean ± σ):     129.3 ms ±   3.9 ms    [User: 38.3 ms, System: 15.1 ms]
  Range (min … max):   121.6 ms … 173.4 ms    400 runs

  Warning: Ignoring non-zero exit code.
  Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.

[xavierog]

Thank you!

Although there still is room for performance improvement in Moulti[1], this issue is now "Python is slow".

Unless your machine is very old (is it?), the most likely explanation is that you installed a non-optimized version of Python through brew -- source: https://www.reddit.com/r/Python/comments/oouh9a/why_is_python_so_much_slower_on_macos/

A possible workaround is to:

1. download and install the official Python package from https://www.python.org/downloads/macos/
2. run the first hyperfine command with that new Python interpreter to ensure we are going the right way
3. reinstall moulti so as to leverage that new Python interpreter: pipx install --python=/path/to/the/new/python/interpreter moulti should do

[1] I think I can shave a few milliseconds by not importing unidiff. And I should have a look at typing-related imports.

[MartyLake]

Well, I did not know that Python could be that fast. After removing asdf and installing the Python Release from your link, here are the results:

/tmp$ hyperfine --warmup=2 --runs=400 --ignore-failure --shell=default 'python3 -c "import sys; print(sys.argv)" a b c'
Benchmark 1: python3 -c "import sys; print(sys.argv)" a b c
  Time (mean ± σ):      12.6 ms ±   0.5 ms    [User: 8.7 ms, System: 3.0 ms]
  Range (min … max):    11.5 ms …  16.1 ms    400 runs

/tmp$ hyperfine --warmup=2 --runs=400 --ignore-failure --shell=none    'moulti wait -m 1'
Benchmark 1: moulti wait -m 1
  Time (mean ± σ):      43.3 ms ±   7.7 ms    [User: 31.9 ms, System: 7.4 ms]
  Range (min … max):    39.5 ms … 169.7 ms    400 runs

  Warning: Ignoring non-zero exit code.
  Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.

[xavierog]

Great!

Your previous Python install was apparently severely unoptimized: my macOS VM yields 22.4 < 25.3±1.6 < 32.6 for Python and 59.7 < 68.0±5.0 < 92.2 for moulti wait using Python 3.12.3 obtained through brew.

These new figures of yours roughly match those on my Linux machine (6.5 < 8.1±1.0 < 12 for Python and 34.1 < 36.9±1.5 < 43.4 for moulti wait). If we focus on the mean time to run moulti wait, you went from 129.3 to 43.3, i.e. -66.5%.

Is the resulting performance acceptable or should we further investigate?

[xavierog]

Yep, I had to use gdate instead date of macOS for the script to work.

FYI, the devel branch no longer depends on GNU date.

[xavierog]

I think I can shave a few milliseconds by not importing unidiff.

Done. Performance-wise, the improvement is meagre (-1.2ms; likely more on non-optimized Python interpreters). But I kept this change since it makes it possible to use most of Moulti even if the unidiff library is not installed.

[MartyLake]

Is the resulting performance acceptable or should we further investigate?

Completely acceptable for me, you can close the issue thanks.

FYI, the devel branch no longer depends on GNU date. Done. Performance-wise, the improvement is meagre (-1.2ms; likely more on non-optimized Python interpreters). But I kept this change since it makes it possible to use most of Moulti even if the unidiff library is not installed.

Sorry but I don’t understand much about python distribution. I installed with pip3 install moulti. How can I switch to the devel branch ?

[xavierog]

I installed with pip3 install moulti. How can I switch to the devel branch ?

If you are not in a hurry, I advise to simply wait until Monday: the contents of the devel branch should land in the master branch and in a 1.8.1 release on Sunday. Additionally, examples/moulti-functions.bash is not shipped through pip/pipx. You likely obtained it through git clone or by browsing GitHub.

If you are in a hurry, this should do:

git clone https://github.com/xavierog/moulti.git -b devel
pip install "$(pwd)/moulti"

[xavierog]

you can close the issue thanks.

This issue is now considered closed -- thanks for your report. Don't forget to star the repository!