Closed MartyLake closed 4 months ago
Hi,
Thanks for your report. The documentation states:
Moulti's performance has room for improvement.
... so I expected a report with such a title... but not with that exact content.
The order of magnitude between the two scenarios is not exactly surprising considering the way moulti_exec
works (multiple fork+exec of a Python program) and I get similar figures on my (Linux) machine: 40ms vs 400ms.
However, the absolute numbers you report are indeed suspicious. It feels like something is going wrong in terms of sleep / network timeout. I will investigate that.
I forgot to ask: what version of Moulti are you using? Did you get troubles with moulti_duration
and its use of date
?
Yep, I had to use gdate
instead date
of macOS for the script to work.
Let's focus on moulti wait -m 1
(without any Moulti instance running) as this command best reflects Moulti's minimal startup time.
The idea here is to use hyperfine
(available through brew
) to get more specific figures.
Could you please run the commands below and report their output here?
hyperfine --warmup=2 --runs=400 --ignore-failure --shell=default 'python3 -c "import sys; print(sys.argv)" a b c'
hyperfine --warmup=2 --runs=400 --ignore-failure --shell=none 'moulti wait -m 1'
There you go:
/tmp$ hyperfine --warmup=2 --runs=400 --ignore-failure --shell=default 'python3 -c "import sys; print(sys.argv)" a b c'
Benchmark 1: python3 -c "import sys; print(sys.argv)" a b c
Time (mean ± σ): 96.8 ms ± 1.6 ms [User: 12.8 ms, System: 10.2 ms]
Range (min … max): 91.1 ms … 105.9 ms 400 runs
/tmp$ hyperfine --warmup=2 --runs=400 --ignore-failure --shell=none 'moulti wait -m 1'
Benchmark 1: moulti wait -m 1
Time (mean ± σ): 129.3 ms ± 3.9 ms [User: 38.3 ms, System: 15.1 ms]
Range (min … max): 121.6 ms … 173.4 ms 400 runs
Warning: Ignoring non-zero exit code.
Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.
Thank you!
Although there still is room for performance improvement in Moulti[1], this issue is now "Python is slow".
Unless your machine is very old (is it?), the most likely explanation is that you installed a non-optimized version of Python through brew
-- source: https://www.reddit.com/r/Python/comments/oouh9a/why_is_python_so_much_slower_on_macos/
A possible workaround is to:
pipx install --python=/path/to/the/new/python/interpreter moulti
should do[1] I think I can shave a few milliseconds by not importing unidiff. And I should have a look at typing-related imports.
Well, I did not know that Python could be that fast. After removing asdf and installing the Python Release from your link, here are the results:
/tmp$ hyperfine --warmup=2 --runs=400 --ignore-failure --shell=default 'python3 -c "import sys; print(sys.argv)" a b c'
Benchmark 1: python3 -c "import sys; print(sys.argv)" a b c
Time (mean ± σ): 12.6 ms ± 0.5 ms [User: 8.7 ms, System: 3.0 ms]
Range (min … max): 11.5 ms … 16.1 ms 400 runs
/tmp$ hyperfine --warmup=2 --runs=400 --ignore-failure --shell=none 'moulti wait -m 1'
Benchmark 1: moulti wait -m 1
Time (mean ± σ): 43.3 ms ± 7.7 ms [User: 31.9 ms, System: 7.4 ms]
Range (min … max): 39.5 ms … 169.7 ms 400 runs
Warning: Ignoring non-zero exit code.
Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.
Great!
Your previous Python install was apparently severely unoptimized: my macOS VM yields 22.4 < 25.3±1.6 < 32.6 for Python and 59.7 < 68.0±5.0 < 92.2 for moulti wait
using Python 3.12.3 obtained through brew
.
These new figures of yours roughly match those on my Linux machine (6.5 < 8.1±1.0 < 12 for Python and 34.1 < 36.9±1.5 < 43.4 for moulti wait
). If we focus on the mean time to run moulti wait
, you went from 129.3 to 43.3, i.e. -66.5%.
Is the resulting performance acceptable or should we further investigate?
Yep, I had to use
gdate
insteaddate
of macOS for the script to work.
FYI, the devel
branch no longer depends on GNU date.
I think I can shave a few milliseconds by not importing unidiff.
Done. Performance-wise, the improvement is meagre (-1.2ms; likely more on non-optimized Python interpreters). But I kept this change since it makes it possible to use most of Moulti even if the unidiff library is not installed.
Is the resulting performance acceptable or should we further investigate?
Completely acceptable for me, you can close the issue thanks.
FYI, the
devel
branch no longer depends on GNU date. Done. Performance-wise, the improvement is meagre (-1.2ms; likely more on non-optimized Python interpreters). But I kept this change since it makes it possible to use most of Moulti even if the unidiff library is not installed.
Sorry but I don’t understand much about python distribution. I installed with pip3 install moulti
. How can I switch to the devel
branch ?
I installed with
pip3 install moulti
. How can I switch to thedevel
branch ?
If you are not in a hurry, I advise to simply wait until Monday: the contents of the devel
branch should land in the master
branch and in a 1.8.1
release on Sunday.
Additionally, examples/moulti-functions.bash
is not shipped through pip/pipx. You likely obtained it through git clone
or by browsing GitHub.
If you are in a hurry, this should do:
git clone https://github.com/xavierog/moulti.git -b devel
pip install "$(pwd)/moulti"
Hello, I really like the principle of that script! However, it is very slow on macOS. not sure why.
script used to time
Here is the one-liner I used:
time (ops=("one" "two"); source ./moulti-functions.bash; if moulti wait -d 1 -m 1; then for needle in "${ops[@]}"; do moulti_exec echo $needle; done; else for needle in "${ops[@]}"; do echo $needle; done; fi)
time with
moulti init
running in other termtime without moulti
Best,