search

fluentpython / example-code-2e

Example code for Fluent Python, 2nd edition (O'Reilly 2022)
https://amzn.to/3J48u2J
MIT License
3.17k stars 902 forks source link

concurrency examples #43

Open amirreza8002 opened 5 months ago

amirreza8002 commented 5 months ago

hi. so i know that you probably know this but if there will be a 3rd edition, exapmles 19-13 and 19-12 (and other examples in chapter 19) need a rework. e.g the mentioned examples actually fail to prove the point of multiprocessing being good for cpu bound work since the prime number example is done in less than a second the perf_countr() actually shows 0.00 and 19-13 actually takes longer due to multiprocessing overhead

below is the result of running them on python 3.12.2 on a not very new xeon cpu with perf_counter()

python sequential.py 

Checking 20 numbers sequentially:
               2  P  0.000001s
 142702110479723  P  0.000002s
 299593572317531  P  0.000001s
3333333333333301  P  0.000001s
3333333333333333  P  0.000001s
3333335652092209  P  0.000001s
4444444444444423  P  0.000001s
4444444444444444     0.000001s
4444444488888889  P  0.000001s
5555553133149889  P  0.000001s
5555555555555503  P  0.000001s
5555555555555555  P  0.000001s
6666666666666666     0.000001s
6666666666666719  P  0.000001s
6666667141414921  P  0.000001s
7777777536340681  P  0.000001s
7777777777777753  P  0.000001s
7777777777777777  P  0.000001s
9999999999999917  P  0.000001s
9999999999999999  P  0.000001s
Total time: 0.00s
python procs.py 

Cheking 20 numbers with 16 processes:
               2  P  0.000035s
 142702110479723  P  0.000003s
 299593572317531  P  0.000002s
3333333333333301  P  0.000001s
3333333333333333  P  0.000001s
3333335652092209  P  0.000001s
4444444444444423  P  0.000001s
4444444444444444     0.000001s
4444444488888889  P  0.000001s
5555555555555503  P  0.000001s
5555555555555555  P  0.000001s
6666666666666666     0.000001s
6666666666666719  P  0.000001s
6666667141414921  P  0.000001s
7777777536340681  P  0.000001s
7777777777777753  P  0.000001s
7777777777777777  P  0.000001s
9999999999999917  P  0.000001s
9999999999999999  P  0.000001s
5555553133149889  P  0.000033s
20 checks in 0.02s

and this is the result using perf_counter_ns()

python sequential.py 

Checking 20 numbers sequentially:
               2  P 487.000000ns
 142702110479723  P 1047.000000ns
 299593572317531  P 500.000000ns
3333333333333301  P 320.000000ns
3333333333333333  P 295.000000ns
3333335652092209  P 279.000000ns
4444444444444423  P 214.000000ns
4444444444444444    296.000000ns
4444444488888889  P 215.000000ns
5555553133149889  P 200.000000ns
5555555555555503  P 222.000000ns
5555555555555555  P 247.000000ns
6666666666666666    290.000000ns
6666666666666719  P 196.000000ns
6666667141414921  P 227.000000ns
7777777536340681  P 202.000000ns
7777777777777753  P 202.000000ns
7777777777777777  P 202.000000ns
9999999999999917  P 202.000000ns
9999999999999999  P 202.000000ns
Total time: 118353.00ns
python procs.py 

Cheking 20 numbers with 16 processes:
               2  P 13804.000000ns
 142702110479723  P 2860.000000ns
 299593572317531  P 1762.000000ns
3333333333333301  P 1324.000000ns
3333333333333333  P 1270.000000ns
3333335652092209  P 1073.000000ns
4444444444444423  P 794.000000ns
4444444444444444    1200.000000ns
4444444488888889  P 810.000000ns
5555553133149889  P 890.000000ns
5555555555555503  P 753.000000ns
5555555555555555  P 995.000000ns
6666666666666666    925.000000ns
6666666666666719  P 813.000000ns
6666667141414921  P 678.000000ns
7777777536340681  P 763.000000ns
7777777777777753  P 703.000000ns
9999999999999917  P 941.000000ns
9999999999999999  P 740.000000ns
7777777777777777  P 26330.000000ns
20 checks in 18213477.00ns
ramalho commented 5 months ago

Thank you very much for this issue report, @amirreza8002 .

I am aware of this issue. After the Second Edition came out I developed a small library that provides samples of large integers of different magnitudes so that you can pick a magnitude that makes the example interesting each machine. Integers of 2**55 magnitude take too long on a Raspberry Pi, but take about a second on a machine with coreI9.

Please see the example in https://github.com/ramalho/python-eng/blob/main/concorrencia/primes/n_primes_proc.ipynb

I will leave this issue open until I get to this chapter while working on the Third Edition.

Thanks!