mishaturnbull
commented
4 weeks ago Performance results captured on the "before" code (non-iterator):
misha@burnt-sand ~/shared/Projects/edgegraph (feature/iterator-traversals *)
>>> scripts/profile.sh
Working directory:
========================================== test session starts ==========================================
platform linux -- Python 3.9.2, pytest-7.4.3, pluggy-1.0.0 -- /usr/bin/python
cachedir: .pytest_cache
rootdir: /mnt/shared/Projects/edgegraph
configfile: pyproject.toml
plugins: profiling-1.7.0, cov-4.1.0
collected 285 items / 282 deselected / 3 selected
tests/integration/test_performance.py::test_trav_versus[dft_iterative]
--------------------------------------------- live log call ---------------------------------------------
2024-10-07 22:39:46 [ INFO] Begin trav-versus routine for dft_iterative (test_performance.py:100)
2024-10-07 22:39:58 [ INFO] dft_iterative performance: total 12.407504009 s, avg 0.00012365613673 s, 4
1890336 ns miss (test_performance.py:120)
PASSED [ 33%]
tests/integration/test_performance.py::test_trav_versus[dft_recursive]
--------------------------------------------- live log call ---------------------------------------------
2024-10-07 22:39:58 [ INFO] Begin trav-versus routine for dft_recursive (test_performance.py:100)
2024-10-07 22:40:10 [ INFO] dft_recursive performance: total 12.192138629 s, avg 0.00012148560659 s, 4
3577970 ns miss (test_performance.py:120)
PASSED [ 66%]
tests/integration/test_performance.py::test_trav_versus[bft]
--------------------------------------------- live log call ---------------------------------------------
2024-10-07 22:40:10 [ INFO] Begin trav-versus routine for bft (test_performance.py:100)
2024-10-07 22:40:23 [ INFO] bft performance: total 12.278670761 s, avg 0.00012232253706 s, 46417055 ns
miss (test_performance.py:120)
PASSED [100%]
Profiling (from /mnt/shared/Projects/edgegraph/prof/combined.prof):
Mon Oct 7 22:40:23 2024 /mnt/shared/Projects/edgegraph/prof/combined.prof
131109306 function calls (130409119 primitive calls) in 36.894 seconds
Ordered by: cumulative time
List reduced from 483 to 20 due to restriction <20>
ncalls tottime percall cumtime percall filename:lineno(function)
3 0.000 0.000 36.894 12.298 runner.py:111(pytest_runtest_protocol)
3 0.000 0.000 36.893 12.298 runner.py:119(runtestprotocol)
9 0.000 0.000 36.893 4.099 runner.py:219(call_and_report)
57/33 0.000 0.000 36.893 1.118 _hooks.py:244(__call__)
57/33 0.000 0.000 36.893 1.118 _manager.py:77(_hookexec)
57/33 0.001 0.000 36.893 1.118 _callers.py:9(_multicall)
9 0.000 0.000 36.892 4.099 runner.py:247(call_runtest_hook)
9 0.000 0.000 36.891 4.099 runner.py:318(from_call)
9 0.000 0.000 36.891 4.099 runner.py:262(<lambda>)
3 0.000 0.000 36.886 12.295 runner.py:160(pytest_runtest_call)
3 0.000 0.000 36.886 12.295 python.py:1790(runtest)
3 0.003 0.001 36.886 12.295 python.py:187(pytest_pyfunc_call)
3 0.287 0.096 36.883 12.294 test_performance.py:88(test_trav_versus)
2400000 8.303 0.000 33.142 0.000 helpers.py:34(neighbors)
7200000 3.773 0.000 15.676 0.000 twoendedlink.py:119(other)
100000 1.019 0.000 12.291 0.000 depthfirst.py:196(dft_iterative)
100000 0.824 0.000 12.150 0.000 breadthfirst.py:110(bft)
100000 0.059 0.000 12.070 0.000 depthfirst.py:99(dft_recursive)
800000/100000 0.773 0.000 11.993 0.000 depthfirst.py:75(_dft_recur)
18300000 5.529 0.000 11.844 0.000 directededge.py:49(v1)
SVG profile in /mnt/shared/Projects/edgegraph/prof/combined.svg.
========================================== slowest 5 durations ==========================================
12.41s call tests/integration/test_performance.py::test_trav_versus[dft_iterative]
12.28s call tests/integration/test_performance.py::test_trav_versus[bft]
12.19s call tests/integration/test_performance.py::test_trav_versus[dft_recursive]
(2 durations < 0.005s hidden. Use -vv to show these durations.)
================================== 3 passed, 282 deselected in 39.15s ===================================
Done! with profiling / "versus" tests as per d3e2a0c690ade4a91e89eecb5a0a1e73714dd696 , but edgegraph/traversal/breadthfirst.py as per 870c8bb.
Would be beneficial to have iterator functions (generators) for traversals rather than "generate a massive in-memory list".
To-determine:
list(func(...))if they want a list? or keep the list flavor?