QuantConnect / pythonnet

Python for .NET is a package that gives Python programmers nearly seamless integration with the .NET Common Language Runtime (CLR) and provides a powerful application scripting tool for .NET developers.
http://pythonnet.github.io
MIT License
28 stars 27 forks source link

Improve property access performance #51

Closed jmerle closed 3 years ago

jmerle commented 3 years ago

What does this implement/fix? Explain your changes.

This PR improves property access performance considerably:

This PR adds 36 tests covering Python code getting/setting C# properties/fields. The changes pass these tests and all of LEAN's regression tests.

Unrelated changes:

IndicatorRibbonBenchmark benchmarks (we currently use a Python.NET debug build in production):

LEAN debug master // Python.NET debug master:
Debug: Algorithm Id:(IndicatorRibbonBenchmark) completed in 149,20 seconds at 11k data points per second. Processing total of 1.578.463 data points.
Debug: Algorithm Id:(IndicatorRibbonBenchmark) completed in 146,29 seconds at 11k data points per second. Processing total of 1.578.463 data points.
Debug: Algorithm Id:(IndicatorRibbonBenchmark) completed in 146,48 seconds at 11k data points per second. Processing total of 1.578.463 data points.
Debug: Algorithm Id:(IndicatorRibbonBenchmark) completed in 148,64 seconds at 11k data points per second. Processing total of 1.578.463 data points.
Debug: Algorithm Id:(IndicatorRibbonBenchmark) completed in 147,71 seconds at 11k data points per second. Processing total of 1.578.463 data points.
Average time: 147,664 seconds

LEAN debug master // Python.NET debug PR:
Debug: Algorithm Id:(IndicatorRibbonBenchmark) completed in 115,87 seconds at 14k data points per second. Processing total of 1.578.463 data points.
Debug: Algorithm Id:(IndicatorRibbonBenchmark) completed in 115,61 seconds at 14k data points per second. Processing total of 1.578.463 data points.
Debug: Algorithm Id:(IndicatorRibbonBenchmark) completed in 120,06 seconds at 13k data points per second. Processing total of 1.578.463 data points.
Debug: Algorithm Id:(IndicatorRibbonBenchmark) completed in 119,92 seconds at 13k data points per second. Processing total of 1.578.463 data points.
Debug: Algorithm Id:(IndicatorRibbonBenchmark) completed in 118,67 seconds at 13k data points per second. Processing total of 1.578.463 data points.
Average time: 118,026 seconds (-20.1% against Python.NET debug master)

LEAN debug master // Python.NET release master:
Debug: Algorithm Id:(IndicatorRibbonBenchmark) completed in 89,52 seconds at 18k data points per second. Processing total of 1.578.463 data points.
Debug: Algorithm Id:(IndicatorRibbonBenchmark) completed in 88,79 seconds at 18k data points per second. Processing total of 1.578.463 data points.
Debug: Algorithm Id:(IndicatorRibbonBenchmark) completed in 88,47 seconds at 18k data points per second. Processing total of 1.578.463 data points.
Average time: 88,93 seconds (-39.8% against Python.NET debug master)

LEAN debug master // Python.NET release PR:
Debug: Algorithm Id:(IndicatorRibbonBenchmark) completed in 65,42 seconds at 24k data points per second. Processing total of 1.578.463 data points.
Debug: Algorithm Id:(IndicatorRibbonBenchmark) completed in 66,76 seconds at 24k data points per second. Processing total of 1.578.463 data points.
Debug: Algorithm Id:(IndicatorRibbonBenchmark) completed in 66,83 seconds at 24k data points per second. Processing total of 1.578.463 data points.
Average time: 66,34 seconds (-55.1% against Python.NET debug master, -25.4% against Python.NET release master)

Does this close any currently open issues?

Closes #50.

Any other comments?

No.

Checklist

Check all those that are applicable and complete.

jmerle commented 3 years ago

Summary of the latest changes:

New benchmarks:

Debug: Algorithm Id:(IndicatorRibbonBenchmark) completed in 62,48 seconds at 25k data points per second. Processing total of 1.578.463 data points.
Debug: Algorithm Id:(IndicatorRibbonBenchmark) completed in 62,13 seconds at 25k data points per second. Processing total of 1.578.463 data points.
Debug: Algorithm Id:(IndicatorRibbonBenchmark) completed in 63,80 seconds at 25k data points per second. Processing total of 1.578.463 data points.
Average time: 62,80 seconds (-57.8% against Python.NET debug master, -29.4% against Python.NET release master)