flow-php / flow

Flow PHP - data processing framework
https://flow-php.com
MIT License
404 stars 23 forks source link

Replaced reflection with bind closure approach to set dataframe pipeline #1058

Closed norberttech closed 2 months ago

norberttech commented 2 months ago

Change Log

Added

Fixed

Changed

  • Replaced reflection with bind closure approach to set dataframe pipeline

Removed

Deprecated

Security


Description

github-actions[bot] commented 2 months ago

Flow PHP - Benchmarks

Results of the benchmarks from this PR are compared with the results from 1.x branch.

Extractors ```shell +-----------------------+-------------------+------+-----+------------------+------------------+-----------------+ | benchmark | subject | revs | its | mem_peak | mode | rstdev | +-----------------------+-------------------+------+-----+------------------+------------------+-----------------+ | AvroExtractorBench | bench_extract_10k | 1 | 3 | 35.290mb +0.00% | 853.353ms +0.43% | ±0.92% +202.19% | | CSVExtractorBench | bench_extract_10k | 1 | 3 | 5.010mb +0.00% | 345.314ms +0.18% | ±0.16% -55.61% | | JsonExtractorBench | bench_extract_10k | 1 | 3 | 5.165mb +0.00% | 1.094s +3.04% | ±0.32% -69.22% | | ParquetExtractorBench | bench_extract_10k | 1 | 3 | 135.837mb +0.00% | 918.820ms +2.29% | ±0.44% -85.64% | | TextExtractorBench | bench_extract_10k | 1 | 3 | 4.925mb +0.00% | 35.803ms +5.51% | ±0.66% -46.70% | | XmlExtractorBench | bench_extract_10k | 1 | 3 | 4.931mb +0.00% | 429.425ms +0.22% | ±0.48% +87.33% | +-----------------------+-------------------+------+-----+------------------+------------------+-----------------+ ```
Transformers ```shell +-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+ | benchmark | subject | revs | its | mem_peak | mode | rstdev | +-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+ | RenameEntryTransformerBench | bench_transform_10k_rows | 1 | 3 | 116.231mb +0.00% | 59.449ms +0.93% | ±0.67% -38.45% | +-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+ ```
Loaders ```shell +--------------------+----------------+------+-----+------------------+------------------+----------------+ | benchmark | subject | revs | its | mem_peak | mode | rstdev | +--------------------+----------------+------+-----+------------------+------------------+----------------+ | AvroLoaderBench | bench_load_10k | 1 | 3 | 96.676mb +0.00% | 455.277ms -0.15% | ±0.01% -99.15% | | CSVLoaderBench | bench_load_10k | 1 | 3 | 55.151mb +0.00% | 69.202ms +0.93% | ±0.58% -54.68% | | JsonLoaderBench | bench_load_10k | 1 | 3 | 107.585mb +0.00% | 50.060ms +0.44% | ±0.91% +65.12% | | ParquetLoaderBench | bench_load_10k | 1 | 3 | 227.005mb +0.00% | 1.406s +0.84% | ±0.53% -41.57% | | TextLoaderBench | bench_load_10k | 1 | 3 | 17.967mb +0.00% | 38.149ms +1.58% | ±0.65% +61.72% | +--------------------+----------------+------+-----+------------------+------------------+----------------+ ```
Building Blocks ```shell +-------------------------+----------------------------+------+-----+------------------+------------------+-----------------+ | benchmark | subject | revs | its | mem_peak | mode | rstdev | +-------------------------+----------------------------+------+-----+------------------+------------------+-----------------+ | RowsBench | bench_chunk_10_on_10k | 2 | 3 | 87.051mb +0.00% | 3.430ms +2.60% | ±2.52% +243.49% | | RowsBench | bench_diff_left_1k_on_10k | 2 | 3 | 102.649mb +0.00% | 190.724ms +3.72% | ±0.99% -24.79% | | RowsBench | bench_diff_right_1k_on_10k | 2 | 3 | 85.369mb +0.00% | 18.911ms -0.04% | ±0.88% +27.96% | | RowsBench | bench_drop_1k_on_10k | 2 | 3 | 88.291mb +0.00% | 1.724ms +6.03% | ±1.39% -9.37% | | RowsBench | bench_drop_right_1k_on_10k | 2 | 3 | 88.291mb +0.00% | 1.684ms +4.42% | ±2.38% +9.63% | | RowsBench | bench_entries_on_10k | 2 | 3 | 85.403mb +0.00% | 2.593ms +2.96% | ±1.48% -20.30% | | RowsBench | bench_filter_on_10k | 2 | 3 | 85.932mb +0.00% | 16.179ms -11.57% | ±0.73% -30.16% | | RowsBench | bench_find_on_10k | 2 | 3 | 85.932mb +0.00% | 16.186ms -6.15% | ±0.50% -79.76% | | RowsBench | bench_find_one_on_10k | 10 | 3 | 83.836mb +0.00% | 1.706μs 0.00% | ±2.72% 0.00% | | RowsBench | bench_first_on_10k | 10 | 3 | 83.836mb +0.00% | 0.400μs +33.33% | ±0.00% +0.00% | | RowsBench | bench_flat_map_on_1k | 2 | 3 | 93.186mb +0.00% | 12.577ms +3.97% | ±1.12% -44.18% | | RowsBench | bench_map_on_10k | 2 | 3 | 122.557mb +0.00% | 61.124ms +4.89% | ±0.68% -34.73% | | RowsBench | bench_merge_1k_on_10k | 2 | 3 | 86.452mb +0.00% | 1.241ms +0.86% | ±0.93% -73.67% | | RowsBench | bench_partition_by_on_10k | 2 | 3 | 89.799mb +0.00% | 63.938ms +3.40% | ±1.42% +38.87% | | RowsBench | bench_remove_on_10k | 2 | 3 | 88.553mb +0.00% | 3.850ms +0.75% | ±0.07% -95.86% | | RowsBench | bench_sort_asc_on_1k | 2 | 3 | 83.914mb +0.00% | 40.847ms +1.26% | ±1.18% -24.28% | | RowsBench | bench_sort_by_on_1k | 2 | 3 | 83.915mb +0.00% | 40.852ms +2.00% | ±1.16% -26.45% | | RowsBench | bench_sort_desc_on_1k | 2 | 3 | 83.914mb +0.00% | 40.838ms +0.56% | ±1.11% +393.94% | | RowsBench | bench_sort_entries_on_1k | 2 | 3 | 86.278mb +0.00% | 7.320ms +2.36% | ±0.87% -45.77% | | RowsBench | bench_sort_on_1k | 2 | 3 | 83.836mb +0.00% | 28.964ms +2.48% | ±1.58% -14.72% | | RowsBench | bench_take_1k_on_10k | 10 | 3 | 83.836mb +0.00% | 13.394μs +1.03% | ±0.35% -83.95% | | RowsBench | bench_take_right_1k_on_10k | 10 | 3 | 83.836mb +0.00% | 15.700μs +0.78% | ±0.52% -34.81% | | RowsBench | bench_unique_on_1k | 2 | 3 | 102.651mb +0.00% | 194.933ms +1.66% | ±0.51% +61.54% | | NativeEntryFactoryBench | bench_entry_factory | 1 | 3 | 116.729mb +0.00% | 520.470ms +1.39% | ±0.22% +189.32% | | NativeEntryFactoryBench | bench_entry_factory | 1 | 3 | 60.207mb +0.00% | 262.192ms +3.13% | ±0.67% -60.37% | | NativeEntryFactoryBench | bench_entry_factory | 1 | 3 | 15.140mb +0.00% | 55.389ms +0.40% | ±0.98% +116.84% | | TypeDetectorBench | bench_type_detector | 1 | 3 | 59.966mb +0.00% | 436.832ms +0.40% | ±0.53% -56.23% | | TypeDetectorBench | bench_type_detector | 1 | 3 | 14.505mb +0.00% | 86.051ms +0.43% | ±0.45% +489.21% | +-------------------------+----------------------------+------+-----+------------------+------------------+-----------------+ ```