flow-php / flow

Flow PHP - data processing framework
https://flow-php.com
MIT License
404 stars 23 forks source link

Improve performance of sorting operation by reducing number of writes to cache #1036

Closed norberttech closed 3 months ago

norberttech commented 3 months ago

Change Log

Added

Fixed

  • Improve performance of sorting operation by reducing number of writes to cache

Changed

Removed

Deprecated

Security


Description

github-actions[bot] commented 3 months ago

Flow PHP - Benchmarks

Results of the benchmarks from this PR are compared with the results from 1.x branch.

Extractors ```shell +-----------------------+-------------------+------+-----+------------------+------------------+-----------------+ | benchmark | subject | revs | its | mem_peak | mode | rstdev | +-----------------------+-------------------+------+-----+------------------+------------------+-----------------+ | AvroExtractorBench | bench_extract_10k | 1 | 3 | 35.284mb +0.00% | 845.630ms -0.43% | ±0.29% +2.91% | | CSVExtractorBench | bench_extract_10k | 1 | 3 | 5.005mb +0.00% | 347.662ms +1.07% | ±1.60% +166.98% | | JsonExtractorBench | bench_extract_10k | 1 | 3 | 5.159mb +0.00% | 1.077s +1.39% | ±1.32% +190.72% | | ParquetExtractorBench | bench_extract_10k | 1 | 3 | 135.826mb +0.00% | 906.518ms -0.57% | ±0.63% +225.43% | | TextExtractorBench | bench_extract_10k | 1 | 3 | 4.915mb +0.00% | 35.459ms -0.57% | ±0.35% -68.85% | | XmlExtractorBench | bench_extract_10k | 1 | 3 | 4.920mb +0.00% | 432.010ms -0.35% | ±0.29% -80.97% | +-----------------------+-------------------+------+-----+------------------+------------------+-----------------+ ```
Transformers ```shell +-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+ | benchmark | subject | revs | its | mem_peak | mode | rstdev | +-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+ | RenameEntryTransformerBench | bench_transform_10k_rows | 1 | 3 | 116.226mb +0.00% | 60.173ms -1.10% | ±0.47% -66.56% | +-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+ ```
Loaders ```shell +--------------------+----------------+------+-----+------------------+------------------+-----------------+ | benchmark | subject | revs | its | mem_peak | mode | rstdev | +--------------------+----------------+------+-----+------------------+------------------+-----------------+ | AvroLoaderBench | bench_load_10k | 1 | 3 | 96.670mb +0.00% | 469.813ms -0.19% | ±0.71% +120.17% | | CSVLoaderBench | bench_load_10k | 1 | 3 | 55.146mb +0.00% | 70.202ms -0.99% | ±0.35% -74.65% | | JsonLoaderBench | bench_load_10k | 1 | 3 | 107.579mb +0.00% | 51.302ms -2.91% | ±0.22% -58.83% | | ParquetLoaderBench | bench_load_10k | 1 | 3 | 226.994mb +0.00% | 1.417s -0.69% | ±0.12% -27.37% | | TextLoaderBench | bench_load_10k | 1 | 3 | 17.962mb +0.00% | 40.154ms -1.99% | ±0.51% -24.39% | +--------------------+----------------+------+-----+------------------+------------------+-----------------+ ```
Building Blocks ```shell +-------------------------+----------------------------+------+-----+------------------+------------------+-----------------+ | benchmark | subject | revs | its | mem_peak | mode | rstdev | +-------------------------+----------------------------+------+-----+------------------+------------------+-----------------+ | RowsBench | bench_chunk_10_on_10k | 2 | 3 | 87.050mb +0.00% | 3.264ms -15.86% | ±1.67% -39.13% | | RowsBench | bench_diff_left_1k_on_10k | 2 | 3 | 102.648mb +0.00% | 187.684ms +0.27% | ±0.73% +99.60% | | RowsBench | bench_diff_right_1k_on_10k | 2 | 3 | 85.368mb +0.00% | 18.790ms -2.15% | ±1.61% -28.42% | | RowsBench | bench_drop_1k_on_10k | 2 | 3 | 88.290mb +0.00% | 1.679ms -22.08% | ±0.88% -58.60% | | RowsBench | bench_drop_right_1k_on_10k | 2 | 3 | 88.290mb +0.00% | 1.711ms -20.04% | ±1.05% -63.91% | | RowsBench | bench_entries_on_10k | 2 | 3 | 85.402mb +0.00% | 2.513ms -9.28% | ±1.72% -13.27% | | RowsBench | bench_filter_on_10k | 2 | 3 | 85.931mb +0.00% | 16.569ms -1.59% | ±0.27% -44.14% | | RowsBench | bench_find_on_10k | 2 | 3 | 85.931mb +0.00% | 16.214ms -7.76% | ±2.38% +14.59% | | RowsBench | bench_find_one_on_10k | 10 | 3 | 83.835mb +0.00% | 1.794μs -10.03% | ±2.67% +11.32% | | RowsBench | bench_first_on_10k | 10 | 3 | 83.835mb +0.00% | 0.400μs 0.00% | ±0.00% 0.00% | | RowsBench | bench_flat_map_on_1k | 2 | 3 | 93.185mb +0.00% | 12.584ms -4.76% | ±0.42% -70.97% | | RowsBench | bench_map_on_10k | 2 | 3 | 122.556mb +0.00% | 62.260ms -1.73% | ±1.72% +178.23% | | RowsBench | bench_merge_1k_on_10k | 2 | 3 | 86.451mb +0.00% | 1.295ms -24.08% | ±2.86% +56.15% | | RowsBench | bench_partition_by_on_10k | 2 | 3 | 89.798mb +0.00% | 63.384ms -4.25% | ±1.62% +139.23% | | RowsBench | bench_remove_on_10k | 2 | 3 | 88.552mb +0.00% | 3.871ms -11.54% | ±0.96% +261.32% | | RowsBench | bench_sort_asc_on_1k | 2 | 3 | 83.913mb +0.00% | 40.255ms -1.50% | ±2.87% +228.30% | | RowsBench | bench_sort_by_on_1k | 2 | 3 | 83.914mb +0.00% | 40.534ms -3.55% | ±0.96% +17.79% | | RowsBench | bench_sort_desc_on_1k | 2 | 3 | 83.913mb +0.00% | 39.631ms -3.76% | ±1.61% +579.53% | | RowsBench | bench_sort_entries_on_1k | 2 | 3 | 86.276mb +0.00% | 7.454ms +1.43% | ±1.87% +161.73% | | RowsBench | bench_sort_on_1k | 2 | 3 | 83.835mb +0.00% | 28.956ms -1.46% | ±3.33% +243.44% | | RowsBench | bench_take_1k_on_10k | 10 | 3 | 83.835mb +0.00% | 14.399μs +5.50% | ±1.70% -37.45% | | RowsBench | bench_take_right_1k_on_10k | 10 | 3 | 83.835mb +0.00% | 17.521μs +1.47% | ±0.71% -70.23% | | RowsBench | bench_unique_on_1k | 2 | 3 | 102.649mb +0.00% | 192.623ms +1.21% | ±0.42% +216.52% | | NativeEntryFactoryBench | bench_entry_factory | 1 | 3 | 116.727mb +0.00% | 505.937ms -2.07% | ±1.53% +74.74% | | NativeEntryFactoryBench | bench_entry_factory | 1 | 3 | 60.205mb +0.00% | 255.585ms -1.24% | ±1.94% +49.24% | | NativeEntryFactoryBench | bench_entry_factory | 1 | 3 | 15.140mb +0.00% | 55.734ms -1.15% | ±1.37% +51.70% | | TypeDetectorBench | bench_type_detector | 1 | 3 | 59.960mb +0.00% | 439.344ms +0.74% | ±0.93% +370.00% | | TypeDetectorBench | bench_type_detector | 1 | 3 | 14.499mb +0.00% | 86.157ms +0.80% | ±0.53% -46.03% | +-------------------------+----------------------------+------+-----+------------------+------------------+-----------------+ ```