flow-php / flow

Flow PHP - data processing framework
https://flow-php.com
MIT License
404 stars 23 forks source link

reorder entries #1059

Closed norberttech closed 2 months ago

norberttech commented 2 months ago

Change Log

Added

  • allow to reorder entries through DataFrame::select
  • DataFrame::reorderEntries() with built-in custom ordering strategies

Fixed

Changed

Removed

Deprecated

Security


Description

github-actions[bot] commented 2 months ago

Flow PHP - Benchmarks

Results of the benchmarks from this PR are compared with the results from 1.x branch.

Extractors ```shell +-----------------------+-------------------+------+-----+------------------+------------------+-----------------+ | benchmark | subject | revs | its | mem_peak | mode | rstdev | +-----------------------+-------------------+------+-----+------------------+------------------+-----------------+ | AvroExtractorBench | bench_extract_10k | 1 | 3 | 35.290mb +0.03% | 855.173ms +2.91% | ±0.71% -35.25% | | CSVExtractorBench | bench_extract_10k | 1 | 3 | 5.010mb +0.18% | 349.809ms +2.10% | ±0.83% +29.66% | | JsonExtractorBench | bench_extract_10k | 1 | 3 | 5.165mb +0.17% | 1.090s -0.73% | ±1.11% -49.44% | | ParquetExtractorBench | bench_extract_10k | 1 | 3 | 135.837mb +0.01% | 965.659ms +9.52% | ±2.46% +443.24% | | TextExtractorBench | bench_extract_10k | 1 | 3 | 4.924mb +0.18% | 35.973ms +1.09% | ±0.72% +11.17% | | XmlExtractorBench | bench_extract_10k | 1 | 3 | 4.930mb +0.18% | 439.120ms +0.61% | ±0.74% +9.48% | +-----------------------+-------------------+------+-----+------------------+------------------+-----------------+ ```
Transformers ```shell +-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+ | benchmark | subject | revs | its | mem_peak | mode | rstdev | +-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+ | RenameEntryTransformerBench | bench_transform_10k_rows | 1 | 3 | 116.230mb +0.01% | 62.444ms +3.70% | ±0.90% -18.29% | +-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+ ```
Loaders ```shell +--------------------+----------------+------+-----+------------------+------------------+----------------+ | benchmark | subject | revs | its | mem_peak | mode | rstdev | +--------------------+----------------+------+-----+------------------+------------------+----------------+ | AvroLoaderBench | bench_load_10k | 1 | 3 | 96.676mb +0.01% | 461.006ms +1.04% | ±0.70% +37.16% | | CSVLoaderBench | bench_load_10k | 1 | 3 | 55.151mb +0.02% | 71.297ms +3.69% | ±0.91% -32.80% | | JsonLoaderBench | bench_load_10k | 1 | 3 | 107.584mb +0.01% | 53.848ms +5.89% | ±0.29% -78.44% | | ParquetLoaderBench | bench_load_10k | 1 | 3 | 227.004mb +0.00% | 1.448s +2.51% | ±0.59% +9.33% | | TextLoaderBench | bench_load_10k | 1 | 3 | 17.967mb +0.05% | 39.319ms +0.71% | ±0.22% -11.57% | +--------------------+----------------+------+-----+------------------+------------------+----------------+ ```
Building Blocks ```shell +-------------------------+----------------------------+------+-----+------------------+------------------+-----------------+ | benchmark | subject | revs | its | mem_peak | mode | rstdev | +-------------------------+----------------------------+------+-----+------------------+------------------+-----------------+ | RowsBench | bench_chunk_10_on_10k | 2 | 3 | 87.051mb +0.01% | 3.917ms +16.30% | ±0.77% -31.46% | | RowsBench | bench_diff_left_1k_on_10k | 2 | 3 | 102.649mb +0.01% | 188.161ms +0.90% | ±0.88% +4.37% | | RowsBench | bench_diff_right_1k_on_10k | 2 | 3 | 85.369mb +0.01% | 19.186ms +0.72% | ±0.62% -22.37% | | RowsBench | bench_drop_1k_on_10k | 2 | 3 | 88.291mb +0.01% | 1.912ms +10.02% | ±2.49% -1.13% | | RowsBench | bench_drop_right_1k_on_10k | 2 | 3 | 88.291mb +0.01% | 1.902ms +9.28% | ±1.68% +20.09% | | RowsBench | bench_entries_on_10k | 2 | 3 | 85.403mb +0.01% | 3.017ms +8.01% | ±1.09% +25.21% | | RowsBench | bench_filter_on_10k | 2 | 3 | 85.932mb +0.01% | 17.218ms +4.40% | ±1.12% -51.48% | | RowsBench | bench_find_on_10k | 2 | 3 | 85.932mb +0.01% | 17.406ms +6.42% | ±0.51% +0.97% | | RowsBench | bench_find_one_on_10k | 10 | 3 | 83.836mb +0.01% | 1.994μs +11.15% | ±2.40% -10.17% | | RowsBench | bench_first_on_10k | 10 | 3 | 83.836mb +0.01% | 0.400μs +33.33% | ±0.00% +0.00% | | RowsBench | bench_flat_map_on_1k | 2 | 3 | 93.186mb +0.01% | 14.601ms +19.67% | ±1.54% -17.73% | | RowsBench | bench_map_on_10k | 2 | 3 | 122.557mb +0.01% | 64.016ms +5.66% | ±0.92% -0.85% | | RowsBench | bench_merge_1k_on_10k | 2 | 3 | 86.452mb +0.01% | 1.621ms +10.72% | ±2.00% -42.60% | | RowsBench | bench_partition_by_on_10k | 2 | 3 | 89.799mb +0.01% | 67.715ms +2.57% | ±0.94% -51.88% | | RowsBench | bench_remove_on_10k | 2 | 3 | 88.553mb +0.01% | 4.204ms +7.00% | ±0.34% -64.59% | | RowsBench | bench_sort_asc_on_1k | 2 | 3 | 83.914mb +0.09% | 40.244ms -2.26% | ±0.80% -52.45% | | RowsBench | bench_sort_by_on_1k | 2 | 3 | 83.915mb +0.09% | 41.709ms +1.31% | ±0.92% -27.45% | | RowsBench | bench_sort_desc_on_1k | 2 | 3 | 83.914mb +0.09% | 41.737ms +3.94% | ±1.99% +176.38% | | RowsBench | bench_sort_entries_on_1k | 2 | 3 | 86.277mb +0.01% | 7.666ms +4.46% | ±1.23% +65.72% | | RowsBench | bench_sort_on_1k | 2 | 3 | 83.836mb +0.01% | 29.855ms +4.17% | ±0.76% -56.55% | | RowsBench | bench_take_1k_on_10k | 10 | 3 | 83.836mb +0.01% | 15.317μs +12.22% | ±1.42% -47.74% | | RowsBench | bench_take_right_1k_on_10k | 10 | 3 | 83.836mb +0.01% | 18.041μs +13.55% | ±1.39% -15.03% | | RowsBench | bench_unique_on_1k | 2 | 3 | 102.650mb +0.01% | 191.133ms +0.12% | ±0.57% -32.69% | | NativeEntryFactoryBench | bench_entry_factory | 1 | 3 | 116.728mb +0.01% | 523.186ms +3.15% | ±0.40% -67.93% | | NativeEntryFactoryBench | bench_entry_factory | 1 | 3 | 60.207mb +0.01% | 265.576ms +1.12% | ±1.11% -0.20% | | NativeEntryFactoryBench | bench_entry_factory | 1 | 3 | 15.140mb +0.06% | 57.508ms +1.09% | ±0.83% -65.69% | | TypeDetectorBench | bench_type_detector | 1 | 3 | 59.966mb +0.01% | 441.674ms +1.44% | ±0.46% +2.21% | | TypeDetectorBench | bench_type_detector | 1 | 3 | 14.505mb +0.06% | 88.682ms +2.09% | ±0.34% -39.99% | +-------------------------+----------------------------+------+-----+------------------+------------------+-----------------+ ```