flow-php / flow

Flow PHP - data processing framework
https://flow-php.com
MIT License
404 stars 23 forks source link

Added path partitions extractor #1082

Closed norberttech closed 1 month ago

norberttech commented 1 month ago

Change Log

Added

  • path partitions extractor
  • Partitions::get(string $name) : Partition

Fixed

Changed

Removed

Deprecated

Security


Description

github-actions[bot] commented 1 month ago

Flow PHP - Benchmarks

Results of the benchmarks from this PR are compared with the results from 1.x branch.

Extractors ```shell +-----------------------+-------------------+------+-----+------------------+------------------+-----------------+ | benchmark | subject | revs | its | mem_peak | mode | rstdev | +-----------------------+-------------------+------+-----+------------------+------------------+-----------------+ | AvroExtractorBench | bench_extract_10k | 1 | 3 | 35.446mb +0.01% | 818.779ms -0.40% | ±0.88% +31.34% | | CSVExtractorBench | bench_extract_10k | 1 | 3 | 5.164mb +0.06% | 338.387ms +0.60% | ±1.40% +286.72% | | JsonExtractorBench | bench_extract_10k | 1 | 3 | 5.197mb +0.06% | 1.077s -0.19% | ±0.46% -54.34% | | ParquetExtractorBench | bench_extract_10k | 1 | 3 | 135.869mb +0.00% | 901.217ms +0.24% | ±0.21% -76.25% | | TextExtractorBench | bench_extract_10k | 1 | 3 | 4.956mb +0.04% | 35.795ms +1.04% | ±1.43% +39.55% | | XmlExtractorBench | bench_extract_10k | 1 | 3 | 4.962mb +0.04% | 433.297ms -0.69% | ±0.18% -55.76% | +-----------------------+-------------------+------+-----+------------------+------------------+-----------------+ ```
Transformers ```shell +-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+ | benchmark | subject | revs | its | mem_peak | mode | rstdev | +-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+ | RenameEntryTransformerBench | bench_transform_10k_rows | 1 | 3 | 116.262mb +0.00% | 60.769ms +0.05% | ±0.61% -61.48% | +-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+ ```
Loaders ```shell +--------------------+----------------+------+-----+------------------+------------------+----------------+ | benchmark | subject | revs | its | mem_peak | mode | rstdev | +--------------------+----------------+------+-----+------------------+------------------+----------------+ | AvroLoaderBench | bench_load_10k | 1 | 3 | 96.831mb +0.00% | 449.189ms -2.06% | ±0.40% -80.53% | | CSVLoaderBench | bench_load_10k | 1 | 3 | 55.243mb +0.01% | 69.603ms +0.03% | ±0.66% -21.17% | | JsonLoaderBench | bench_load_10k | 1 | 3 | 107.617mb +0.00% | 51.536ms +0.11% | ±1.09% +9.65% | | ParquetLoaderBench | bench_load_10k | 1 | 3 | 227.036mb +0.00% | 1.402s -0.49% | ±0.80% +82.46% | | TextLoaderBench | bench_load_10k | 1 | 3 | 17.998mb +0.02% | 38.669ms -1.53% | ±0.26% -71.12% | +--------------------+----------------+------+-----+------------------+------------------+----------------+ ```
Building Blocks ```shell +-------------------------+----------------------------+------+-----+------------------+------------------+-----------------+ | benchmark | subject | revs | its | mem_peak | mode | rstdev | +-------------------------+----------------------------+------+-----+------------------+------------------+-----------------+ | RowsBench | bench_chunk_10_on_10k | 2 | 3 | 87.080mb +0.00% | 3.669ms +1.00% | ±3.28% +24.89% | | RowsBench | bench_diff_left_1k_on_10k | 2 | 3 | 102.678mb +0.00% | 193.453ms +3.72% | ±0.43% -23.60% | | RowsBench | bench_diff_right_1k_on_10k | 2 | 3 | 85.398mb +0.00% | 19.342ms +1.96% | ±0.63% -8.95% | | RowsBench | bench_drop_1k_on_10k | 2 | 3 | 88.320mb +0.00% | 1.959ms +0.78% | ±0.98% -55.72% | | RowsBench | bench_drop_right_1k_on_10k | 2 | 3 | 88.320mb +0.00% | 1.945ms +7.08% | ±0.85% -76.03% | | RowsBench | bench_entries_on_10k | 2 | 3 | 85.432mb +0.00% | 3.095ms -0.71% | ±0.20% -86.11% | | RowsBench | bench_filter_on_10k | 2 | 3 | 85.961mb +0.00% | 16.950ms -7.21% | ±0.70% -76.68% | | RowsBench | bench_find_on_10k | 2 | 3 | 85.961mb +0.00% | 17.039ms -2.78% | ±1.02% +99.94% | | RowsBench | bench_find_one_on_10k | 10 | 3 | 83.865mb +0.00% | 1.894μs 0.00% | ±2.53% 0.00% | | RowsBench | bench_first_on_10k | 10 | 3 | 83.865mb +0.00% | 0.400μs 0.00% | ±0.00% 0.00% | | RowsBench | bench_flat_map_on_1k | 2 | 3 | 93.215mb +0.00% | 12.504ms +0.90% | ±1.86% -47.63% | | RowsBench | bench_map_on_10k | 2 | 3 | 122.586mb +0.00% | 60.887ms +0.10% | ±0.76% -10.14% | | RowsBench | bench_merge_1k_on_10k | 2 | 3 | 86.481mb +0.00% | 1.733ms +9.72% | ±3.10% +5.46% | | RowsBench | bench_partition_by_on_10k | 2 | 3 | 89.828mb +0.00% | 65.610ms -0.78% | ±0.58% +244.53% | | RowsBench | bench_remove_on_10k | 2 | 3 | 88.582mb +0.00% | 4.613ms +6.79% | ±2.48% -27.81% | | RowsBench | bench_sort_asc_on_1k | 2 | 3 | 84.009mb +0.00% | 39.339ms -2.94% | ±2.94% +249.61% | | RowsBench | bench_sort_by_on_1k | 2 | 3 | 84.009mb +0.00% | 39.731ms -1.35% | ±1.01% -44.49% | | RowsBench | bench_sort_desc_on_1k | 2 | 3 | 84.009mb +0.00% | 39.807ms -6.09% | ±1.21% -59.08% | | RowsBench | bench_sort_entries_on_1k | 2 | 3 | 86.306mb +0.00% | 7.416ms -2.94% | ±0.66% -47.20% | | RowsBench | bench_sort_on_1k | 2 | 3 | 83.865mb +0.00% | 28.879ms -1.21% | ±0.42% -53.77% | | RowsBench | bench_take_1k_on_10k | 10 | 3 | 83.865mb +0.00% | 14.088μs +3.31% | ±0.67% -67.22% | | RowsBench | bench_take_right_1k_on_10k | 10 | 3 | 83.865mb +0.00% | 16.441μs +0.38% | ±1.53% +47.96% | | RowsBench | bench_unique_on_1k | 2 | 3 | 102.679mb +0.00% | 193.439ms +1.18% | ±0.64% +78.57% | | NativeEntryFactoryBench | bench_entry_factory | 1 | 3 | 116.810mb +0.00% | 496.405ms +0.82% | ±0.09% -88.73% | | NativeEntryFactoryBench | bench_entry_factory | 1 | 3 | 60.288mb +0.01% | 250.706ms +0.34% | ±2.93% +609.55% | | NativeEntryFactoryBench | bench_entry_factory | 1 | 3 | 15.222mb +0.02% | 53.906ms -0.74% | ±0.91% -72.04% | | TypeDetectorBench | bench_type_detector | 1 | 3 | 59.994mb +0.00% | 432.884ms -0.54% | ±0.66% -77.23% | | TypeDetectorBench | bench_type_detector | 1 | 3 | 14.533mb +0.01% | 86.826ms +0.39% | ±3.53% +523.00% | +-------------------------+----------------------------+------+-----+------------------+------------------+-----------------+ ```