flow-php / flow

Flow PHP - data processing framework
https://flow-php.com
MIT License
404 stars 23 forks source link

Fixed changing schema of dataset when using group by #1079

Closed norberttech closed 1 month ago

norberttech commented 1 month ago

Change Log

Added

  • Enabled parallelization for cs fixer
  • Allowed to pass both string/reference to aggregating functions

Fixed

  • Group by will no longer change type of entries used for groupping

Changed

Removed

Deprecated

Security


Description

github-actions[bot] commented 1 month ago

Flow PHP - Benchmarks

Results of the benchmarks from this PR are compared with the results from 1.x branch.

Extractors ```shell +-----------------------+-------------------+------+-----+------------------+------------------+----------------+ | benchmark | subject | revs | its | mem_peak | mode | rstdev | +-----------------------+-------------------+------+-----+------------------+------------------+----------------+ | AvroExtractorBench | bench_extract_10k | 1 | 3 | 35.422mb +0.04% | 840.600ms +2.77% | ±0.87% -63.35% | | CSVExtractorBench | bench_extract_10k | 1 | 3 | 5.142mb +0.30% | 337.578ms +1.00% | ±0.08% -82.61% | | JsonExtractorBench | bench_extract_10k | 1 | 3 | 5.173mb +0.30% | 1.081s +0.84% | ±1.06% +93.55% | | ParquetExtractorBench | bench_extract_10k | 1 | 3 | 135.845mb +0.01% | 903.490ms +0.36% | ±0.23% -76.14% | | TextExtractorBench | bench_extract_10k | 1 | 3 | 4.933mb +0.31% | 35.552ms -4.81% | ±1.09% -51.95% | | XmlExtractorBench | bench_extract_10k | 1 | 3 | 4.939mb +0.31% | 434.777ms -0.81% | ±0.53% -73.67% | +-----------------------+-------------------+------+-----+------------------+------------------+----------------+ ```
Transformers ```shell +-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+ | benchmark | subject | revs | its | mem_peak | mode | rstdev | +-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+ | RenameEntryTransformerBench | bench_transform_10k_rows | 1 | 3 | 116.239mb +0.01% | 60.559ms +2.47% | ±0.39% -55.77% | +-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+ ```
Loaders ```shell +--------------------+----------------+------+-----+------------------+------------------+----------------+ | benchmark | subject | revs | its | mem_peak | mode | rstdev | +--------------------+----------------+------+-----+------------------+------------------+----------------+ | AvroLoaderBench | bench_load_10k | 1 | 3 | 96.807mb +0.02% | 449.489ms -0.33% | ±0.47% -70.80% | | CSVLoaderBench | bench_load_10k | 1 | 3 | 55.220mb +0.03% | 68.952ms +2.77% | ±0.81% +13.78% | | JsonLoaderBench | bench_load_10k | 1 | 3 | 107.593mb +0.01% | 52.564ms +2.73% | ±0.77% -31.50% | | ParquetLoaderBench | bench_load_10k | 1 | 3 | 227.013mb +0.01% | 1.426s +2.04% | ±0.27% -27.08% | | TextLoaderBench | bench_load_10k | 1 | 3 | 17.976mb +0.09% | 38.907ms +1.25% | ±0.49% +3.40% | +--------------------+----------------+------+-----+------------------+------------------+----------------+ ```
Building Blocks ```shell +-------------------------+----------------------------+------+-----+------------------+------------------+-----------------+ | benchmark | subject | revs | its | mem_peak | mode | rstdev | +-------------------------+----------------------------+------+-----+------------------+------------------+-----------------+ | RowsBench | bench_chunk_10_on_10k | 2 | 3 | 87.059mb +0.02% | 3.833ms +14.59% | ±1.20% -13.69% | | RowsBench | bench_diff_left_1k_on_10k | 2 | 3 | 102.657mb +0.02% | 185.218ms -0.79% | ±0.68% -67.99% | | RowsBench | bench_diff_right_1k_on_10k | 2 | 3 | 85.377mb +0.02% | 19.125ms +2.98% | ±0.63% -32.23% | | RowsBench | bench_drop_1k_on_10k | 2 | 3 | 88.299mb +0.02% | 2.200ms +29.30% | ±2.88% +71.68% | | RowsBench | bench_drop_right_1k_on_10k | 2 | 3 | 88.299mb +0.02% | 2.282ms +32.22% | ±2.41% +60.22% | | RowsBench | bench_entries_on_10k | 2 | 3 | 85.411mb +0.02% | 3.298ms +31.21% | ±2.97% -10.75% | | RowsBench | bench_filter_on_10k | 2 | 3 | 85.940mb +0.02% | 15.599ms -3.80% | ±1.31% +23.03% | | RowsBench | bench_find_on_10k | 2 | 3 | 85.940mb +0.02% | 15.674ms -1.50% | ±0.50% -5.43% | | RowsBench | bench_find_one_on_10k | 10 | 3 | 83.845mb +0.02% | 2.000μs +25.48% | ±0.00% -100.00% | | RowsBench | bench_first_on_10k | 10 | 3 | 83.845mb +0.02% | 0.400μs +33.33% | ±0.00% +0.00% | | RowsBench | bench_flat_map_on_1k | 2 | 3 | 93.194mb +0.02% | 12.773ms +4.91% | ±0.26% -31.66% | | RowsBench | bench_map_on_10k | 2 | 3 | 122.565mb +0.01% | 62.455ms +4.23% | ±0.89% -34.46% | | RowsBench | bench_merge_1k_on_10k | 2 | 3 | 86.460mb +0.02% | 1.688ms +37.92% | ±1.25% -19.20% | | RowsBench | bench_partition_by_on_10k | 2 | 3 | 89.807mb +0.02% | 62.815ms +0.51% | ±0.70% -23.45% | | RowsBench | bench_remove_on_10k | 2 | 3 | 88.561mb +0.02% | 4.479ms +16.38% | ±0.82% +95.63% | | RowsBench | bench_sort_asc_on_1k | 2 | 3 | 83.988mb +0.02% | 40.422ms +2.56% | ±2.73% +180.26% | | RowsBench | bench_sort_by_on_1k | 2 | 3 | 83.989mb +0.02% | 40.182ms +2.80% | ±2.02% +55.55% | | RowsBench | bench_sort_desc_on_1k | 2 | 3 | 83.988mb +0.02% | 40.618ms +4.39% | ±1.29% +11.38% | | RowsBench | bench_sort_entries_on_1k | 2 | 3 | 86.286mb +0.02% | 7.674ms +4.93% | ±1.97% +475.88% | | RowsBench | bench_sort_on_1k | 2 | 3 | 83.845mb +0.02% | 30.432ms +6.83% | ±1.48% +577.51% | | RowsBench | bench_take_1k_on_10k | 10 | 3 | 83.845mb +0.02% | 13.780μs +5.19% | ±1.23% -1.67% | | RowsBench | bench_take_right_1k_on_10k | 10 | 3 | 83.845mb +0.02% | 16.688μs +6.89% | ±0.57% -5.81% | | RowsBench | bench_unique_on_1k | 2 | 3 | 102.659mb +0.02% | 198.344ms +3.58% | ±1.13% -48.07% | | NativeEntryFactoryBench | bench_entry_factory | 1 | 3 | 116.789mb +0.01% | 498.845ms -0.43% | ±1.15% +12.69% | | NativeEntryFactoryBench | bench_entry_factory | 1 | 3 | 60.267mb +0.03% | 252.934ms +1.49% | ±0.63% +33.93% | | NativeEntryFactoryBench | bench_entry_factory | 1 | 3 | 15.201mb +0.10% | 55.857ms +5.06% | ±0.47% -22.86% | | TypeDetectorBench | bench_type_detector | 1 | 3 | 59.975mb +0.03% | 437.555ms +1.37% | ±1.22% -57.84% | | TypeDetectorBench | bench_type_detector | 1 | 3 | 14.514mb +0.11% | 87.367ms +2.66% | ±0.94% +26.28% | +-------------------------+----------------------------+------+-----+------------------+------------------+-----------------+ ```