flow-php / flow

Flow PHP - data processing framework
https://flow-php.com
MIT License
404 stars 23 forks source link

Added possibility to get a hash from an entire row or all rows #957

Closed norberttech closed 5 months ago

norberttech commented 5 months ago

Change Log

Added

  • Row::hash and Rows::hash

Fixed

Changed

Removed

Deprecated

Security


Description

github-actions[bot] commented 5 months ago

Flow PHP - Benchmarks

Results of the benchmarks from this PR are compared with the results from 1.x branch.

Extractors ```shell +-----------------------+-------------------+------+-----+------------------+------------------+-----------------+ | benchmark | subject | revs | its | mem_peak | mode | rstdev | +-----------------------+-------------------+------+-----+------------------+------------------+-----------------+ | AvroExtractorBench | bench_extract_10k | 1 | 3 | 35.264mb +0.01% | 830.306ms +1.19% | ±0.47% -77.39% | | CSVExtractorBench | bench_extract_10k | 1 | 3 | 4.984mb +0.10% | 343.089ms +0.19% | ±0.57% +331.80% | | JsonExtractorBench | bench_extract_10k | 1 | 3 | 5.134mb +0.10% | 1.056s +0.83% | ±0.23% -89.85% | | ParquetExtractorBench | bench_extract_10k | 1 | 3 | 239.838mb +0.00% | 1.265s +0.82% | ±0.68% +22.96% | | TextExtractorBench | bench_extract_10k | 1 | 3 | 4.903mb +0.01% | 35.280ms +0.89% | ±1.08% -7.66% | | XmlExtractorBench | bench_extract_10k | 1 | 3 | 4.904mb +0.01% | 434.567ms +0.38% | ±1.17% +172.48% | +-----------------------+-------------------+------+-----+------------------+------------------+-----------------+ ```
Transformers ```shell +-----------------------------+--------------------------+------+-----+------------------+-----------------+------------------+ | benchmark | subject | revs | its | mem_peak | mode | rstdev | +-----------------------------+--------------------------+------+-----+------------------+-----------------+------------------+ | RenameEntryTransformerBench | bench_transform_10k_rows | 1 | 3 | 110.604mb +0.00% | 66.152ms -0.44% | ±0.84% +1000.18% | +-----------------------------+--------------------------+------+-----+------------------+-----------------+------------------+ ```
Loaders ```shell +--------------------+----------------+------+-----+------------------+------------------+------------------+ | benchmark | subject | revs | its | mem_peak | mode | rstdev | +--------------------+----------------+------+-----+------------------+------------------+------------------+ | AvroLoaderBench | bench_load_10k | 1 | 3 | 95.645mb +0.01% | 470.077ms +2.45% | ±0.96% +31.57% | | CSVLoaderBench | bench_load_10k | 1 | 3 | 54.120mb +0.01% | 72.616ms +0.47% | ±2.15% +1602.26% | | JsonLoaderBench | bench_load_10k | 1 | 3 | 106.549mb +0.00% | 54.511ms +2.75% | ±1.13% +169.22% | | ParquetLoaderBench | bench_load_10k | 1 | 3 | 321.760mb +0.00% | 1.514s +2.27% | ±0.68% +309.66% | | TextLoaderBench | bench_load_10k | 1 | 3 | 17.943mb +0.03% | 40.997ms +0.89% | ±0.44% -68.34% | +--------------------+----------------+------+-----+------------------+------------------+------------------+ ```
Building Blocks ```shell +-------------------------+----------------------------+------+-----+------------------+------------------+-----------------+ | benchmark | subject | revs | its | mem_peak | mode | rstdev | +-------------------------+----------------------------+------+-----+------------------+------------------+-----------------+ | NativeEntryFactoryBench | bench_entry_factory | 1 | 3 | 116.710mb +0.00% | 503.896ms +4.46% | ±0.54% -56.54% | | NativeEntryFactoryBench | bench_entry_factory | 1 | 3 | 60.189mb +0.01% | 251.380ms +0.92% | ±1.57% -7.50% | | NativeEntryFactoryBench | bench_entry_factory | 1 | 3 | 15.122mb +0.04% | 53.608ms +1.23% | ±1.20% +70.99% | | TypeDetectorBench | bench_type_detector | 1 | 3 | 59.961mb +0.00% | 435.288ms -0.98% | ±0.45% -37.77% | | TypeDetectorBench | bench_type_detector | 1 | 3 | 14.500mb +0.00% | 86.604ms -0.54% | ±1.27% +25.24% | | RowsBench | bench_chunk_10_on_10k | 2 | 3 | 76.679mb +0.01% | 3.949ms +4.50% | ±0.57% -68.51% | | RowsBench | bench_diff_left_1k_on_10k | 2 | 3 | 96.405mb +0.01% | 183.699ms -3.70% | ±0.08% -82.05% | | RowsBench | bench_diff_right_1k_on_10k | 2 | 3 | 74.931mb +0.01% | 18.747ms -2.23% | ±1.26% +33.63% | | RowsBench | bench_drop_1k_on_10k | 2 | 3 | 77.919mb +0.01% | 1.828ms -3.44% | ±3.67% +70.04% | | RowsBench | bench_drop_right_1k_on_10k | 2 | 3 | 77.919mb +0.01% | 1.842ms -2.26% | ±2.62% -2.53% | | RowsBench | bench_entries_on_10k | 2 | 3 | 75.031mb +0.01% | 2.782ms +1.07% | ±3.68% +37.45% | | RowsBench | bench_filter_on_10k | 2 | 3 | 75.560mb +0.01% | 15.267ms +1.89% | ±1.96% +98.01% | | RowsBench | bench_find_on_10k | 2 | 3 | 75.560mb +0.01% | 15.078ms -0.22% | ±1.97% +315.27% | | RowsBench | bench_find_one_on_10k | 10 | 3 | 73.464mb +0.01% | 2.100μs +5.32% | ±0.00% -100.00% | | RowsBench | bench_first_on_10k | 10 | 3 | 73.464mb +0.01% | 0.300μs -25.00% | ±0.00% -100.00% | | RowsBench | bench_flat_map_on_1k | 2 | 3 | 87.018mb +0.01% | 14.266ms +1.75% | ±3.18% +3.16% | | RowsBench | bench_map_on_10k | 2 | 3 | 116.379mb +0.00% | 65.435ms -0.05% | ±0.11% -91.36% | | RowsBench | bench_merge_1k_on_10k | 2 | 3 | 76.079mb +0.01% | 1.646ms +16.62% | ±2.95% +52.73% | | RowsBench | bench_partition_by_on_10k | 2 | 3 | 79.426mb +0.01% | 59.290ms -2.78% | ±0.76% +74.91% | | RowsBench | bench_remove_on_10k | 2 | 3 | 78.181mb +0.01% | 4.021ms +3.06% | ±3.33% +26.01% | | RowsBench | bench_sort_asc_on_1k | 2 | 3 | 73.542mb +0.01% | 40.719ms -0.83% | ±0.45% -15.26% | | RowsBench | bench_sort_by_on_1k | 2 | 3 | 73.542mb +0.01% | 41.893ms +2.29% | ±1.62% +388.16% | | RowsBench | bench_sort_desc_on_1k | 2 | 3 | 73.542mb +0.01% | 42.173ms +1.95% | ±0.88% -5.12% | | RowsBench | bench_sort_entries_on_1k | 2 | 3 | 75.905mb +0.01% | 7.514ms +1.23% | ±1.38% +128.50% | | RowsBench | bench_sort_on_1k | 2 | 3 | 73.464mb +0.01% | 30.175ms +3.79% | ±2.41% +155.20% | | RowsBench | bench_take_1k_on_10k | 10 | 3 | 73.464mb +0.01% | 14.312μs +7.41% | ±1.85% +31.99% | | RowsBench | bench_take_right_1k_on_10k | 10 | 3 | 73.464mb +0.01% | 16.500μs +4.04% | ±0.49% -68.33% | | RowsBench | bench_unique_on_1k | 2 | 3 | 96.407mb +0.07% | 186.703ms -1.18% | ±0.81% +17.47% | +-------------------------+----------------------------+------+-----+------------------+------------------+-----------------+ ```