flow-php / flow

Flow PHP - data processing framework
https://flow-php.com
MIT License
404 stars 23 forks source link

Added missing tests for parquet rows normalizer #1054

Closed norberttech closed 2 months ago

norberttech commented 2 months ago

Change Log

Added

  • missing tests for parquet rows normalizer

Fixed

Changed

Removed

Deprecated

Security


Description

github-actions[bot] commented 2 months ago

Flow PHP - Benchmarks

Results of the benchmarks from this PR are compared with the results from 1.x branch.

Extractors ```shell +-----------------------+-------------------+------+-----+------------------+------------------+-----------------+ | benchmark | subject | revs | its | mem_peak | mode | rstdev | +-----------------------+-------------------+------+-----+------------------+------------------+-----------------+ | AvroExtractorBench | bench_extract_10k | 1 | 3 | 35.299mb +0.00% | 853.750ms -3.90% | ±0.91% -57.97% | | CSVExtractorBench | bench_extract_10k | 1 | 3 | 5.019mb +0.01% | 345.395ms +0.19% | ±0.35% -58.99% | | JsonExtractorBench | bench_extract_10k | 1 | 3 | 5.174mb +0.01% | 1.073s -0.06% | ±2.19% +563.88% | | ParquetExtractorBench | bench_extract_10k | 1 | 3 | 135.846mb +0.00% | 921.819ms -0.55% | ±0.87% -54.69% | | TextExtractorBench | bench_extract_10k | 1 | 3 | 4.928mb +0.01% | 35.025ms -3.16% | ±1.06% +19.18% | | XmlExtractorBench | bench_extract_10k | 1 | 3 | 4.934mb +0.01% | 430.850ms -1.08% | ±0.72% -76.63% | +-----------------------+-------------------+------+-----+------------------+------------------+-----------------+ ```
Transformers ```shell +-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+ | benchmark | subject | revs | its | mem_peak | mode | rstdev | +-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+ | RenameEntryTransformerBench | bench_transform_10k_rows | 1 | 3 | 116.240mb +0.00% | 60.802ms -2.11% | ±0.46% -38.32% | +-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+ ```
Loaders ```shell +--------------------+----------------+------+-----+------------------+------------------+-----------------+ | benchmark | subject | revs | its | mem_peak | mode | rstdev | +--------------------+----------------+------+-----+------------------+------------------+-----------------+ | AvroLoaderBench | bench_load_10k | 1 | 3 | 96.685mb +0.00% | 459.912ms -1.21% | ±0.63% -46.18% | | CSVLoaderBench | bench_load_10k | 1 | 3 | 55.164mb +0.00% | 69.084ms +0.21% | ±0.45% -31.72% | | JsonLoaderBench | bench_load_10k | 1 | 3 | 107.594mb +0.00% | 51.333ms -1.49% | ±0.91% +108.45% | | ParquetLoaderBench | bench_load_10k | 1 | 3 | 227.013mb +0.00% | 1.427s -0.21% | ±0.71% +187.61% | | TextLoaderBench | bench_load_10k | 1 | 3 | 17.976mb +0.00% | 38.662ms -1.54% | ±0.69% -29.37% | +--------------------+----------------+------+-----+------------------+------------------+-----------------+ ```
Building Blocks ```shell +-------------------------+----------------------------+------+-----+------------------+------------------+-----------------+ | benchmark | subject | revs | its | mem_peak | mode | rstdev | +-------------------------+----------------------------+------+-----+------------------+------------------+-----------------+ | RowsBench | bench_chunk_10_on_10k | 2 | 3 | 87.060mb +0.00% | 3.380ms -8.96% | ±3.36% +74.34% | | RowsBench | bench_diff_left_1k_on_10k | 2 | 3 | 102.658mb +0.00% | 188.673ms +0.65% | ±0.42% +63.47% | | RowsBench | bench_diff_right_1k_on_10k | 2 | 3 | 85.378mb +0.00% | 18.766ms +0.26% | ±0.80% +169.48% | | RowsBench | bench_drop_1k_on_10k | 2 | 3 | 88.300mb +0.00% | 1.777ms -1.73% | ±1.98% +28.28% | | RowsBench | bench_drop_right_1k_on_10k | 2 | 3 | 88.300mb +0.00% | 1.826ms +0.99% | ±0.78% -57.66% | | RowsBench | bench_entries_on_10k | 2 | 3 | 85.412mb +0.00% | 2.772ms +0.48% | ±1.46% -50.23% | | RowsBench | bench_filter_on_10k | 2 | 3 | 85.941mb +0.00% | 16.518ms -2.06% | ±2.13% +140.31% | | RowsBench | bench_find_on_10k | 2 | 3 | 85.941mb +0.00% | 16.653ms -2.05% | ±1.01% -20.64% | | RowsBench | bench_find_one_on_10k | 10 | 3 | 83.846mb +0.00% | 1.900μs 0.00% | ±0.00% 0.00% | | RowsBench | bench_first_on_10k | 10 | 3 | 83.846mb +0.00% | 0.400μs 0.00% | ±0.00% 0.00% | | RowsBench | bench_flat_map_on_1k | 2 | 3 | 93.195mb +0.00% | 12.606ms -0.25% | ±3.02% +15.08% | | RowsBench | bench_map_on_10k | 2 | 3 | 122.566mb +0.00% | 62.311ms -2.01% | ±0.28% -67.06% | | RowsBench | bench_merge_1k_on_10k | 2 | 3 | 86.461mb +0.00% | 1.460ms -6.17% | ±0.45% +120.11% | | RowsBench | bench_partition_by_on_10k | 2 | 3 | 89.808mb +0.00% | 65.294ms -4.07% | ±0.43% -75.22% | | RowsBench | bench_remove_on_10k | 2 | 3 | 88.562mb +0.00% | 4.049ms +0.14% | ±3.60% +153.23% | | RowsBench | bench_sort_asc_on_1k | 2 | 3 | 83.989mb +0.00% | 40.584ms +2.37% | ±0.59% -58.22% | | RowsBench | bench_sort_by_on_1k | 2 | 3 | 83.990mb +0.00% | 40.297ms +0.70% | ±2.12% +393.61% | | RowsBench | bench_sort_desc_on_1k | 2 | 3 | 83.989mb +0.00% | 41.054ms +2.64% | ±0.30% -43.72% | | RowsBench | bench_sort_entries_on_1k | 2 | 3 | 86.287mb +0.00% | 7.675ms +4.44% | ±2.72% +95.65% | | RowsBench | bench_sort_on_1k | 2 | 3 | 83.846mb +0.00% | 29.445ms -1.19% | ±1.68% +18.92% | | RowsBench | bench_take_1k_on_10k | 10 | 3 | 83.846mb +0.00% | 13.782μs +2.24% | ±1.03% -17.60% | | RowsBench | bench_take_right_1k_on_10k | 10 | 3 | 83.846mb +0.00% | 16.317μs 0.00% | ±1.33% 0.00% | | RowsBench | bench_unique_on_1k | 2 | 3 | 102.660mb +0.00% | 197.260ms +2.40% | ±0.69% -57.73% | | NativeEntryFactoryBench | bench_entry_factory | 1 | 3 | 116.738mb +0.00% | 518.847ms -1.29% | ±0.64% +67.97% | | NativeEntryFactoryBench | bench_entry_factory | 1 | 3 | 60.216mb +0.00% | 264.263ms +2.99% | ±1.50% +97.01% | | NativeEntryFactoryBench | bench_entry_factory | 1 | 3 | 15.150mb +0.00% | 56.700ms +2.42% | ±1.09% +93.57% | | TypeDetectorBench | bench_type_detector | 1 | 3 | 59.970mb +0.00% | 436.122ms -0.48% | ±0.12% -75.06% | | TypeDetectorBench | bench_type_detector | 1 | 3 | 14.509mb +0.00% | 87.371ms +2.02% | ±2.92% +320.17% | +-------------------------+----------------------------+------+-----+------------------+------------------+-----------------+ ```