flow-php / flow

Flow PHP - data processing framework
https://flow-php.com
MIT License
404 stars 23 forks source link

Added Schema::keep - to keep selected schema entries #996

Closed norberttech closed 4 months ago

norberttech commented 4 months ago

Change Log

Added

  • Schema::keep() - keep selected schema entries

Fixed

Changed

Removed

Deprecated

Security


Description

github-actions[bot] commented 4 months ago

Flow PHP - Benchmarks

Results of the benchmarks from this PR are compared with the results from 1.x branch.

Extractors ```shell +-----------------------+-------------------+------+-----+------------------+------------------+-----------------+ | benchmark | subject | revs | its | mem_peak | mode | rstdev | +-----------------------+-------------------+------+-----+------------------+------------------+-----------------+ | AvroExtractorBench | bench_extract_10k | 1 | 3 | 35.279mb +0.00% | 825.664ms +2.16% | ±2.48% +184.08% | | CSVExtractorBench | bench_extract_10k | 1 | 3 | 5.002mb +0.01% | 342.336ms +0.82% | ±0.43% +58.65% | | JsonExtractorBench | bench_extract_10k | 1 | 3 | 5.152mb +0.01% | 1.061s +1.07% | ±0.52% -40.49% | | ParquetExtractorBench | bench_extract_10k | 1 | 3 | 135.815mb +0.00% | 899.080ms +1.98% | ±0.71% -27.33% | | TextExtractorBench | bench_extract_10k | 1 | 3 | 4.910mb +0.01% | 35.894ms +2.26% | ±1.93% +176.71% | | XmlExtractorBench | bench_extract_10k | 1 | 3 | 4.915mb +0.01% | 432.641ms +0.55% | ±0.75% +14.47% | +-----------------------+-------------------+------+-----+------------------+------------------+-----------------+ ```
Transformers ```shell +-----------------------------+--------------------------+------+-----+------------------+-----------------+-----------------+ | benchmark | subject | revs | its | mem_peak | mode | rstdev | +-----------------------------+--------------------------+------+-----+------------------+-----------------+-----------------+ | RenameEntryTransformerBench | bench_transform_10k_rows | 1 | 3 | 110.616mb +0.00% | 63.871ms -1.20% | ±2.68% +894.32% | +-----------------------------+--------------------------+------+-----+------------------+-----------------+-----------------+ ```
Loaders ```shell +--------------------+----------------+------+-----+------------------+------------------+----------------+ | benchmark | subject | revs | its | mem_peak | mode | rstdev | +--------------------+----------------+------+-----+------------------+------------------+----------------+ | AvroLoaderBench | bench_load_10k | 1 | 3 | 95.659mb +0.00% | 466.592ms +0.57% | ±0.41% -29.89% | | CSVLoaderBench | bench_load_10k | 1 | 3 | 54.141mb +0.00% | 72.174ms +3.45% | ±1.25% -32.22% | | JsonLoaderBench | bench_load_10k | 1 | 3 | 106.567mb +0.00% | 53.633ms +2.39% | ±1.32% +14.84% | | ParquetLoaderBench | bench_load_10k | 1 | 3 | 224.383mb +0.00% | 1.442s +2.25% | ±0.57% +65.64% | | TextLoaderBench | bench_load_10k | 1 | 3 | 17.957mb +0.00% | 39.904ms +5.00% | ±0.12% -60.19% | +--------------------+----------------+------+-----+------------------+------------------+----------------+ ```
Building Blocks ```shell +-------------------------+----------------------------+------+-----+------------------+------------------+------------------+ | benchmark | subject | revs | its | mem_peak | mode | rstdev | +-------------------------+----------------------------+------+-----+------------------+------------------+------------------+ | NativeEntryFactoryBench | bench_entry_factory | 1 | 3 | 116.714mb +0.00% | 486.733ms -0.21% | ±1.36% +84.82% | | NativeEntryFactoryBench | bench_entry_factory | 1 | 3 | 60.192mb +0.00% | 250.167ms +0.80% | ±0.86% -11.37% | | NativeEntryFactoryBench | bench_entry_factory | 1 | 3 | 15.127mb +0.00% | 54.119ms +3.59% | ±0.52% +46.03% | | TypeDetectorBench | bench_type_detector | 1 | 3 | 59.958mb +0.00% | 436.102ms +3.05% | ±1.33% +176.95% | | TypeDetectorBench | bench_type_detector | 1 | 3 | 14.497mb +0.00% | 87.586ms +2.51% | ±0.38% -7.72% | | RowsBench | bench_chunk_10_on_10k | 2 | 3 | 76.682mb +0.00% | 3.909ms +18.78% | ±2.04% +49.71% | | RowsBench | bench_diff_left_1k_on_10k | 2 | 3 | 96.409mb +0.00% | 180.659ms -0.10% | ±1.01% -17.96% | | RowsBench | bench_diff_right_1k_on_10k | 2 | 3 | 74.935mb +0.00% | 17.943ms -1.50% | ±0.81% +286.67% | | RowsBench | bench_drop_1k_on_10k | 2 | 3 | 77.922mb +0.00% | 1.838ms +16.82% | ±3.72% +151.63% | | RowsBench | bench_drop_right_1k_on_10k | 2 | 3 | 77.922mb +0.00% | 1.861ms +17.67% | ±3.76% +159.28% | | RowsBench | bench_entries_on_10k | 2 | 3 | 75.034mb +0.00% | 3.024ms +22.69% | ±2.55% +335.24% | | RowsBench | bench_filter_on_10k | 2 | 3 | 75.563mb +0.00% | 15.047ms +5.61% | ±1.85% +49.26% | | RowsBench | bench_find_on_10k | 2 | 3 | 75.563mb +0.00% | 15.656ms +10.85% | ±0.64% -78.58% | | RowsBench | bench_find_one_on_10k | 10 | 3 | 73.468mb +0.00% | 1.906μs +5.89% | ±2.44% +0.00% | | RowsBench | bench_first_on_10k | 10 | 3 | 73.468mb +0.00% | 0.400μs 0.00% | ±0.00% 0.00% | | RowsBench | bench_flat_map_on_1k | 2 | 3 | 87.022mb +0.00% | 12.935ms -0.26% | ±0.64% -31.58% | | RowsBench | bench_map_on_10k | 2 | 3 | 116.383mb +0.00% | 67.831ms +3.84% | ±2.09% +333.39% | | RowsBench | bench_merge_1k_on_10k | 2 | 3 | 76.083mb +0.00% | 1.503ms +20.45% | ±2.24% +128.92% | | RowsBench | bench_partition_by_on_10k | 2 | 3 | 79.430mb +0.00% | 60.226ms +6.55% | ±0.85% -30.49% | | RowsBench | bench_remove_on_10k | 2 | 3 | 78.185mb +0.00% | 3.975ms +4.98% | ±0.65% +46.66% | | RowsBench | bench_sort_asc_on_1k | 2 | 3 | 73.546mb +0.00% | 39.858ms -1.97% | ±0.27% -45.29% | | RowsBench | bench_sort_by_on_1k | 2 | 3 | 73.546mb +0.00% | 40.446ms -0.02% | ±0.83% -49.34% | | RowsBench | bench_sort_desc_on_1k | 2 | 3 | 73.546mb +0.00% | 41.189ms +1.10% | ±1.59% +66.93% | | RowsBench | bench_sort_entries_on_1k | 2 | 3 | 75.909mb +0.00% | 7.735ms +4.92% | ±2.87% +2808.10% | | RowsBench | bench_sort_on_1k | 2 | 3 | 73.468mb +0.00% | 28.935ms -0.36% | ±1.19% +292.29% | | RowsBench | bench_take_1k_on_10k | 10 | 3 | 73.468mb +0.00% | 13.359μs -0.16% | ±1.85% +46.76% | | RowsBench | bench_take_right_1k_on_10k | 10 | 3 | 73.468mb +0.00% | 16.620μs +4.45% | ±1.03% +74.10% | | RowsBench | bench_unique_on_1k | 2 | 3 | 96.476mb +0.00% | 184.913ms -0.37% | ±0.50% -45.78% | +-------------------------+----------------------------+------+-----+------------------+------------------+------------------+ ```