flow-php / flow

Flow PHP - data processing framework
https://flow-php.com
MIT License
404 stars 23 forks source link

Improved schema manipulation #991

Closed norberttech closed 4 months ago

norberttech commented 4 months ago

Change Log

Added

  • Schema::remove
  • Schema::gracefulRemove
  • Schema::add
  • Schema::replace

Fixed

Changed

  • Schema became mutable object [BC Break]

Removed

Deprecated

  • Schema::without

Security


Description

github-actions[bot] commented 4 months ago

Flow PHP - Benchmarks

Results of the benchmarks from this PR are compared with the results from 1.x branch.

Extractors ```shell +-----------------------+-------------------+------+-----+------------------+------------------+-----------------+ | benchmark | subject | revs | its | mem_peak | mode | rstdev | +-----------------------+-------------------+------+-----+------------------+------------------+-----------------+ | AvroExtractorBench | bench_extract_10k | 1 | 3 | 35.269mb +0.00% | 827.620ms +0.42% | ±0.18% -74.26% | | CSVExtractorBench | bench_extract_10k | 1 | 3 | 4.989mb +0.01% | 341.457ms -0.06% | ±0.34% -36.11% | | JsonExtractorBench | bench_extract_10k | 1 | 3 | 5.139mb +0.01% | 1.037s -1.64% | ±0.52% -64.93% | | ParquetExtractorBench | bench_extract_10k | 1 | 3 | 135.794mb +0.01% | 913.049ms +2.85% | ±1.59% +182.01% | | TextExtractorBench | bench_extract_10k | 1 | 3 | 4.900mb +0.01% | 35.139ms -0.66% | ±0.19% -84.60% | | XmlExtractorBench | bench_extract_10k | 1 | 3 | 4.902mb +0.01% | 443.471ms +3.37% | ±0.68% -0.28% | +-----------------------+-------------------+------+-----+------------------+------------------+-----------------+ ```
Transformers ```shell +-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+ | benchmark | subject | revs | its | mem_peak | mode | rstdev | +-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+ | RenameEntryTransformerBench | bench_transform_10k_rows | 1 | 3 | 110.608mb +0.00% | 63.806ms -0.29% | ±1.34% -16.71% | +-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+ ```
Loaders ```shell +--------------------+----------------+------+-----+------------------+------------------+-----------------+ | benchmark | subject | revs | its | mem_peak | mode | rstdev | +--------------------+----------------+------+-----+------------------+------------------+-----------------+ | AvroLoaderBench | bench_load_10k | 1 | 3 | 95.649mb +0.00% | 469.048ms +0.02% | ±1.04% +2.04% | | CSVLoaderBench | bench_load_10k | 1 | 3 | 54.125mb +0.00% | 72.730ms +0.98% | ±0.97% +3.44% | | JsonLoaderBench | bench_load_10k | 1 | 3 | 106.554mb +0.00% | 52.788ms +1.86% | ±0.80% -43.10% | | ParquetLoaderBench | bench_load_10k | 1 | 3 | 224.362mb +0.01% | 1.416s +0.54% | ±0.33% +102.11% | | TextLoaderBench | bench_load_10k | 1 | 3 | 17.948mb +0.00% | 40.968ms +2.10% | ±0.39% -19.34% | +--------------------+----------------+------+-----+------------------+------------------+-----------------+ ```
Building Blocks ```shell +-------------------------+----------------------------+------+-----+------------------+------------------+------------------+ | benchmark | subject | revs | its | mem_peak | mode | rstdev | +-------------------------+----------------------------+------+-----+------------------+------------------+------------------+ | NativeEntryFactoryBench | bench_entry_factory | 1 | 3 | 116.715mb +0.00% | 490.023ms +0.70% | ±1.13% +1311.16% | | NativeEntryFactoryBench | bench_entry_factory | 1 | 3 | 60.193mb +0.00% | 248.582ms -1.31% | ±1.11% -55.30% | | NativeEntryFactoryBench | bench_entry_factory | 1 | 3 | 15.128mb +0.00% | 53.315ms +1.08% | ±2.95% +172.13% | | TypeDetectorBench | bench_type_detector | 1 | 3 | 59.959mb +0.00% | 433.906ms +0.70% | ±0.18% -53.06% | | TypeDetectorBench | bench_type_detector | 1 | 3 | 14.498mb +0.00% | 85.352ms -0.12% | ±1.15% +47.22% | | RowsBench | bench_chunk_10_on_10k | 2 | 3 | 76.683mb +0.00% | 3.377ms +3.00% | ±2.21% +66.74% | | RowsBench | bench_diff_left_1k_on_10k | 2 | 3 | 96.410mb +0.00% | 182.577ms +1.75% | ±0.60% -5.83% | | RowsBench | bench_diff_right_1k_on_10k | 2 | 3 | 74.936mb +0.00% | 17.972ms -2.60% | ±0.74% -58.94% | | RowsBench | bench_drop_1k_on_10k | 2 | 3 | 77.923mb +0.00% | 1.723ms +4.81% | ±3.44% +229.95% | | RowsBench | bench_drop_right_1k_on_10k | 2 | 3 | 77.923mb +0.00% | 1.753ms +4.33% | ±1.60% -54.13% | | RowsBench | bench_entries_on_10k | 2 | 3 | 75.035mb +0.00% | 2.568ms +1.91% | ±2.68% +167.08% | | RowsBench | bench_filter_on_10k | 2 | 3 | 75.564mb +0.00% | 14.542ms +2.72% | ±0.68% +171.68% | | RowsBench | bench_find_on_10k | 2 | 3 | 75.564mb +0.00% | 14.492ms +1.49% | ±0.84% -42.02% | | RowsBench | bench_find_one_on_10k | 10 | 3 | 73.468mb +0.00% | 1.894μs +5.57% | ±2.53% -5.36% | | RowsBench | bench_first_on_10k | 10 | 3 | 73.468mb +0.00% | 0.400μs 0.00% | ±0.00% 0.00% | | RowsBench | bench_flat_map_on_1k | 2 | 3 | 87.023mb +0.00% | 12.954ms +1.62% | ±1.14% +112.37% | | RowsBench | bench_map_on_10k | 2 | 3 | 116.384mb +0.00% | 65.505ms -0.11% | ±1.34% +213.14% | | RowsBench | bench_merge_1k_on_10k | 2 | 3 | 76.084mb +0.00% | 1.311ms -0.53% | ±2.14% +196.35% | | RowsBench | bench_partition_by_on_10k | 2 | 3 | 79.431mb +0.00% | 56.512ms -2.30% | ±0.40% -81.18% | | RowsBench | bench_remove_on_10k | 2 | 3 | 78.185mb +0.00% | 3.894ms -4.70% | ±1.24% -2.60% | | RowsBench | bench_sort_asc_on_1k | 2 | 3 | 73.546mb +0.00% | 39.773ms -0.82% | ±1.85% +200.57% | | RowsBench | bench_sort_by_on_1k | 2 | 3 | 73.547mb +0.00% | 40.181ms -3.49% | ±0.52% -69.56% | | RowsBench | bench_sort_desc_on_1k | 2 | 3 | 73.546mb +0.00% | 40.169ms -1.23% | ±1.44% +84.45% | | RowsBench | bench_sort_entries_on_1k | 2 | 3 | 75.910mb +0.00% | 7.253ms -0.56% | ±0.97% +22.30% | | RowsBench | bench_sort_on_1k | 2 | 3 | 73.468mb +0.00% | 28.875ms -0.90% | ±0.86% +90.49% | | RowsBench | bench_take_1k_on_10k | 10 | 3 | 73.468mb +0.00% | 13.543μs +1.64% | ±2.41% +72.01% | | RowsBench | bench_take_right_1k_on_10k | 10 | 3 | 73.468mb +0.00% | 15.900μs +0.63% | ±1.03% +98.74% | | RowsBench | bench_unique_on_1k | 2 | 3 | 96.477mb +0.00% | 185.335ms +0.57% | ±0.31% -49.27% | +-------------------------+----------------------------+------+-----+------------------+------------------+------------------+ ```