flow-php / flow

Flow PHP - data processing framework
https://flow-php.com
MIT License
404 stars 23 forks source link

Added missing rdsl dependency #983

Closed norberttech closed 4 months ago

norberttech commented 4 months ago

Change Log

Added

Fixed

  • Added missing rdsl dependency

Changed

Removed

Deprecated

Security


Description

RDSL should be considered as a part of ETL, it's exposed as a standalone library since it's just a generic mechanism allowing to define functions execution chain but it's some that should be always present with the ETL.

github-actions[bot] commented 4 months ago

Flow PHP - Benchmarks

Results of the benchmarks from this PR are compared with the results from 1.x branch.

Extractors ```shell +-----------------------+-------------------+------+-----+------------------+------------------+-----------------+ | benchmark | subject | revs | its | mem_peak | mode | rstdev | +-----------------------+-------------------+------+-----+------------------+------------------+-----------------+ | AvroExtractorBench | bench_extract_10k | 1 | 3 | 35.272mb -0.03% | 812.800ms +0.35% | ±0.87% +231.19% | | CSVExtractorBench | bench_extract_10k | 1 | 3 | 4.992mb -0.21% | 339.794ms -0.58% | ±0.12% +5.74% | | JsonExtractorBench | bench_extract_10k | 1 | 3 | 5.142mb -0.20% | 1.049s -0.25% | ±0.53% +42.74% | | ParquetExtractorBench | bench_extract_10k | 1 | 3 | 135.798mb -0.01% | 899.885ms +0.97% | ±1.66% +140.53% | | TextExtractorBench | bench_extract_10k | 1 | 3 | 4.906mb -0.21% | 35.106ms -0.15% | ±0.56% -34.62% | | XmlExtractorBench | bench_extract_10k | 1 | 3 | 4.907mb -0.21% | 432.016ms +0.04% | ±0.45% -76.76% | +-----------------------+-------------------+------+-----+------------------+------------------+-----------------+ ```
Transformers ```shell +-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+ | benchmark | subject | revs | its | mem_peak | mode | rstdev | +-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+ | RenameEntryTransformerBench | bench_transform_10k_rows | 1 | 3 | 110.612mb -0.01% | 64.841ms +0.02% | ±1.34% -28.37% | +-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+ ```
Loaders ```shell +--------------------+----------------+------+-----+------------------+------------------+-----------------+ | benchmark | subject | revs | its | mem_peak | mode | rstdev | +--------------------+----------------+------+-----+------------------+------------------+-----------------+ | AvroLoaderBench | bench_load_10k | 1 | 3 | 95.653mb -0.01% | 457.972ms -1.40% | ±0.63% -48.36% | | CSVLoaderBench | bench_load_10k | 1 | 3 | 54.128mb -0.02% | 72.264ms +1.33% | ±0.74% +87.95% | | JsonLoaderBench | bench_load_10k | 1 | 3 | 106.557mb -0.01% | 51.906ms -1.01% | ±0.42% +21.86% | | ParquetLoaderBench | bench_load_10k | 1 | 3 | 224.367mb -0.00% | 1.423s +1.09% | ±0.60% +236.76% | | TextLoaderBench | bench_load_10k | 1 | 3 | 17.951mb -0.06% | 40.533ms -0.12% | ±0.20% -73.38% | +--------------------+----------------+------+-----+------------------+------------------+-----------------+ ```
Building Blocks ```shell +-------------------------+----------------------------+------+-----+------------------+------------------+------------------+ | benchmark | subject | revs | its | mem_peak | mode | rstdev | +-------------------------+----------------------------+------+-----+------------------+------------------+------------------+ | RowsBench | bench_chunk_10_on_10k | 2 | 3 | 76.687mb -0.01% | 3.494ms +3.09% | ±2.67% +78.09% | | RowsBench | bench_diff_left_1k_on_10k | 2 | 3 | 96.413mb -0.01% | 180.716ms -0.75% | ±0.58% -30.89% | | RowsBench | bench_diff_right_1k_on_10k | 2 | 3 | 74.939mb -0.01% | 18.148ms -0.15% | ±1.12% +158.87% | | RowsBench | bench_drop_1k_on_10k | 2 | 3 | 77.926mb -0.01% | 1.751ms -0.71% | ±2.15% +144.54% | | RowsBench | bench_drop_right_1k_on_10k | 2 | 3 | 77.926mb -0.01% | 1.828ms +8.03% | ±2.09% +323.10% | | RowsBench | bench_entries_on_10k | 2 | 3 | 75.039mb -0.01% | 2.546ms -4.60% | ±1.67% -18.97% | | RowsBench | bench_filter_on_10k | 2 | 3 | 75.568mb -0.01% | 15.292ms +4.63% | ±0.88% -18.26% | | RowsBench | bench_find_on_10k | 2 | 3 | 75.568mb -0.01% | 15.311ms +5.82% | ±1.53% +20.93% | | RowsBench | bench_find_one_on_10k | 10 | 3 | 73.472mb -0.01% | 1.794μs +5.15% | ±2.67% -1.89% | | RowsBench | bench_first_on_10k | 10 | 3 | 73.472mb -0.01% | 0.400μs 0.00% | ±0.00% 0.00% | | RowsBench | bench_flat_map_on_1k | 2 | 3 | 87.026mb -0.01% | 12.694ms -0.71% | ±1.95% +36.18% | | RowsBench | bench_map_on_10k | 2 | 3 | 116.387mb -0.01% | 65.266ms +2.08% | ±0.57% +22.48% | | RowsBench | bench_merge_1k_on_10k | 2 | 3 | 76.087mb -0.01% | 1.251ms -16.14% | ±3.33% +2237.42% | | RowsBench | bench_partition_by_on_10k | 2 | 3 | 79.434mb -0.01% | 57.333ms -0.20% | ±0.99% +22.49% | | RowsBench | bench_remove_on_10k | 2 | 3 | 78.189mb -0.01% | 3.961ms +1.50% | ±1.90% +23.53% | | RowsBench | bench_sort_asc_on_1k | 2 | 3 | 73.550mb -0.01% | 40.263ms -1.81% | ±0.95% +9.91% | | RowsBench | bench_sort_by_on_1k | 2 | 3 | 73.550mb -0.01% | 39.734ms -3.50% | ±0.10% -87.44% | | RowsBench | bench_sort_desc_on_1k | 2 | 3 | 73.550mb -0.01% | 40.344ms -2.39% | ±0.67% +14.31% | | RowsBench | bench_sort_entries_on_1k | 2 | 3 | 75.913mb -0.01% | 7.422ms +0.66% | ±0.33% -60.16% | | RowsBench | bench_sort_on_1k | 2 | 3 | 73.472mb -0.01% | 28.434ms -3.53% | ±1.30% +136.22% | | RowsBench | bench_take_1k_on_10k | 10 | 3 | 73.472mb -0.01% | 13.606μs +0.94% | ±0.35% -62.48% | | RowsBench | bench_take_right_1k_on_10k | 10 | 3 | 73.472mb -0.01% | 15.879μs +1.10% | ±0.78% +161.25% | | RowsBench | bench_unique_on_1k | 2 | 3 | 96.480mb -0.01% | 183.051ms -2.16% | ±0.98% +44.87% | | NativeEntryFactoryBench | bench_entry_factory | 1 | 3 | 116.718mb -0.01% | 497.032ms -0.37% | ±1.15% +39.48% | | NativeEntryFactoryBench | bench_entry_factory | 1 | 3 | 60.196mb -0.02% | 243.126ms -1.18% | ±1.50% -9.44% | | NativeEntryFactoryBench | bench_entry_factory | 1 | 3 | 15.130mb -0.07% | 51.913ms +0.11% | ±0.96% -6.67% | | TypeDetectorBench | bench_type_detector | 1 | 3 | 59.964mb -0.02% | 430.939ms -0.40% | ±0.79% +118.21% | | TypeDetectorBench | bench_type_detector | 1 | 3 | 14.504mb -0.07% | 87.139ms +1.96% | ±1.92% +210.25% | +-------------------------+----------------------------+------+-----+------------------+------------------+------------------+ ```