flow-php / flow

Flow PHP - data processing framework
https://flow-php.com
MIT License
470 stars 26 forks source link

Added output to examples #1021

Closed norberttech closed 6 months ago

norberttech commented 6 months ago

Change Log

Added

  • output on landing page examples
  • writing to output.txt directly from examples

Fixed

  • when writing directly to file stream, loader is not closing stream after each chunk of rows

Changed

Removed

Deprecated

Security


Description

image
github-actions[bot] commented 6 months ago

Flow PHP - Benchmarks

Results of the benchmarks from this PR are compared with the results from 1.x branch.

Extractors ```shell +-----------------------+-------------------+------+-----+------------------+------------------+-----------------+ | benchmark | subject | revs | its | mem_peak | mode | rstdev | +-----------------------+-------------------+------+-----+------------------+------------------+-----------------+ | AvroExtractorBench | bench_extract_10k | 1 | 3 | 35.279mb +0.00% | 824.330ms +0.38% | ±0.74% +69.76% | | CSVExtractorBench | bench_extract_10k | 1 | 3 | 5.002mb +0.01% | 343.149ms +0.18% | ±0.22% -70.15% | | JsonExtractorBench | bench_extract_10k | 1 | 3 | 5.152mb +0.01% | 1.048s -0.50% | ±0.87% +331.90% | | ParquetExtractorBench | bench_extract_10k | 1 | 3 | 135.823mb +0.00% | 896.813ms +0.04% | ±0.47% +10.85% | | TextExtractorBench | bench_extract_10k | 1 | 3 | 4.910mb +0.01% | 35.404ms -0.11% | ±0.35% -63.83% | | XmlExtractorBench | bench_extract_10k | 1 | 3 | 4.916mb +0.01% | 427.368ms -1.37% | ±0.86% +32.80% | +-----------------------+-------------------+------+-----+------------------+------------------+-----------------+ ```
Transformers ```shell +-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+ | benchmark | subject | revs | its | mem_peak | mode | rstdev | +-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+ | RenameEntryTransformerBench | bench_transform_10k_rows | 1 | 3 | 110.616mb +0.00% | 64.385ms -0.96% | ±0.84% -30.60% | +-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+ ```
Loaders ```shell +--------------------+----------------+------+-----+------------------+------------------+------------------+ | benchmark | subject | revs | its | mem_peak | mode | rstdev | +--------------------+----------------+------+-----+------------------+------------------+------------------+ | AvroLoaderBench | bench_load_10k | 1 | 3 | 95.659mb +0.00% | 466.607ms +1.10% | ±1.03% +1777.63% | | CSVLoaderBench | bench_load_10k | 1 | 3 | 54.141mb +0.00% | 72.156ms +1.60% | ±0.64% +140.51% | | JsonLoaderBench | bench_load_10k | 1 | 3 | 106.567mb +0.00% | 52.736ms +1.86% | ±0.94% +157.59% | | ParquetLoaderBench | bench_load_10k | 1 | 3 | 224.386mb +0.00% | 1.416s +0.05% | ±0.74% +110.23% | | TextLoaderBench | bench_load_10k | 1 | 3 | 17.957mb +0.00% | 39.709ms -0.33% | ±0.72% +241.26% | +--------------------+----------------+------+-----+------------------+------------------+------------------+ ```
Building Blocks ```shell +-------------------------+----------------------------+------+-----+------------------+------------------+-------------------------------+ | benchmark | subject | revs | its | mem_peak | mode | rstdev | +-------------------------+----------------------------+------+-----+------------------+------------------+-------------------------------+ | RowsBench | bench_chunk_10_on_10k | 2 | 3 | 76.682mb +0.00% | 3.392ms +1.78% | ±1.78% +415.78% | | RowsBench | bench_diff_left_1k_on_10k | 2 | 3 | 96.409mb +0.00% | 179.432ms -0.90% | ±1.32% -19.36% | | RowsBench | bench_diff_right_1k_on_10k | 2 | 3 | 74.935mb +0.00% | 18.399ms +0.71% | ±0.85% -22.40% | | RowsBench | bench_drop_1k_on_10k | 2 | 3 | 77.922mb +0.00% | 1.719ms +1.30% | ±0.77% -67.98% | | RowsBench | bench_drop_right_1k_on_10k | 2 | 3 | 77.922mb +0.00% | 1.730ms -1.46% | ±1.09% -39.83% | | RowsBench | bench_entries_on_10k | 2 | 3 | 75.035mb +0.00% | 2.583ms -10.30% | ±1.19% -53.65% | | RowsBench | bench_filter_on_10k | 2 | 3 | 75.564mb +0.00% | 16.646ms +10.36% | ±2.68% +76.95% | | RowsBench | bench_find_on_10k | 2 | 3 | 75.564mb +0.00% | 16.452ms +11.87% | ±0.73% -28.36% | | RowsBench | bench_find_one_on_10k | 10 | 3 | 73.468mb +0.00% | 1.794μs -5.58% | ±2.67% +22832449392073000.00% | | RowsBench | bench_first_on_10k | 10 | 3 | 73.468mb +0.00% | 0.300μs -25.00% | ±0.00% -100.00% | | RowsBench | bench_flat_map_on_1k | 2 | 3 | 87.022mb +0.00% | 13.002ms -1.11% | ±0.38% -65.26% | | RowsBench | bench_map_on_10k | 2 | 3 | 116.383mb +0.00% | 65.517ms -1.05% | ±0.54% +12.97% | | RowsBench | bench_merge_1k_on_10k | 2 | 3 | 76.083mb +0.00% | 1.276ms -1.97% | ±1.34% -21.39% | | RowsBench | bench_partition_by_on_10k | 2 | 3 | 79.430mb +0.00% | 59.295ms +1.10% | ±1.55% +67.93% | | RowsBench | bench_remove_on_10k | 2 | 3 | 78.185mb +0.00% | 3.863ms -5.90% | ±0.82% -77.55% | | RowsBench | bench_sort_asc_on_1k | 2 | 3 | 73.546mb +0.00% | 39.818ms -1.26% | ±3.09% +129.92% | | RowsBench | bench_sort_by_on_1k | 2 | 3 | 73.546mb +0.00% | 39.848ms -0.42% | ±0.58% -42.45% | | RowsBench | bench_sort_desc_on_1k | 2 | 3 | 73.546mb +0.00% | 40.591ms -0.42% | ±0.28% -56.18% | | RowsBench | bench_sort_entries_on_1k | 2 | 3 | 75.909mb +0.00% | 7.295ms -1.96% | ±0.75% +176.98% | | RowsBench | bench_sort_on_1k | 2 | 3 | 73.468mb +0.00% | 28.739ms -1.05% | ±0.47% -28.04% | | RowsBench | bench_take_1k_on_10k | 10 | 3 | 73.468mb +0.00% | 13.206μs -4.22% | ±0.36% -81.09% | | RowsBench | bench_take_right_1k_on_10k | 10 | 3 | 73.468mb +0.00% | 15.906μs -1.69% | ±0.30% -61.49% | | RowsBench | bench_unique_on_1k | 2 | 3 | 96.476mb +0.00% | 188.472ms +1.86% | ±0.75% +116.76% | | NativeEntryFactoryBench | bench_entry_factory | 1 | 3 | 116.714mb +0.00% | 501.823ms -0.68% | ±0.55% -59.34% | | NativeEntryFactoryBench | bench_entry_factory | 1 | 3 | 60.192mb +0.00% | 252.834ms +1.67% | ±1.32% +105.64% | | NativeEntryFactoryBench | bench_entry_factory | 1 | 3 | 15.127mb +0.00% | 53.012ms -0.38% | ±0.86% +393.90% | | TypeDetectorBench | bench_type_detector | 1 | 3 | 59.958mb +0.00% | 431.720ms +0.11% | ±0.55% -41.14% | | TypeDetectorBench | bench_type_detector | 1 | 3 | 14.497mb +0.00% | 85.805ms -1.77% | ±0.53% -39.56% | +-------------------------+----------------------------+------+-----+------------------+------------------+-------------------------------+ ```