django-ftl / fluent-compiler

High performance Python implementation of Fluent, Mozilla's l10n language
Other
21 stars 4 forks source link

Add a benchmark for compiling large fluent files #30

Closed leamingrad closed 1 month ago

leamingrad commented 1 month ago

This PR adds a new benchmark to the project for compiling a fluent file with 10K items it.

While the benchmark itself relies on a static file, I've included the script that I used to generate it.

Sample results ``` ❯ ./compiler.py -k 10k --benchmark-warmup=off ================================================================================================= test session starts ================================================================================================== platform darwin -- Python 3.10.14, pytest-8.2.1, pluggy-1.5.0 benchmark: 4.0.0 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000) rootdir: /Volumes/Code/fluent-compiler configfile: pyproject.toml plugins: anyio-4.3.0, hypothesis-6.102.6, benchmark-4.0.0 collected 4 items / 3 deselected / 1 selected compiler.py . [100%] ----------------------------------------------- benchmark: 1 tests ----------------------------------------------- Name (time in s) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations ------------------------------------------------------------------------------------------------------------------ test_file_with_10k_items 9.7863 10.1248 9.9234 0.1377 9.8794 0.2118 1;0 0.1008 5 1 ------------------------------------------------------------------------------------------------------------------ Legend: Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile. OPS: Operations Per Second, computed as 1 / Mean ====================================================================================== 1 passed, 3 deselected in 69.96s (0:01:09) ====================================================================================== ```
spookylukey commented 1 month ago

The test fails due to an issue on master I have resolved, this is good to go. Thanks!