search

Eventual-Inc / Daft

Distributed data engine for Python/SQL designed for the cloud, powered by Rust
https://getdaft.io
Apache License 2.0
2.34k stars 164 forks source link

[BUG] Check env in benchmarking script #3297

Closed colin-ho closed 1 week ago

colin-ho commented 1 week ago

Using ctx.get_or_create_runner in benchmarking warmup code / metrics builder causes subsequent ray.inits to crash. Just check the DAFT_RUNNER environment var instead, which should be set.

Tested:

codspeed-hq[bot] commented 1 week ago

CodSpeed Performance Report

Merging #3297 will degrade performances by 47.65%

Comparing colin/fix-benchmarking-script (1d52ee2) with main (711e862)

Summary

❌ 2 regressions ✅ 15 untouched benchmarks

:warning: Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

Benchmark main colin/fix-benchmarking-script Change
❌ test_iter_rows_first_row[100 Small Files] 269 ms 373.6 ms -28%
❌ test_show[100 Small Files] 22.3 ms 42.5 ms -47.65%
codecov[bot] commented 1 week ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 77.50%. Comparing base (711e862) to head (1d52ee2). Report is 5 commits behind head on main.

Additional details and impacted files [![Impacted file tree graph](https://app.codecov.io/gh/Eventual-Inc/Daft/pull/3297/graphs/tree.svg?width=650&height=150&src=pr&token=J430QVFE89&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Eventual-Inc)](https://app.codecov.io/gh/Eventual-Inc/Daft/pull/3297?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Eventual-Inc) ```diff @@ Coverage Diff @@ ## main #3297 +/- ## ========================================== - Coverage 77.50% 77.50% -0.01% ========================================== Files 666 666 Lines 81335 81333 -2 ========================================== - Hits 63041 63036 -5 - Misses 18294 18297 +3 ``` [see 3 files with indirect coverage changes](https://app.codecov.io/gh/Eventual-Inc/Daft/pull/3297/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Eventual-Inc)