apache / datafusion-ballista

Apache DataFusion Ballista Distributed Query Engine

https://datafusion.apache.org/ballista

Apache License 2.0

1.46k stars 185 forks source link

[fea] Avoid multithreaded write lock conflicts in event queue. #754

Closed Ted-Jiang closed 1 year ago

Ted-Jiang commented 1 year ago

Which issue does this PR close?

Closes #753.

Rationale for this change

After fix concurrency visit lock before four long running event reduce to 200~500 micro seconds cost.

2023-04-17T05:36:45.695838Z  WARN tokio-runtime-worker ThreadId(16) ballista_scheduler::scheduler_server::query_stage_scheduler: [METRICS] JobUpdated : job_id=jr3EVUI. events cost 553 us!

What changes are included in this PR?

Are there any user-facing changes?

Ted-Jiang commented 1 year ago

I will merge this one, If anyone have problem i will be happy to fix it.