search

dart-lang / sdk

The Dart SDK, including the VM, JS and Wasm compilers, analysis, core libraries, and more.
https://dart.dev
BSD 3-Clause "New" or "Revised" License
10.3k stars 1.59k forks source link

Add LSP analysis server benchmarks #44853

Open DanTup opened 3 years ago

DanTup commented 3 years ago

@devoncarew I've added some new analysis server benchmarks that may need tracking.

They were added in 74d8cc96365d844d5b33fbaa34ac0426720ebe0a and have the same IDs as the previous analysis server benchmarks, but are prefixed with "lsp-":

analysis-server-cold       : Analysis Server benchmarks of a large project on start-up, no existing driver cache.
lsp-analysis-server-cold   : LSP Analysis Server benchmarks of a large project on start-up, no existing driver cache.
analysis-server            : Analysis Server benchmarks of a large project, with an existing driver cache.
lsp-analysis-server        : LSP Analysis Server benchmarks of a large project, with an existing driver cache.
analysis-flutter-analyze   : Clone the flutter/flutter repo and run 'flutter analyze --flutter-repo' with the current Dart VM and analysis server.

Also related: in 12c0ca85dfc6ede77ebea342583dffd09107333c the existing analysis server benchmarks "edit" measurement was updated to edit a priority file (which is a little slower than analyzing a non-priority file) which better matches the real-world case. There will be an obvious increase in those timings as a result, but it's just a difference in what's being measured rather than a performance change (this also makes the timings comparable with the LSP one of the same name).

srawlins commented 1 year ago

We may still need to track these in golem...

epompeii commented 1 year ago

I'm curious how you all store and analyze your benchmarks. What is golem? My reason for asking is I've been work on a continuous benchmarking tool called Bencher: https://github.com/bencherdev/bencher And I'm trying to get a better idea of how others are already solving this problem.