search

haskell / haskell-language-server

Official haskell ide support via language server (LSP). Successor of ghcide & haskell-ide-engine.
Apache License 2.0
2.71k stars 368 forks source link

Run ghcide benchmark suite on HLS #1201

Open pepeiborra opened 3 years ago

pepeiborra commented 3 years ago

Ghcide has an excellent benchmark suite that tracks time and space usage across multiple revisions, experiments and examples. But by running it only on exe:ghcide, we are missing crucial data on the impact of HLS plugins on performance.

What are the steps required to run the ghcide benchmark suite on HLS plugins? And on HLS?

There' more than one option:

:heavy_check_mark: 1. Port the benchmark suite to HLS

We just move the Cabal stanzas (ghcide-bench and bench-hist) and the code across, add new experiments to exercise the relevant plugins, and run it on the whole of HLS + all the plugins.

2. Extract the benchmark suite to a package lsp-bench

As things stand right now, its' not even possible to package integration tests for plugins. All the integration tests live in HLS (func-test). We probably want to avoid ending up in the same situation with benchmarks.

A standalone package would be a way to allow each plugin to define its own benchmark suite.

rayshih commented 3 years ago

I'm interested

rayshih commented 3 years ago

Just started to read the code, it seems like we are using our own the benchmark utils. Should we consider to migrate to something like Criterion?

pepeiborra commented 3 years ago

That seems unrelated to this issue. If you think migrating to Criterion would bring significant benefits, please feel free to open a new issue to discuss

jneira commented 3 years ago

It seems 1 is already done (actual hls bench came from ghcide one)

pepeiborra commented 2 years ago

I am working on this

wz1000 commented 2 years ago

I've got a patch for this which I just implemented today. Sorry for not updating the ticket!.

michaelpj commented 10 months ago

@wz1000 did you land the patch?