Memory leak when interacting with embedded files

[jssblck]

Your environment

Which OS do you use?

• macOS Ventura using M1 Pro

Which version of GHC do you use and how did you install it?

• 9.4.7 via ghcup

How is your project built (alternative: link to the project)?

• https://github.com/fossas/fossa-cli
• Built via cabal build or cabal run fossa.
• Be aware it has dependencies on some other binaries, which I think is what is causing this issue. More details in reproduction steps.

Which LSP client (editor/plugin) do you use?

• VS Code with https://marketplace.visualstudio.com/items?itemName=haskell.haskell

Which version of HLS do you use and how did you install it?

• 2.2.0.0, via ghcup
• 2.3.0.0, via ghcup (problem is present in both, although seems exacerbated in 2.3.0.0)

Have you configured HLS in any way (especially: a hie.yaml file)?

• Yes, in the repo linked. Reproduced here:

  cradle:
  cabal:
  - path: "."
    component: "lib:spectrometer"
  - path: "./src"
    component: "lib:spectrometer"
  - path: "./test"
    component: "test:unit-tests"
  - path: "./integration-test"
    component: "test:integration-tests"
  - path: "./bench"
    component: "bench"

Steps to reproduce

First, ensure the binary dependencies are present:

# Generate random standins for other binaries of the same size
mkdir vendor-bins
dd if=/dev/urandom of=vendor-bins/index.gob.xz bs=1M count=25
dd if=/dev/urandom of=vendor-bins/lernie bs=1M count=4
dd if=/dev/urandom of=vendor-bins/themis-cli bs=1M count=19

# Either build the actual Rust binaries...
cargo build --release

# ... or generate random standins for them too
mkdir -p target/release
dd if=/dev/urandom of=target/release/millhone bs=1M count=9
dd if=/dev/urandom of=target/release/berkeleydb bs=1M count=1

Next, build: cabal build.

Next, open the project in VS Code and navigate to src/App/Fossa/Analyze.hs. Wait for indexing to finish (denoted in the bottom of the VS Code window).

Finally, open src/App/Fossa/EmbeddedBinary.hs. It'll flash that it's indexing and then seemingly finish.

Expected behaviour

HLS is actually finished analyzing the file, and its memory usage is reasonable.

Actual behaviour

HLS memory usage grows until macOS warns me that I must kill applications. If I don't respond to this prompt quickly, the entire system locks up and must be hard powered off.

Debug information

Everything works properly inside HLS other than this growing memory usage. I'm not sure what to do to debug this further.

Potentially related, we also had to apply some workarounds to get the embedded binaries to build under 9.4.7 at all: https://github.com/snoyberg/file-embed/issues/45.

[jssblck]

Note: file embedding is definitely the issue.

We have a workaround we do in our codebase here that completely avoids the problem.