SQLite3 returned ErrorConstraint while attempting to perform step: UNIQUE constraint failed: defs.hieFile, defs.occ

[Kleidukos]

Your environment

• Which OS do you use? Fedora 37

• Which version of GHC do you use and how did you install it? GHC 9.4.4 via ghcup

• How is your project built (alternative: link to the project)? cabal

• Which LSP client (editor/plugin) do you use? Neovim + nvim_lsp

• Which version of HLS do you use and how did you install it? 1.9.1.0 via ghcup

Expected behaviour

I get this message:

LSP[hls] Error condition, please check your setup and/or the [issue tracker](https://github.com/haskell/haskell-language-server/issues): 
HieDb writer thread SQLite error:
  SQLite3 returned ErrorConstraint while attempting to perform step: UNIQUE constraint failed: defs.hieFile, defs.occ

[wz1000]

Could you share the your hiedb database (this includes information on the location and name of all symbols in your code)?

Running the following command in the root directory of the project should give you the location of the database:

$ haskell-language-server hiedb ls
2023-03-30T10:29:54.241012Z | Info | No log file specified; using stderr.
2023-03-30T10:29:54.241280Z | Info | haskell-language-server version: 1.10.0.0 (GHC: 9.6.1) (PATH: /home/zubin/haskell-language-server/dist-newstyle/build/x86_64-linux/ghc-9.6.1/haskell-language-server-1.10.0.0/x/haskell-language-server/build/haskell-language-server/haskell-language-server) (GIT hash: 477a92385237e437c60770381573ebc9d170e357)
2023-03-30T10:29:54.241734Z | Info | Directory: /home/zubin/haskell-language-server
2023-03-30T10:29:54.241851Z | Info | Logging heap statistics every 60.00s
 Using hiedb at: /home/zubin/.cache/ghcide/2bdb1d6badd4919c833ed2fe59ddc413f923da5a-haskell-language-server-9.6.1-1.hiedb

[wz1000]

If you could minimise it to a particular file that this is triggered on that would also be very helpful.

[Kleidukos]

Unfortunately this is a proprietary codebase. I will try to reproduce it on an open-source project as well later this week.

[daniel-chambers]

I can reproduce this consistently on the Hasura OSS codebase. It's not a small repro sorry, but it is at least a repro.

Environment:

• VSCode
• HLS 2.0.0.0
• GHC 9.4.5 (Necessary GHC version for Hasura at this time)
• Cabal: 3.8.1.0
• Ubuntu 20.04
• Native dependencies for Hasura build (hopefully a complete list):
  • freetds-dev
  • g++
  • gnupg
  • libgnutls28-dev
  • libkrb5-dev
  • libpcre3-dev
  • libpq-dev
  • libssl-dev
  • lz4
  • netcat
  • postgresql-client
  • unixodbc-dev
  • upx
  • xz-utils
  • zlib1g-dev
  • msodbcsql18

Steps:

• Clone https://github.com/hasura/graphql-engine.git
• Checkout a5e51ac39afff084102713411bd34f44d09b0b3a (this is the commit I just used to repro this issue)
• cp cabal/dev.project cabal.project.local
• cabal build exe:graphql-engine should work (not needed for the repro, but more to check that nothing else is missing in your env, eg native deps)
• Open the repo in VSCode
• Open a Haskell file in server/src-lib, for example (at random): server/src-lib/Hasura/Server/Auth/JWT.hs
• Wait while HLS processes
• Receive the error:

  Error condition, please check your setup and/or the [issue tracker](https://github.com/haskell/haskell-language-server/issues): HieDb writer thread SQLite error: SQLite3 returned ErrorConstraint while attempting to perform step: UNIQUE constraint failed: defs.hieFile, defs.occ

[tonyday567]

I consistently get this error with ghc-9.8.1 but not with ghc-9.6.3, with the following environment and steps:

• emacs
• hls-2.4.0.0 (installed from ghcup)
• ghc-9.8.1
• OSX 10.15.7
• cabal 3.10.2.0

Steps:

• clone https://github.com/tonyday567/chart-svg.git
• checkout 940b47a458ad898c24930723c954f24f7d84df7d (hls-not-ok branch)
• ghcup set ghc 9.8.1
• open repo in emacs
• open src/Chart.hs
• start the lsp process
• receive:

[eglot] Server reports (type=1): Error condition, please check your setup and/or the [issue tracker](https://github.com/haskell/haskell-language-server/issues): 
HieDb writer thread SQLite error:
  SQLite3 returned ErrorConstraint while attempting to perform step: UNIQUE constraint failed: defs.hieFile, defs.occ

• Note that hls continues to work after this.
• ghcup set ghc 9.6.3
• repeat the above steps with no error.

Just speculating, but the big change for me between ghc 9.6.3 and 9.8.1 was that the codebase in the reproduction steps below stopped compiling and DuplicateRecordFiles is needed.

haskell-language-server-wrapper hiedb ls gives:

No 'hie.yaml' found. Try to discover the project type!
Run entered for haskell-language-server-wrapper(haskell-language-server-wrapper) Version 2.4.0.0 x86_64 ghc-9.0.2
Current directory: /Users/tonyday/haskell/chart-svg
Operating system: darwin
Arguments: ["hiedb","ls"]
Cradle directory: /Users/tonyday/haskell/chart-svg
Cradle type: Cabal

Tool versions found on the $PATH
cabal:          3.10.2.0
stack:          2.13.1
ghc:            9.8.1

Consulting the cradle to get project GHC version...
2023-12-04T21:44:16.113545Z | Debug | executing command: cabal exec -v0 -- ghc --print-libdir
2023-12-04T21:44:16.749892Z | Debug | executing command: cabal exec -v0 -- ghc -package-env=- -ignore-dot-ghci -e Control.Monad.join (Control.Monad.fmap System.IO.putStr System.Environment.getExecutablePath)
2023-12-04T21:44:17.380903Z | Debug | executing command: cabal --builddir=/Users/tonyday/.cache/hie-bios/dist-chart-svg-63a94dea6f6ba3eb9ad6853e524110c5 v2-exec --with-compiler /Users/tonyday/.cache/hie-bios/wrapper-b54f81dea4c0e6d1626911c526bc4e36 --with-hc-pkg /Users/tonyday/.cache/hie-bios/ghc-pkg-2f10a194b7bf58b55ed393a80dec3695 ghc -v0 -- --numeric-version
Project GHC version: 9.8.1
haskell-language-server exe candidates: ["haskell-language-server-9.8.1","haskell-language-server"]
Launching haskell-language-server exe at:/Users/tonyday/.ghcup/bin/haskell-language-server-9.8.1
2023-12-04T21:44:18.091624Z | Debug | executing command: cabal exec -v0 -- ghc --print-libdir
2023-12-04T21:44:18.371349Z | Debug | executing command: cabal exec -v0 -- ghc -package-env=- -ignore-dot-ghci -e Control.Monad.join (Control.Monad.fmap System.IO.putStr System.Environment.getExecutablePath)
2023-12-04T21:44:19.018879Z | Debug | executing command: cabal --builddir=/Users/tonyday/.cache/hie-bios/dist-chart-svg-63a94dea6f6ba3eb9ad6853e524110c5 v2-exec --with-compiler /Users/tonyday/.cache/hie-bios/wrapper-b54f81dea4c0e6d1626911c526bc4e36 --with-hc-pkg /Users/tonyday/.cache/hie-bios/ghc-pkg-2f10a194b7bf58b55ed393a80dec3695 ghc -v0 -- -v0 -package-env=- -ignore-dot-ghci -e Control.Monad.join (Control.Monad.fmap System.IO.putStr System.Environment.getExecutablePath)
2023-12-04T21:44:19.674694Z | Debug | executing command: cabal exec -v0 -- ghc --print-libdir
2023-12-04T21:44:19.953042Z | Debug | executing command: cabal exec -v0 -- ghc -package-env=- -ignore-dot-ghci -e Control.Monad.join (Control.Monad.fmap System.IO.putStr System.Environment.getExecutablePath)
2023-12-04T21:44:20.611680Z | Debug | executing command: cabal --builddir=/Users/tonyday/.cache/hie-bios/dist-chart-svg-63a94dea6f6ba3eb9ad6853e524110c5 v2-exec --with-compiler /Users/tonyday/.cache/hie-bios/wrapper-b54f81dea4c0e6d1626911c526bc4e36 --with-hc-pkg /Users/tonyday/.cache/hie-bios/ghc-pkg-2f10a194b7bf58b55ed393a80dec3695 ghc -v0 -- --print-libdir
2023-12-04T21:44:23.061847Z | Info | haskell-language-server version: 2.4.0.0 (GHC: 9.8.1) (PATH: /Users/tonyday/.ghcup/hls/2.4.0.0/lib/haskell-language-server-2.4.0.0/bin/haskell-language-server-9.8.1)
2023-12-04T21:44:23.063169Z | Info | Directory: /Users/tonyday/haskell/chart-svg
2023-12-04T21:44:23.063566Z | Info | Logging heap statistics every 60.00s
 Using hiedb at: /Users/tonyday/.cache/ghcide/cb5eba2c944c2f574cd89b2fccbb672504994c45-chart-svg-9.8.1-1.hiedb
/Users/tonyday/.cache/ghcide/chart-svg-0.6-inplace-375a63c439ad3ea1c5534acf8f15b5b7ad93fa61/Chart/Data.hie  Chart.Data  chart-svg-0.6-inplace
/Users/tonyday/.cache/ghcide/chart-svg-0.6-inplace-375a63c439ad3ea1c5534acf8f15b5b7ad93fa61/Data/Path.hie   Data.Path   chart-svg-0.6-inplace
/Users/tonyday/.cache/ghcide/chart-svg-0.6-inplace-375a63c439ad3ea1c5534acf8f15b5b7ad93fa61/Data/Path/Parser.hie    Data.Path.Parser    chart-svg-0.6-inplace
/Users/tonyday/.cache/ghcide/chart-svg-0.6-inplace-375a63c439ad3ea1c5534acf8f15b5b7ad93fa61/Data/Colour.hie Data.Colour chart-svg-0.6-inplace
/Users/tonyday/.cache/ghcide/chart-svg-0.6-inplace-375a63c439ad3ea1c5534acf8f15b5b7ad93fa61/Chart/Style.hie Chart.Style chart-svg-0.6-inplace
/Users/tonyday/.cache/ghcide/chart-svg-0.6-inplace-375a63c439ad3ea1c5534acf8f15b5b7ad93fa61/Chart/Primitive.hie Chart.Primitive chart-svg-0.6-inplace
/Users/tonyday/.cache/ghcide/chart-svg-0.6-inplace-375a63c439ad3ea1c5534acf8f15b5b7ad93fa61/Chart/Surface.hie   Chart.Surface   chart-svg-0.6-inplace
/Users/tonyday/.cache/ghcide/chart-svg-0.6-inplace-375a63c439ad3ea1c5534acf8f15b5b7ad93fa61/Chart/Markup.hie    Chart.Markup    chart-svg-0.6-inplace
/Users/tonyday/.cache/ghcide/chart-svg-0.6-inplace-375a63c439ad3ea1c5534acf8f15b5b7ad93fa61/Chart/Compound.hie  Chart.Compound  chart-svg-0.6-inplace
/Users/tonyday/.cache/ghcide/chart-svg-0.6-inplace-375a63c439ad3ea1c5534acf8f15b5b7ad93fa61/Chart/Bar.hie   Chart.Bar   chart-svg-0.6-inplace
/Users/tonyday/.cache/ghcide/chart-svg-0.6-inplace-375a63c439ad3ea1c5534acf8f15b5b7ad93fa61/Chart/Examples.hie  Chart.Examples  chart-svg-0.6-inplace
/Users/tonyday/.cache/ghcide/chart-svg-0.6-inplace-375a63c439ad3ea1c5534acf8f15b5b7ad93fa61/Chart.hie   Chart   chart-svg-0.6-inplace

[wz1000]

Thanks for the reproducer @tonyday567, I have found the bug in hiedb, it's due to the NameSpace type in GHC being expanded for duplicate record fields. Fix requires a bit of re-working how of things work in hiedb but will be coming shortly.