search

alexwl / haskell-code-explorer

Web application for exploring and understanding Haskell codebases
MIT License
512 stars 20 forks source link

segfault related to TemplateHaskell #36

Open gelisam opened 5 years ago

gelisam commented 5 years ago

Building the following code (e.g. with stack build) and then running haskell-code-explorer results in a segfault (yes, a segfault in Haskell, yikes!):

module Lib where

import Prelude

import qualified Data.Yaml as Yaml
import Language.Haskell.TH (Exp, Q, runIO)
import Language.Haskell.TH.Syntax (addDependentFile)

migrations :: ()
migrations = $(do
  _ <- runIO (Yaml.decodeFileEither "file.txt" :: IO (Either Yaml.ParseException Yaml.Value))
  [|()|])

The above snippet is of course silly, but I encountered the bug in a real project and the above is the result of deleting as much as possible from that project in order to create a minimal repro case. I have put the code in a git repository for your convenience: https://github.com/gelisam/haskell-code-explorer-bug

Note that since this is TemplateHaskell code, the Yaml.decodeFileEither "file.txt" is executed at compile time. Building the project succeeds, so it doesn't seem to be a bug in the yaml package. However, replacing Yaml.decodeFileEither "file.txt" with the simpler readFile "file.txt" does not trigger the bug, so there must be some weird interaction between the yaml library and haskell-code-explorer. Since the symptom is a segfault, I suspect it has to do with the libyaml C library it uses under the hood.

alexwl commented 5 years ago

Interesting,

There was a bug in yaml library that caused a segfault (https://github.com/snoyberg/yaml/issues/5), but that bug has been fixed since version 0.4.1.2.

I successfully indexed the package from https://github.com/gelisam/haskell-code-explorer-bug using haskell-code-indexer built with ghc-8.6.3.

I also tried: -haskell-code-indexer built with ghc-8.6.4 (changed resolver in stack.yaml to lts-13.12): no errors -haskell-code-indexer built with ghc-8.4.4 (changed resolver in stack.yaml to lts-12.16): no errors -haskell-code-indexer built with ghc-8.2.2 (changed resolver in stack.yaml to lts-11.3): got "unhandled ELF relocation(RelA) type 42" (it's a known GHC bug that has already been fixed).

It doesn't matter whether file.txt exists or not, still no segfault.

At the moment it is not clear how to reproduce the bug.

Can you post the full output of haskell-code-indexer -v?

gelisam commented 5 years ago

Oh! I forgot to mention something important: we tried it on two other macOS machines, and the segfault only occurs on Mojave.

alexwl commented 5 years ago

It may be related to https://gitlab.haskell.org/ghc/ghc/issues/16117 (Random segfaults at runtime on macOS Mojave caused by GHC 8.6.3 linker misaligning sections)

gelisam commented 5 years ago

Huh, that looks pretty nasty. Since you cannot reproduce it, I guess I'll just have to wait until that bug is fixed and then check if the segfault still occurs. Thanks!

gelisam commented 5 years ago

Still, here is the full output of haskell-code-indexer -v:

$ stack install
haskell-code-explorer-bug-0.0.0: configure (lib)
Configuring haskell-code-explorer-bug-0.0.0...
haskell-code-explorer-bug-0.0.0: build (lib)
Preprocessing library for haskell-code-explorer-bug-0.0.0..
Building library for haskell-code-explorer-bug-0.0.0..
[1 of 2] Compiling Lib              ( src/Lib.hs, .stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/Lib.o )
[2 of 2] Compiling Paths_haskell_code_explorer_bug ( .stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/autogen/Paths_haskell_code_explorer_bug.hs, .stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/Paths_haskell_code_explorer_bug.o )
haskell-code-explorer-bug-0.0.0: copy/register
Installing library in /Users/gelisam/working/haskell/haskell-code-explorer-bug/.stack-work/install/x86_64-osx/lts-13.5/8.6.3/lib/x86_64-osx-ghc-8.6.3/haskell-code-explorer-bug-0.0.0-LClv1IjW8llBci7bWj9Oxx
Registering library for haskell-code-explorer-bug-0.0.0..
$ haskell-code-indexer -v
2019-04-30 10:26:51.834241 EDT : [info] haskell-code-indexer version 0.1.0.0, GHC version 8.6.3.0
2019-04-30 10:26:51.83692 EDT : [debug] IndexerConfig {configPackageDirectoryPath = ".", configPackageDistDirRelativePath = Nothing, configOutputDirectoryName = Nothing, configLog = StdOut, configMinLogLevel = LevelDebug, configSourceCodePreprocessing = AfterPreprocessing, configCompression = Gzip, configGhcOptions = [], configIgnoreDirectories = []}
2019-04-30 10:26:51.839883 EDT : [info] Found stack.yaml. Executing "stack path --dist-dir" to get dist directory.
2019-04-30 10:26:52.050873 EDT : [info] Stack dist directory : .stack-work/dist/x86_64-osx/Cabal-2.4.0.1
2019-04-30 10:26:53.328096 EDT : [info] Indexing haskell-code-explorer-bug-0.0.0
2019-04-30 10:26:53.335399 EDT : [debug] Component id : setup
2019-04-30 10:26:53.335457 EDT : [debug] Modules : []
2019-04-30 10:26:53.33548 EDT : [debug] GHC command line options : 
2019-04-30 10:26:53.684626 EDT : [debug] Component id : lib
2019-04-30 10:26:53.684683 EDT : [debug] Modules : ["Lib","Paths_haskell_code_explorer_bug"]
2019-04-30 10:26:53.684704 EDT : [debug] GHC command line options : -fbuilding-cabal-package -O -outputdir .stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build -odir .stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build -hidir .stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build -stubdir .stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build -i -i.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build -isrc -i.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/autogen -i.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/global-autogen -I.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/autogen -I.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/global-autogen -I.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build -optP-include -optP.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/autogen/cabal_macros.h -this-unit-id haskell-code-explorer-bug-0.0.0-LClv1IjW8llBci7bWj9Oxx -hide-all-packages -Wmissing-home-modules -no-user-package-db -package-db /Users/gelisam/.stack/snapshots/x86_64-osx/lts-13.5/8.6.3/pkgdb -package-db /Users/gelisam/working/haskell/haskell-code-explorer-bug/.stack-work/install/x86_64-osx/lts-13.5/8.6.3/pkgdb -package-id base-4.12.0.0 -package-id template-haskell-2.14.0.0 -package-id yaml-0.11.0.0-GAWf3vpOhGMK7o9d6zMyJu -XHaskell2010 -XTemplateHaskell Lib Paths_haskell_code_explorer_bug
Segmentation fault: 11
alexwl commented 5 years ago

Thanks!

I tried indexing on macOS Mojave (10.14) and also got "Segmentation fault: 11" with both GHG 8.6.3 and GHC 8.6.4.

Now I can investigate this issue.

There is a bug in persistent library that might have the same cause: https://github.com/yesodweb/persistent/issues/830 (Segmentation fault: 11 on macOS).

I'll try to reproduce the segfault without the involvement of haskell-code-indexer.

alexwl commented 5 years ago

I've managed to reproduce the segfault on macOS Mojave using only GHC API.

This code segfaults with "Segmentation fault: 11" message:

import Control.Monad.IO.Class
import qualified Data.List as L
import GHC
import Packages
import DynFlags
import GHC.Paths (libdir)
import Digraph (flattenSCCs)

ghcFlags = "-fbuilding-cabal-package -O -outputdir .stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build -odir .stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build -hidir .stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build -stubdir .stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build -i -i.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build -isrc -i.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/autogen -i.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/global-autogen -I.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/autogen -I.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/global-autogen -I.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build -optP-include -optP.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/autogen/cabal_macros.h -this-unit-id haskell-code-explorer-bug-0.0.0-34p2k9iR4eu61IAyrOu1BO -hide-all-packages -Wmissing-home-modules -no-user-package-db -package-db /Users/alexwl/.stack/snapshots/x86_64-osx/lts-13.5/8.6.3/pkgdb -package-db /Users/alexwl/Documents/haskell-code-explorer-bug/.stack-work/install/x86_64-osx/lts-13.5/8.6.3/pkgdb -package-id base-4.12.0.0 -package-id yaml-0.11.0.0-GAWf3vpOhGMK7o9d6zMyJu -package-id template-haskell-2.14.0.0 -XHaskell2010 -XTemplateHaskell"

main = do
  runGhcT (Just libdir) $ do
    flags <- getSessionDynFlags
    (flags', _, _) <-
      parseDynamicFlagsCmdLine flags (map noLoc $ words ghcFlags)
    (flags'', _) <- liftIO $ initPackages flags'
    setSessionDynFlags $
      L.foldl'
        gopt_set
        (flags''
           {hscTarget = HscAsm, ghcLink = LinkInMemory, ghcMode = CompManager})
        [Opt_Haddock]
    let modules = ["Lib"] -- file from  https://github.com/gelisam/haskell-code-explorer-bug
    targets <- mapM (`guessTarget` Nothing) modules
    setTargets targets
    _ <- load LoadAllTargets
    modGraph <- getModuleGraph
    let topSortMods = flattenSCCs (topSortModuleGraph False modGraph Nothing)
        modSum = head topSortMods
    parsedModule <- parseModule modSum
    typecheckedModule <- typecheckModule parsedModule
    liftIO $ print "ok"
gelisam commented 5 years ago

The LinkInMemory bit reminds me of the https://gitlab.haskell.org/ghc/ghc/issues/16117 bug you linked to earlier, is there an alternative value ghcLink can be given?

alexwl commented 5 years ago

is there an alternative value ghcLink can be given?

There are 5 possible values (https://hackage.haskell.org/package/ghc-8.6.4/docs/DynFlags.html#t:GhcLink): NoLink Don't link at all LinkBinary Link object code into a binary LinkInMemory Use the in-memory dynamic linker (works for both bytecode and object code). LinkDynLib Link objects into a dynamic lib (DLL on Windows, DSO on ELF platforms) LinkStaticLib Link objects into a static lib

I tried them all but still got the segfault.