Closed uhlajs closed 3 months ago
yannham
commented
3 months ago This is a tough one, because while nickel export work, nickel eval does loop as well. That being said nickel eval doesn't crash, it just loops forever, so I wonder if the background thread is doing something different, or if it's another part of NLS that should be protected against such cyclic definitions.
jneem
commented
3 months ago Yes, the background thread is doing it a little differently: its eval function (eval_permissive) recurses in places where the original one doesn't. The same thing happens in the example in #1815: nickel eval loops and eats all my memory, while nls crashes with a stack overflow.
Having said that, for me it looks like nls's protection against problems in eval is kicking in as expected: the background eval process crashes (and so we lose eval diagnostics, which is sad), but the main nls process keeps running.
yannham
commented
3 months ago @uhlajs can you confirm that even after the crash, normal NLS functions are still working as expected (like goto definition or show information on hover)? If this is just the background evaluation process crashing, this is not ideal, but less critical.
uhlajs
commented
3 months ago Negative (goto definition does not work), I am getting following message and there is no nls process running.
Client 1 quit with exit code 0 and signal 6I also tried to comment out the problematic line, then open editor and uncomment it. Here is result:
$ pgrep nls
2863627
2863634
$ ps 2863627 2863634
PID TTY STAT TIME COMMAND
2863627 ? Ssl 0:00 nls
2863634 ? S 0:00 /nix/store/xjynz8qzh3abmsxa0yi0g8gh6am250dz-nickel-1.5.0-nls/bin/nls --main-server /tmp/nix-shell.
# Now I have uncommented and saved file.
$ pgrep nls
# Empty resultInteresting. Here is what happens for me:
https://github.com/tweag/nickel/assets/1448113/2aa1b149-97ea-48ce-9ba0-d0a60fac9418
On uncommenting that line, you can see a zombie process appear in htop (that's the crashed background process), but typechecking still works in the editor afterwards.
Is there any way you can get a backtrace of the crashing nls? If nvim propagates its environment to nls, it should be enough to set RUST_BACKTRACE=1. (If nvim doesn't propagate its environment, it might require a wrapper script...)
Regarding trying to detect the recursion, the problem I'm having is that the environment is different each time we recurse so it isn't an obvious infinite loop to the interpreter. Probably for nls it's ok to bound the recursion depth?
uhlajs
commented
3 months ago @jneem Here I am running similar commands:
https://github.com/tweag/nickel/assets/13243167/bef27c8f-8d59-4aea-8fcd-f07cc1958958
As you can see the nls is completely terminated after saving changes. Here are generated coredumps if that helps.
I also tried to run latest code with #1878 changes, but the result is still the same.
Unfortunately, I wasn't able to get backtrace with RUST_BACKTRACE=1. I also tried to create a wrapper script (see below), but there is still not backtrace in the logs.
Content of /home/honza/tmp/nls-test/nls:
#!/bin/bash
export RUST_BACKTRACE=1
/home/honza/git/github/nickel/target/debug/nls "$@"Full log:
[START][2024-04-03 11:16:37] LSP logging initiated
[INFO][2024-04-03 11:16:37] .../vim/lsp/rpc.lua:662 "Starting RPC client" { args = {}, cmd = "nls", extra = { cwd = "/var/home/honza/tmp/nls-test" }}
[DEBUG][2024-04-03 11:16:37] .../vim/lsp/rpc.lua:284 "rpc.send" { id = 1, jsonrpc = "2.0", method = "initialize", params = { capabilities = { textDocument = { callHierarchy = { dynamicRegistration = false }, codeAction = { codeActionLiteralSupport = { codeActionKind = { valueSet = { "", "quickfix", "refactor", "refactor.extract", "refactor.inline", "refactor.rewrite", "source", "source.organizeImports" } } }, dataSupport = true, dynamicRegistration = false, isPreferredSupport = true, resolveSupport = { properties = { "edit" } } }, completion = { completionItem = { commitCharactersSupport = true, deprecatedSupport = true, documentationFormat = { "markdown", "plaintext" }, insertReplaceSupport = true, insertTextModeSupport = { valueSet = { 1, 2 } }, labelDetailsSupport = true, preselectSupport = true, resolveSupport = { properties = { "documentation", "detail", "additionalTextEdits" } }, snippetSupport = true, tagSupport = { valueSet = { 1 } } }, completionItemKind = { valueSet = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25 } }, completionList = { itemDefaults = { "commitCharacters", "editRange", "insertTextFormat", "insertTextMode", "data" } }, contextSupport = true, dynamicRegistration = false, insertTextMode = 1 }, declaration = { linkSupport = true }, definition = { linkSupport = true }, documentHighlight = { dynamicRegistration = false }, documentSymbol = { dynamicRegistration = false, hierarchicalDocumentSymbolSupport = true, symbolKind = { valueSet = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26 } } }, hover = { contentFormat = { "markdown", "plaintext" }, dynamicRegistration = false }, implementation = { linkSupport = true }, publishDiagnostics = { relatedInformation = true, tagSupport = { valueSet = { 1, 2 } } }, references = { dynamicRegistration = false }, rename = { dynamicRegistration = false, prepareSupport = true }, semanticTokens = { augmentsSyntaxTokens = true, dynamicRegistration = false, formats = { "relative" }, multilineTokenSupport = false, overlappingTokenSupport = true, requests = { full = { delta = true }, range = false }, serverCancelSupport = false, tokenModifiers = { "declaration", "definition", "readonly", "static", "deprecated", "abstract", "async", "modification", "documentation", "defaultLibrary" }, tokenTypes = { "namespace", "type", "class", "enum", "interface", "struct", "typeParameter", "parameter", "variable", "property", "enumMember", "event", "function", "method", "macro", "keyword", "modifier", "comment", "string", "number", "regexp", "operator", "decorator" } }, signatureHelp = { dynamicRegistration = false, signatureInformation = { activeParameterSupport = true, documentationFormat = { "markdown", "plaintext" }, parameterInformation = { labelOffsetSupport = true } } }, synchronization = { didSave = true, dynamicRegistration = false, willSave = true, willSaveWaitUntil = true }, typeDefinition = { linkSupport = true } }, window = { showDocument = { support = true }, showMessage = { messageActionItem = { additionalPropertiesSupport = false } }, workDoneProgress = true }, workspace = { applyEdit = true, configuration = true, didChangeWatchedFiles = { dynamicRegistration = false, relativePatternSupport = true }, semanticTokens = { refreshSupport = true }, symbol = { dynamicRegistration = false, hierarchicalWorkspaceSymbolSupport = true, symbolKind = { valueSet = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26 } } }, workspaceEdit = { resourceOperations = { "rename", "create", "delete" } }, workspaceFolders = true } }, clientInfo = { name = "Neovim", version = "0.9.5" }, initializationOptions = vim.empty_dict(), processId = 613557, rootPath = "/var/home/honza/tmp/nls-test", rootUri = "file:///var/home/honza/tmp/nls-test", trace = "off", workspaceFolders = { { name = "/var/home/honza/tmp/nls-test", uri = "file:///var/home/honza/tmp/nls-test" } } }}
[DEBUG][2024-04-03 11:16:37] .../vim/lsp/rpc.lua:387 "rpc.receive" { id = 1, jsonrpc = "2.0", result = { capabilities = { codeActionProvider = true, completionProvider = { triggerCharacters = { ".", '"', "/" } }, definitionProvider = true, documentFormattingProvider = true, documentSymbolProvider = true, executeCommandProvider = { commands = { "eval" } }, hoverProvider = { workDoneProgress = false }, referencesProvider = true, renameProvider = true, textDocumentSync = { change = 1, openClose = true } } }}
[DEBUG][2024-04-03 11:16:37] .../vim/lsp/rpc.lua:284 "rpc.send" { jsonrpc = "2.0", method = "initialized", params = vim.empty_dict()}
[INFO][2024-04-03 11:16:37] .../lua/vim/lsp.lua:1344 "LSP[nickel_ls]" "server_capabilities" { server_capabilities = { codeActionProvider = true, completionProvider = { triggerCharacters = { ".", '"', "/" } }, definitionProvider = true, documentFormattingProvider = true, documentSymbolProvider = true, executeCommandProvider = { commands = { "eval" } }, hoverProvider = { workDoneProgress = false }, referencesProvider = true, renameProvider = true, textDocumentSync = { change = 1, openClose = true } }}
[DEBUG][2024-04-03 11:16:37] .../vim/lsp/rpc.lua:284 "rpc.send" { jsonrpc = "2.0", method = "textDocument/didOpen", params = { textDocument = { languageId = "nickel", text = '{\n applications = {\n # apps | not_exported = applications,\n message = "message",\n blah | String = 5,\n }\n}\n', uri = "file:///var/home/honza/tmp/nls-test/test.ncl", version = 0 } }}
[DEBUG][2024-04-03 11:16:37] .../lua/vim/lsp.lua:1391 "LSP[nickel_ls]" "client.request" 1 "textDocument/documentSymbol" { textDocument = { uri = "file:///var/home/honza/tmp/nls-test/test.ncl" }} <function 1> 1
[DEBUG][2024-04-03 11:16:37] .../vim/lsp/rpc.lua:284 "rpc.send" { id = 2, jsonrpc = "2.0", method = "textDocument/documentSymbol", params = { textDocument = { uri = "file:///var/home/honza/tmp/nls-test/test.ncl" } }}
[DEBUG][2024-04-03 11:16:37] .../vim/lsp/rpc.lua:387 "rpc.receive" { jsonrpc = "2.0", method = "textDocument/publishDiagnostics", params = { diagnostics = {}, uri = "file:///var/home/honza/tmp/nls-test/test.ncl" }}
[DEBUG][2024-04-03 11:16:37] .../vim/lsp/rpc.lua:387 "rpc.receive" { id = 2, jsonrpc = "2.0", result = { { children = { { children = {}, detail = "String", kind = 13, name = "message", range = { ["end"] = { character = 11, line = 3 }, start = { character = 4, line = 3 } }, selectionRange = { ["end"] = { character = 11, line = 3 }, start = { character = 4, line = 3 } } }, { children = {}, detail = "String", kind = 13, name = "blah", range = { ["end"] = { character = 8, line = 4 }, start = { character = 4, line = 4 } }, selectionRange = { ["end"] = { character = 8, line = 4 }, start = { character = 4, line = 4 } } } }, detail = "Dyn", kind = 13, name = "applications", range = { ["end"] = { character = 14, line = 1 }, start = { character = 2, line = 1 } }, selectionRange = { ["end"] = { character = 14, line = 1 }, start = { character = 2, line = 1 } } } }}
[DEBUG][2024-04-03 11:16:37] .../lua/vim/lsp.lua:1391 "LSP[nickel_ls]" "client.request" 1 "textDocument/documentSymbol" { textDocument = { uri = "file:///var/home/honza/tmp/nls-test/test.ncl" }} <function 1> 1
[DEBUG][2024-04-03 11:16:37] .../vim/lsp/rpc.lua:284 "rpc.send" { id = 3, jsonrpc = "2.0", method = "textDocument/documentSymbol", params = { textDocument = { uri = "file:///var/home/honza/tmp/nls-test/test.ncl" } }}
[DEBUG][2024-04-03 11:16:37] .../vim/lsp/rpc.lua:387 "rpc.receive" { id = 3, jsonrpc = "2.0", result = { { children = { { children = {}, detail = "String", kind = 13, name = "message", range = { ["end"] = { character = 11, line = 3 }, start = { character = 4, line = 3 } }, selectionRange = { ["end"] = { character = 11, line = 3 }, start = { character = 4, line = 3 } } }, { children = {}, detail = "String", kind = 13, name = "blah", range = { ["end"] = { character = 8, line = 4 }, start = { character = 4, line = 4 } }, selectionRange = { ["end"] = { character = 8, line = 4 }, start = { character = 4, line = 4 } } } }, detail = "Dyn", kind = 13, name = "applications", range = { ["end"] = { character = 14, line = 1 }, start = { character = 2, line = 1 } }, selectionRange = { ["end"] = { character = 14, line = 1 }, start = { character = 2, line = 1 } } } }}
[DEBUG][2024-04-03 11:16:37] .../vim/lsp/rpc.lua:387 "rpc.receive" { jsonrpc = "2.0", method = "textDocument/publishDiagnostics", params = { diagnostics = { { message = "contract broken by the value of `blah`", range = { ["end"] = { character = 21, line = 4 }, start = { character = 20, line = 4 } }, severity = 1 }, { message = "expected type", range = { ["end"] = { character = 17, line = 4 }, start = { character = 11, line = 4 } }, severity = 4 }, { message = "applied to this expression", range = { ["end"] = { character = 21, line = 4 }, start = { character = 20, line = 4 } }, severity = 4 } }, uri = "file:///var/home/honza/tmp/nls-test/test.ncl" }}
[DEBUG][2024-04-03 11:16:39] .../vim/lsp/rpc.lua:284 "rpc.send" { jsonrpc = "2.0", method = "textDocument/didChange", params = { contentChanges = { { text = '{\n applications = {\n apps | not_exported = applications,\n message = "message",\n blah | String = 5,\n }\n}\n' } }, textDocument = { uri = "file:///var/home/honza/tmp/nls-test/test.ncl", version = 4 } }}
[DEBUG][2024-04-03 11:16:39] .../lua/vim/lsp.lua:1391 "LSP[nickel_ls]" "client.request" 1 "textDocument/documentSymbol" { textDocument = { uri = "file:///var/home/honza/tmp/nls-test/test.ncl" }} <function 1> 1
[DEBUG][2024-04-03 11:16:39] .../vim/lsp/rpc.lua:284 "rpc.send" { id = 4, jsonrpc = "2.0", method = "textDocument/documentSymbol", params = { textDocument = { uri = "file:///var/home/honza/tmp/nls-test/test.ncl" } }}
[DEBUG][2024-04-03 11:16:39] .../vim/lsp/rpc.lua:387 "rpc.receive" { jsonrpc = "2.0", method = "textDocument/publishDiagnostics", params = { diagnostics = {}, uri = "file:///var/home/honza/tmp/nls-test/test.ncl" }}
[ERROR][2024-04-03 11:16:39] .../vim/lsp/rpc.lua:734 "rpc" "nls" "stderr" "\nthread 'main' has overflowed its stack\nfatal runtime error: stack overflow\n"
[DEBUG][2024-04-03 11:16:39] .../vim/lsp/rpc.lua:284 "rpc.send" { jsonrpc = "2.0", method = "textDocument/didChange", params = { contentChanges = { { text = '{\n applications = {\n apps | not_exported = applications,\n message = "message",\n blah | String = 5,\n }\n}\n' } }, textDocument = { uri = "file:///var/home/honza/tmp/nls-test/test.ncl", version = 5 } }}
[ERROR][2024-04-03 11:16:39] .../vim/lsp/rpc.lua:734 "rpc" "nls" "stderr" '/home/honza/tmp/nls-test/nls: line 4: 613566 Aborted (core dumped) /home/honza/git/github/nickel/target/debug/nls "$@"\n'Thanks for the core dumps, those are super helpful. One of them is an infinite recursion in eval_permissive::inner, which should be fixed by #1878. The other is an infinite recursion when listing document symbols, which totally makes sense and I can reproduce it. (I guess some editor/lsp configs request document symbols eagerly; mine didn't, so it didn't crash)
uhlajs
commented
3 months ago Great, let me know if I can be more helpful here.
uhlajs
commented
3 months ago Thank you @jneem! I can confirm that #1881 resolves issues with nls crashing.
Unfortunately, the nls behavior is still not correct. With new patch, nls reports the same warning/error multiple times. There are 3 valid diagnostic messages, but after uncommenting the recursive line, nls reports 378 (=3*126).
https://github.com/tweag/nickel/assets/13243167/c12a670f-8fb1-4ccf-ae3b-a4b23606a1f8
Shall I open new issue or do you want to reuse this one?
yannham
commented
3 months ago @uhlajs thanks for all the reporting :slightly_smiling_face: that's super useful! If you don't mind, I think opening a new issue is preferable for multiple reasons (searchability, self-containment, etc.), as NLS indeed doesn't crash anymore on recursive definitions.
uhlajs
commented
3 months ago No problem -> moving the further discussion to #1882.
Describe the bug
nlscrashes with stack overflow error when parsing nickel source with recursive definition.To Reproduce
Open file with following content in text editor with LSP support and running
nls. I can reproduce it with lunarvim, but this should be easily reproduce with vanillaneovimor any other text editor.Expected behavior
Since this is a valid nickel source code, I would expect no
nlscrashes.Environment
Additional logs