shutdown sent between initialize and initialized

[PerMildner]

Sublime Text (sometimes?) sends a shutdown request after initialize but before subsequent initialized and didOpen, see attached log. This is not a valid sequence of messages.

After shutdown the server should only accept exit so it is effectively stopped. The specification says "If a server receives requests after a shutdown request those requests should be errored with InvalidRequest".

Please include in your report:

• OS and language server macOS 10.14.6. Non-public language server
• How you installed LSP (Package Control or from git?) Installed with Package Control
• Minimal reproduction steps No open files or folders. Open a file associated to a language server. Not clear whether it happens every time.
• Log

  
  startup, version: 3207 osx x64 channel: stable
  executable: /Applications/Sublime Text.app/Contents/MacOS/Sublime Text
  working dir: /
  packages path: /Users/perm/Library/Application Support/Sublime Text 3/Packages
  state path: /Users/perm/Library/Application Support/Sublime Text 3/Local
  zip path: /Applications/Sublime Text.app/Contents/MacOS/Packages
  zip path: /Users/perm/Library/Application Support/Sublime Text 3/Installed Packages
  ignored_packages: ["Vintage"]
  pre session restore time: 0.181761
  using gpu buffer for window
  startup time: 0.258246
  environment variables loaded using: /bin/bash -l
  reloading plugin Default.arithmetic
  reloading plugin Default.auto_indent_tag
  reloading plugin Default.block
  reloading plugin Default.colors
  reloading plugin Default.comment
  reloading plugin Default.convert_color_scheme
  reloading plugin Default.convert_syntax
  reloading plugin Default.copy_path
  reloading plugin Default.detect_indentation
  reloading plugin Default.echo
  reloading plugin Default.exec
  reloading plugin Default.fold
  reloading plugin Default.font
  reloading plugin Default.goto_line
  reloading plugin Default.history_list
  reloading plugin Default.indentation
  reloading plugin Default.install_package_control
  reloading plugin Default.kill_ring
  reloading plugin Default.mark
  reloading plugin Default.new_templates
  reloading plugin Default.open_context_url
  reloading plugin Default.open_in_browser
  reloading plugin Default.pane
  reloading plugin Default.paragraph
  reloading plugin Default.paste_from_history
  reloading plugin Default.profile
  reloading plugin Default.quick_panel
  reloading plugin Default.rename
  reloading plugin Default.run_syntax_tests
  reloading plugin Default.save_on_focus_lost
  reloading plugin Default.scroll
  reloading plugin Default.set_unsaved_view_name
  reloading plugin Default.settings
  reloading plugin Default.show_scope_name
  reloading plugin Default.side_bar
  reloading plugin Default.sort
  reloading plugin Default.switch_file
  reloading plugin Default.symbol
  reloading plugin Default.transform
  reloading plugin Default.transpose
  reloading plugin Default.ui
  reloading plugin CSS.css_completions
  reloading plugin Diff.diff
  reloading plugin HTML.encode_html_entities
  reloading plugin HTML.html_completions
  reloading plugin ShellScript.ShellScript
  reloading plugin 0_package_control_loader.00-package_control
  reloading plugin 0_package_control_loader.01-pygments
  reloading plugin 0_package_control_loader.50-markupsafe
  reloading plugin 0_package_control_loader.50-pymdownx
  reloading plugin 0_package_control_loader.50-python-markdown
  reloading plugin 0_package_control_loader.50-pyyaml
  reloading plugin 0_package_control_loader.51-python-jinja2
  reloading plugin 0_package_control_loader.55-mdpopups
  reloading plugin LSP.boot
  reloading plugin Package Control.1_reloader
  reloading plugin Package Control.2_bootstrap
  reloading plugin Package Control.Package Control
  plugins loaded
  LSP: global configs ['jdtls=False', 'phpls=False', 'flow=False', 'reason=False', 'cquery=False', 'clangd=False', 'lsp-tsserver=False', 'eslint=False', 'ocaml=False', 'ruby=False', 'javascript-typescript-langserver=False', 'pyls=False', 'haskell-ide-engine=False', 'ra-lsp=False', 'polymer-ide=False', 'gopls=False', 'vscode-css=False', 'intelephense-ls=False', 'metals=False', 'typescript-language-server=False', 'golsp=False', 'dart=False', 'spider-nc=True', 'rls=False', 'bashls=False']
  LSP: Couldn't determine project directory since no folders are open and the current file isn't saved on the disk.
  LSP: window 2 starting 1 initial views
  LSP: window 2 requests spider-nc for /Users/perm/sicstus/lsptestfolder/foo.pl
  LSP: Couldn't determine project directory since no folders are open! Using /Users/perm/sicstus/lsptestfolder as a fallback.
  LSP: starting in /Users/perm/sicstus/lsptestfolder
  LSP: starting ['/usr/bin/nc', 'localhost', '56789']
  LSP:  --> initialize
  LSP: window 2 added session spider-nc
  LSP: Couldn't determine project directory since no folders are open! Using /Users/perm/sicstus/lsptestfolder as a fallback.
  LSP: project path changed, ending existing sessions
  LSP: new path = /Users/perm/sicstus/lsptestfolder
  LSP: unloading session spider-nc
  LSP:  --> shutdown
  Package Control: Skipping automatic upgrade, last run at 2019-08-01 16:21:54, next run at 2019-08-01 17:21:54 or after
  LSP:      {'capabilities': {'documentFormattingProvider': True, 'implementationProvider': False, 'codeActionProvider': False, 'callHierarchyProvider': False, 'renameProvider': True, 'typeDefinitionProvider': False, 'workspaceSymbolProvider': True, 'definitionProvider': True, 'documentRangeFormattingProvider': True, 'workspace': {'workspaceFolders': {'supported': True, 'changeNotifications': True}}, 'typeHierarchyProvider': False, 'documentHighlightProvider': True, 'referencesProvider': True, 'textDocumentSync': 2, 'hoverProvider': True, 'documentSymbolProvider': True, 'completionProvider': {'triggerCharacters': ['(']}}}
  LSP:  --> initialized
  LSP:  --> textDocument/didOpen
  LSP:      None
  LSP:  --> exit
  LSP: session spider-nc ended
  LSP: clients for window 2 unloaded
  LSP: window 2 requests spider-nc for /Users/perm/sicstus/lsptestfolder/foo.pl
  LSP: Couldn't determine project directory since no folders are open! Using /Users/perm/sicstus/lsptestfolder as a fallback.
  LSP: starting in /Users/perm/sicstus/lsptestfolder
  LSP: starting ['/usr/bin/nc', 'localhost', '56789']
  LSP:  --> initialize
  LSP: window 2 added session spider-nc
  LSP: LSP stdout process ended.
  using gpu buffer for window
  Unable to open /Users/perm/Library/Application Support/Sublime Text 3/Packages/Default/Preferences.sublime-settings
  LSP: Couldn't determine project directory since no folders are open! Using /Users/perm/Library/Application Support/Sublime Text 3/Packages/User as a fallback.
  LSP: window 3 requests spider-nc for /Users/perm/Library/Application Support/Sublime Text 3/Packages/Default/Preferences.sublime-settings
  LSP: Couldn't determine project directory since no folders are open! Using /Users/perm/Library/Application Support/Sublime Text 3/Packages/User as a fallback.
  LSP: starting in /Users/perm/Library/Application Support/Sublime Text 3/Packages/User
  LSP: starting ['/usr/bin/nc', 'localhost', '56789']
  LSP:  --> initialize
  LSP: window 3 added session spider-nc
  LSP: LSP stdout process ended.
  LSP: no config found or enabled for view /Users/perm/Library/Application Support/Sublime Text 3/Packages/Default/Preferences.sublime-settings
  LSP: no config found or enabled for view /Users/perm/Library/Application Support/Sublime Text 3/Packages/Default/Preferences.sublime-settings
  LSP: Couldn't determine project directory since no folders are open! Using /Users/perm/Library/Application Support/Sublime Text 3/Packages/User as a fallback.
  LSP: Couldn't determine project directory since no folders are open! Using /Users/perm/Library/Application Support/Sublime Text 3/Packages/User as a fallback.
  LSP: Couldn't determine project directory since no folders are open and the current file isn't saved on the disk.
  LSP: project path changed, ending existing sessions
  LSP: new path = None
  LSP: unloading session spider-nc
  LSP:  --> shutdown
  Failure writing to stdout
  Traceback (most recent call last):
  File "/Users/perm/Library/Application Support/Sublime Text 3/Installed Packages/LSP.sublime-package/plugin/core/transports.py", line 227, in write_stdin
  self.process.stdin.flush()
  BrokenPipeError: [Errno 32] Broken pipe

LSP: Communication to server closed, exiting LSP: transport failed LSP: Couldn't determine project directory since no folders are open! Using /Users/perm/sicstus/lsptestfolder as a fallback. reloading settings Packages/User/Preferences.sublime-settings reloading settings Packages/User/Preferences.sublime-settings reloading settings Packages/User/Preferences.sublime-settings reloading settings Packages/User/Preferences.sublime-settings

[tomv564]

The shutdown message is sent because LSP thinks you have opened a different project.

LSP is also logging that you have not created a sublime project for your 'lsptestproject' folder. Create a project for this folder and I bet this issue will disappear. LSP tries to handle support project-less operation on a best-effort basis, but bugs like these are expected.

Explanation: LSP detects a supported file (foo.pl) for which it starts a session in lsptestfolder and then an unsaved file is opened for which project_path is None. When project path changes from we assume you've stopped working in lsptestfolder, so we end that session.

This logic works reliably with sublime projects but needs substantial changes for project-less windows. If someone can find a really elegant way to manage sessions then please propose a design in a new issue or PR.

[tomv564]

I feel LSP does too little to warn users about project-less mode. Perhaps we can either:

• Disable it behind a setting flag (this will cause many more "it's not working" issues)
• Add a warning dialog every time a window starts a project-less session.

[PerMildner]

The problem is not so much the shutdown message (and the missing exit message that ought to follow the shutdown). The problem is that didOpen is sent to a server that Sublime Text knows it has shut down. This is a LSP protocol violation.

If a proper shutdown (shutdown + exit) was followed by a proper startup (launch server, initialize, ...) before the didOpen, then it would perhaps not work perfectly for project-less files, but at least it would adhere to the LSP protocol. As it now stands a conformant server will just report a InvalidRequest error for all subsequent requests.

Perhaps it would be possibly to simply start one server instance for each project-less file? Or, not use the single server instance except for the first file, and not use LSP for subsequent files.

Other clients seems to just default to the surrounding folder as rootURI (and workspace folder) so everything works OK as long as all files are in, or below, that initial folder. Then, there is the workspace/didChangeWorkspaceFolders LSP-message that perhaps could be used to just add any new containing folder. I know nothing about ST projects, though.

[PerMildner]

Looking some more at this. It seems exit is sent, at least sometimes, some time after the shutdown, but I do not see it always (so, there is not a fundamental bug that "exit is not sent after shutdown").

Looking at the source (*) I suspect a thread synchronization issue. I see nothing that prevents the didOpen (which seems to be posted from its own async thread) from getting sent to the server after a shutdown has been sent.

Also, once, when flipping around among the file tabs in Sublime Text, which triggers the shutdown/restart/whatever of the server, it appeared that several shutdown messages was buffered somewhere and then all was received by the same server. Since there are so many moving parts, and timing is involved, this is really hard to troubleshoot, so this particular symptom may be a false alarm.

(*) Be aware that I know nothing about the Sublime Text API, nor of Python, so this is just informed guesswork.

[tomv564]

Yeah, it's hard to reason about it - improving behaviour in single-file mode is the first priority as it can be hiding other bugs as well. There is no "shutting down" state for the session, but that would prevent a late didOpen being sent to it. Let's keep this issue open to fix the single-file mode first, then look at the behaviour again.

[tomv564]

I think with #713 some of the unintended shutdowns should have disappeared. From your logs, I am not sure why your spider-nc config is being started for a preferences edit:

window 3 requests spider-nc for /Users/perm/Library/Application Support/Sublime Text 3/Packages/Default/Preferences.sublime-settings

I'll close the issue now without adding a "shutting down" state to sessions - assuming LSP will rarely have a legitimate reason to shutdown sessions that were just started.