search

wbond / package_control

The Sublime Text package manager
https://packagecontrol.io
4.8k stars 818 forks source link

0_package_control_loader is sometimes left in ignored_packages on removing unused dependencies #1485

Closed rchl closed 4 years ago

rchl commented 4 years ago

I don't have a deterministic way to reproduce it but sometimes, when PC removes unused dependencies, the 0_package_control_loader package is left in ignored_packages which breaks a lot of things.

I have reproduced this with debug: true enabled but there is not much logging to show:

ST console log ``` startup, version: 4081 osx x64 channel: dev executable: /Applications/Sublime Text.app/Contents/MacOS/Sublime Text working dir: / packages path: /Users/me/Library/Application Support/Sublime Text/Packages state path: /Users/me/Library/Application Support/Sublime Text/Local zip path: /Applications/Sublime Text.app/Contents/MacOS/Packages zip path: /Users/me/Library/Application Support/Sublime Text/Installed Packages ignored_packages: ["AngularJS", "ANSIescape", "ASP", "ayu", "Bootstrap 4 Autocomplete", "Bootstrap 4 Snippets", "D", "Debugger", "Erlang", "GoldenRatio", "Haskell", "IntakeToolkit", "Jade", "Lisp", "LSP-pyls", "Origami", "URLEncode", "Vintage"] pre session restore time: 1.50425 OpenGL Context Information: GL API Version: 4.1 ATI-3.10.15 GLSL Version: 4.10 Vendor: ATI Technologies Inc. Renderer: AMD Radeon Pro 560X OpenGL Engine loading dictionary Packages/Language - English/en_US.dic startup time: 1.75592 first paint time: 1.83739 git: tracking working dir /Users/me/workspace/github/package_control git: tracking working dir /Users/me/workspace/lsp/LSP-dockerfile git: tracking working dir /Users/me/workspace/lsp/lsp_utils git: tracking working dir /Users/me/workspace/lsp/LSP-json git: tracking working dir /Users/me/workspace/lsp/LSP-eslint git: tracking working dir /Users/me/workspace/lsp/LSP-intelephense git: tracking working dir /Users/me/workspace/lsp/LSP-vue git: tracking working dir /Users/me/workspace/lsp/LSP-pyls git: tracking working dir /Users/me/workspace/lsp/LSP-typescript git: tracking working dir /Users/me/workspace/lsp/LSP-pyright git: tracking working dir /Users/me/workspace/lsp/LSP-html git: tracking working dir /Users/me/workspace/lsp/LSP-lemminx git: tracking working dir /Users/me/workspace/lsp/LSP-css git: tracking working dir /Users/me/workspace/lsp/LSP-bash git: tracking working dir /Users/me/workspace/lsp/LSP git: tracking working dir /Users/me/workspace/lsp/LSP-elm git: tracking working dir /Users/me/workspace/lsp/LSP-svelte git: tracking working dir /Users/me/workspace/lsp/LSP-stylelint git: tracking working dir /Users/me/workspace/lsp/LSP-yaml environment variables loaded using: /bin/zsh -l reloading python 3.3 plugin 0_package_control_loader.00-package_control reloading python 3.3 plugin 0_package_control_loader.01-package_setting_context reloading python 3.3 plugin 0_package_control_loader.01-pygments reloading python 3.3 plugin 0_package_control_loader.01-sublime_lib reloading python 3.3 plugin 0_package_control_loader.10-lsp_utils reloading python 3.3 plugin 0_package_control_loader.10-pathtools reloading python 3.3 plugin 0_package_control_loader.10-wcwidth reloading python 3.3 plugin 0_package_control_loader.15-coverage reloading python 3.3 plugin 0_package_control_loader.20-watchdog reloading python 3.3 plugin 0_package_control_loader.40-ptyprocess reloading python 3.3 plugin 0_package_control_loader.50-dateutil reloading python 3.3 plugin 0_package_control_loader.50-markupsafe reloading python 3.3 plugin 0_package_control_loader.50-pathlib reloading plugin Default.arithmetic reloading python 3.3 plugin 0_package_control_loader.50-pymdownx reloading python 3.3 plugin 0_package_control_loader.50-pyte reloading python 3.3 plugin 0_package_control_loader.50-python-markdown reloading python 3.3 plugin 0_package_control_loader.50-pyyaml reloading python 3.3 plugin 0_package_control_loader.51-python-jinja2 reloading python 3.3 plugin 0_package_control_loader.55-mdpopups reloading python 3.3 plugin AppendSemiColon.AppendSemiColon reloading plugin Default.auto_indent_tag reloading python 3.3 plugin ChannelRepositoryTools.ordereddict reloading plugin Default.block reloading python 3.3 plugin ChannelRepositoryTools.tests reloading plugin Default.colors reloading plugin Default.comment reloading plugin Default.convert_color_scheme reloading python 3.3 plugin ChannelRepositoryTools.upgrade reloading plugin Default.convert_syntax reloading plugin Default.copy_path reloading plugin Default.echo reloading plugin Default.exec reloading plugin Default.fold reloading plugin Default.font reloading plugin Default.goto_line reloading plugin Default.history_list reloading plugin Default.indentation reloading plugin Default.install_package_control reloading python 3.3 plugin Class Navigator.class_navigator reloading python 3.3 plugin Codec.codec_base62 adding pybase62 to sys.path reloading python 3.3 plugin Codec.codec_base64 reloading python 3.3 plugin Codec.codec_hex reloading python 3.3 plugin Codec.codec_idn reloading python 3.3 plugin Codec.codec_json reloading python 3.3 plugin Codec.codec_quopri reloading python 3.3 plugin Codec.codec_xml reloading python 3.3 plugin Codec.sublime_codec reloading python 3.3 plugin DoxyDoxygen.Doxy reloading plugin Default.kill_ring reloading plugin Default.mark reloading plugin Default.new_templates ------==== DoxyDoxygen plugins loaded ====------ 0 sub-modules to reload reloading plugin Default.open_context_url reloading plugin Default.open_in_browser reloading plugin Default.pane reloading plugin Default.paragraph reloading plugin Default.paste_from_history reloading plugin Default.profile reloading plugin Default.quick_panel reloading plugin Default.rename reloading plugin Default.run_syntax_tests reloading plugin Default.save_on_focus_lost reloading plugin Default.scroll reloading plugin Default.set_unsaved_view_name reloading plugin Default.settings reloading plugin Default.show_scope_name reloading plugin Default.side_bar reloading plugin Default.sort reloading plugin Default.switch_file reloading plugin Default.symbol reloading plugin Default.transform reloading plugin Default.transpose reloading plugin Default.ui reloading plugin CSS.css_completions reloading plugin Diff.diff reloading plugin HTML.encode_html_entities reloading plugin HTML.html_completions reloading python 3.3 plugin Emmet2.abbreviation reloading plugin ShellScript.ShellScript reloading plugin FileManager.FileManager reloading python 3.3 plugin Emmet2.balance reloading python 3.3 plugin Emmet2.comment reloading python 3.3 plugin Emmet2.convert_data_url reloading python 3.3 plugin Emmet2.emmet_sublime reloading python 3.3 plugin Emmet2.evaluate_math reloading python 3.3 plugin Emmet2.go_to_edit_point reloading python 3.3 plugin Emmet2.go_to_tag_pair reloading plugin FileManager.prevent_default reloading plugin LESS.less_completions reloading python 3.3 plugin Emmet2.html_highlight reloading python 3.3 plugin Emmet2.inc_dec_number reloading python 3.3 plugin Emmet2.preview reloading python 3.3 plugin Emmet2.remove_tag reloading python 3.3 plugin Emmet2.select_item reloading python 3.3 plugin Emmet2.split_join_tag reloading python 3.3 plugin Emmet2.syntax reloading plugin OpenUri.boot reloading python 3.3 plugin Emmet2.tracker reloading python 3.3 plugin Emmet2.update_image_size reloading python 3.3 plugin Emmet2.utils reloading python 3.3 plugin Emmet2.wrap_with_abbreviation reloading python 3.3 plugin ESLint Fix.eslint-fix reloading python 3.3 plugin ExternalDiff.ExternalDiff reloading python 3.3 plugin FileClose.FileClose reloading python 3.3 plugin Git blame.git-blame reloading python 3.3 plugin GitSavvy.__init__ reloading python 3.3 plugin GitSavvy.git_savvy reloading plugin Pretty JSON.PrettyJson reloading plugin Pretty JSON.PrettyJsonListeners reloading plugin Sass.sass_completions reloading plugin UnitTesting.ut reloading plugin User._example_command reloading plugin User._test-threading reloading plugin User.augment_plugin_profiling reloading plugin User.cleanup_docx_html reloading plugin User.convert_into_template_literal reloading plugin User.CopyPathEx reloading plugin User.double_clicker reloading plugin User.extend_search_results reloading plugin User.find-clipboard-strings reloading plugin User.fix_osx_path reloading plugin User.header_guard reloading plugin User.npm-package-build reloading plugin User.on_text_changed reloading plugin User.pre-save reloading plugin User.switch_file_ex reloading plugin User.timeout reloading plugin User.view_event_listener reloading plugin User.wrap_in_console reloading python 3.3 plugin GotoDocumentation.goto_documentation reloading python 3.3 plugin Indent To Parenthesis.indent_to_parenthesis reloading python 3.3 plugin Jump Along Indent.file_scanner reloading python 3.3 plugin Jump Along Indent.jump_along_indent reloading python 3.3 plugin Jump Along Indent.view_helper reloading python 3.3 plugin Less2Css.less2css reloading python 3.3 plugin Less2Css.lesscompiler reloading python 3.3 plugin MarkdownPreview.markdown_preview reloading python 3.3 plugin MarkdownPreview.markdown_settings reloading python 3.3 plugin MarkdownPreview.markdown_wrapper reloading python 3.3 plugin MonkeyC.commands reloading python 3.3 plugin OverrideAudit.override_audit reloading python 3.3 plugin PackageDev._logging reloading python 3.3 plugin PackageDev.main reloading python 3.3 plugin PackageResourceViewer.package_resource_viewer reloading python 3.3 plugin PackageResourceViewer.package_resources reloading python 3.3 plugin ProjectManager.json_file reloading python 3.3 plugin ProjectManager.project_manager reloading python 3.3 plugin RawLineEdit.raw_line_edit reloading python 3.3 plugin RawLineEdit.support reloading python 3.3 plugin Rsync SSH.rsync_ssh reloading python 3.3 plugin SelectUntil.edit reloading python 3.3 plugin SelectUntil.select-until reloading python 3.3 plugin Skins.skins reloading python 3.3 plugin Terminus.main reloading python 3.3 plugin ClangFormat.ClangFormat reloading python 3.3 plugin JsPrettier.JsPrettier reloading python 3.3 plugin LSP.boot reloading python 3.3 plugin LSP-bash.plugin reloading python 3.3 plugin LSP-css.plugin reloading python 3.3 plugin LSP-dockerfile.plugin reloading python 3.3 plugin LSP-elm.plugin reloading python 3.3 plugin LSP-eslint.plugin reloading python 3.3 plugin LSP-html.plugin reloading python 3.3 plugin LSP-intelephense.plugin reloading python 3.3 plugin LSP-json.plugin reloading python 3.3 plugin LSP-lemminx.plugin reloading python 3.3 plugin LSP-pyright.plugin reloading python 3.3 plugin LSP-stylelint.plugin reloading python 3.3 plugin LSP-svelte.plugin reloading python 3.3 plugin LSP-typescript.plugin reloading python 3.3 plugin LSP-vue.plugin reloading python 3.3 plugin LSP-yaml.plugin reloading python 3.3 plugin Package Control.1_reloader reloading python 3.3 plugin Package Control.2_bootstrap reloading python 3.3 plugin Package Control.Package Control reloading python 3.3 plugin package_setting_context.__init__ reloading python 3.3 plugin PhpArrayConverter.PhpArrayConverter reloading python 3.3 plugin PlainTasks.APlainTasksCommon reloading python 3.3 plugin PlainTasks.PlainTasks reloading python 3.3 plugin PlainTasks.PlainTasksDates reloading python 3.3 plugin PlainTasks.PlainTasksToHTML reloading python 3.3 plugin PlainTasks.plist_parser reloading python 3.3 plugin TodoReview.TodoReview reloading python 3.3 plugin TrailingSpaces.trailing_spaces reloading python 3.3 plugin UnitTesting33.ut plugins loaded OverrideAudit: Initializing OverrideAudit: Sublime version is unchanged; skipping automatic report LSP: global configs ['LSP-eslint=True', 'LSP-bash=True', 'LSP-css=True', 'LSP-pyright=True', 'LSP-vue=True', 'LSP-html=True', 'LSP-elm=True', 'LSP-typescript=True', 'LSP-intelephense=True', 'lemminx=True', 'LSP-json=True', 'LSP-svelte=True', 'LSP-dockerfile=True', 'LSP-yaml=True', 'LSP-stylelint=True', 'phan=False', 'sorbet=False', 'flow=False', 'haskell-ide-engine=False', 'sourcekit-lsp=False', 'pyls=False', 'dart=False', 'terraform-ls=True', 'jedi-language-server=False', 'gopls=True', 'golsp=False', 'clangd=False', 'polymer-ide=False', 'jdtls=False', 'rlang=False', 'elixir-ls=False', 'ocaml=False', 'ruby=False', 'vscode-css=False', 'erlang-ls=False', 'rust-analyzer=False', 'reason=False'] [OpenUri][INFO] Set log level: INFO [OpenUri][INFO] Activated schemes: ['file://', 'ftp://', 'ftps://', 'http://', 'https://', 'mailto:'] reloading python 3.3 plugin UnitTesting33.ut LSP: starting ['node', '/Users/me/Library/Application Support/Sublime Text/Cache/LSP-pyright/language-server/node_modules/pyright/langserver.index.js', '--stdio'] in /Users/me/workspace/lsp/LSP LSP: LSP-pyright: supported code action kinds: ['quickfix', 'source.organizeImports'] LSP: LSP-pyright: registering capability: workspace.workspaceFolders Package Control: Removed directory for unneeded dependency watchdog ignored packages updated to: ["0_package_control_loader", "AngularJS", "ANSIescape", "ASP", "ayu", "Bootstrap 4 Autocomplete", "Bootstrap 4 Snippets", "D", "Debugger", "Erlang", "GoldenRatio", "Haskell", "IntakeToolkit", "Jade", "Lisp", "LSP-pyls", "Origami", "URLEncode", "Vintage"] unloading python 3.3 plugin 0_package_control_loader.00-package_control unloading python 3.3 plugin 0_package_control_loader.01-package_setting_context unloading python 3.3 plugin 0_package_control_loader.01-pygments unloading python 3.3 plugin 0_package_control_loader.01-sublime_lib unloading python 3.3 plugin 0_package_control_loader.10-lsp_utils unloading python 3.3 plugin 0_package_control_loader.10-pathtools unloading python 3.3 plugin 0_package_control_loader.10-wcwidth unloading python 3.3 plugin 0_package_control_loader.15-coverage unloading python 3.3 plugin 0_package_control_loader.20-watchdog unloading python 3.3 plugin 0_package_control_loader.40-ptyprocess unloading python 3.3 plugin 0_package_control_loader.50-dateutil unloading python 3.3 plugin 0_package_control_loader.50-markupsafe unloading python 3.3 plugin 0_package_control_loader.50-pathlib unloading python 3.3 plugin 0_package_control_loader.50-pymdownx unloading python 3.3 plugin 0_package_control_loader.50-pyte unloading python 3.3 plugin 0_package_control_loader.50-python-markdown unloading python 3.3 plugin 0_package_control_loader.50-pyyaml unloading python 3.3 plugin 0_package_control_loader.51-python-jinja2 unloading python 3.3 plugin 0_package_control_loader.55-mdpopups Package Control: Removed directory for unneeded dependency pathtools reloading settings Packages/User/Package Control.sublime-settings reloading settings Packages/User/Preferences.sublime-settings ignored packages updated to: ["AngularJS", "ANSIescape", "ASP", "ayu", "Bootstrap 4 Autocomplete", "Bootstrap 4 Snippets", "D", "Debugger", "Erlang", "GoldenRatio", "Haskell", "IntakeToolkit", "Jade", "Lisp", "LSP-pyls", "Origami", "URLEncode", "Vintage"] ignored packages updated to: ["0_package_control_loader", "AngularJS", "ANSIescape", "ASP", "ayu", "Bootstrap 4 Autocomplete", "Bootstrap 4 Snippets", "D", "Debugger", "Erlang", "GoldenRatio", "Haskell", "IntakeToolkit", "Jade", "Lisp", "LSP-pyls", "Origami", "URLEncode", "Vintage"] reloading settings Packages/User/Package Control.sublime-settings reloading settings Packages/User/Preferences.sublime-settings reloading settings Packages/User/Preferences.sublime-settings Package Control: Skipping automatic upgrade, last run at 2020-08-08 22:24:44, next run at 2020-08-08 23:24:44 or after ```

The dependencies that were no longer used in this session were pathtools and watchdog.

I have been on Package Control c12a0b7f593e3cf647fa7318649aab310ae61320 commit when this happened.

rchl commented 4 years ago

I suppose it's weird that the 0_package_control_loader is removed from ignored_packages and then immediately added back:

ignored packages updated to: ["AngularJS", "ANSIescape", "ASP", "ayu", "Bootstrap 4 Autocomplete", "Bootstrap 4 Snippets", "D", "Debugger", "Erlang", "GoldenRatio", "Haskell", "IntakeToolkit", "Jade", "Lisp", "LSP-pyls", "Origami", "URLEncode", "Vintage"]
ignored packages updated to: ["0_package_control_loader", "AngularJS", "ANSIescape", "ASP", "ayu", "Bootstrap 4 Autocomplete", "Bootstrap 4 Snippets", "D", "Debugger", "Erlang", "GoldenRatio", "Haskell", "IntakeToolkit", "Jade", "Lisp", "LSP-pyls", "Origami", "URLEncode", "Vintage"]
rchl commented 4 years ago

I've spent too much time debugging this but it looks to me like some core bug. Nothing in Python space seems to be doing anything wrong. The relevant logs are:

load_settings('Preferences.sublime-settings')
GET.ignored_packages ['0_package_control_loader', 'AngularJS', 'ANSIescape', 'ASP', 'ayu', 'Bootstrap 4 Autocomplete', 'Bootstrap 4 Snippets', 'D', 'Debugger', 'Erlang', 'GoldenRatio', 'Haskell', 'IntakeToolkit', 'Jade', 'Lisp', 'LSP-pyls', 'Origami', 'URLEncode', 'Vintage']
SET ignored_packages ['AngularJS', 'ANSIescape', 'ASP', 'ayu', 'Bootstrap 4 Autocomplete', 'Bootstrap 4 Snippets', 'D', 'Debugger', 'Erlang', 'GoldenRatio', 'Haskell', 'IntakeToolkit', 'Jade', 'Lisp', 'LSP-pyls', 'Origami', 'URLEncode', 'Vintage']
ignored packages updated to: ["AngularJS", "ANSIescape", "ASP", "ayu", "Bootstrap 4 Autocomplete", "Bootstrap 4 Snippets", "D", "Debugger", "Erlang", "GoldenRatio", "Haskell", "IntakeToolkit", "Jade", "Lisp", "LSP-pyls", "Origami", "URLEncode", "Vintage"]
save_settings('Preferences.sublime-settings')
load_settings('Package Control.sublime-settings')
SET.in_process_packages []
save_settings('Package Control.sublime-settings')
ignored packages updated to: ["0_package_control_loader", "AngularJS", "ANSIescape", "ASP", "ayu", "Bootstrap 4 Autocomplete", "Bootstrap 4 Snippets", "D", "Debugger", "Erlang", "GoldenRatio", "Haskell", "IntakeToolkit", "Jade", "Lisp", "LSP-pyls", "Origami", "URLEncode", "Vintage"]

After 0_package_control_loader is removed from ignored packages and settings are saved, the Package Control.sublime-settings are loaded and saved and this might be triggering revert of changes to Preferences.sublime-settings. At least it seems like that from the outside but I can't reproduce that from the console easily.

FichteFoll commented 4 years ago

Potentially related: #1164

wbond commented 4 years ago

I added some code in fd5d782db27ebbe084ca3d77f7309a27c22de7df that should ensure we haven't touched any loader code for a while (900ms) before we try and re-enable 0_package_control_loader. I'm planning on doing more testing, but if you have a change I wouldn't mind a second attempt.

rchl commented 4 years ago

I gave it a good 6 or 7 tries and couldn't reproduce. After going back to 3 commits before, it reproduced on the second try. So it's looking good to me. 👍

wbond commented 4 years ago

Awesome, thanks for confirming!

I should have a beta release in the next couple days.

evandrocoan commented 2 years ago

@rchl I suppose it's weird that the 0_package_control_loader is removed from ignored_packages and then immediately added back:

ignored packages updated to: ["AngularJS", "ANSIescape", "ASP", "ayu", "Bootstrap 4 Autocomplete", "Bootstrap 4 Snippets", "D", "Debugger", "Erlang", "GoldenRatio", "Haskell", "IntakeToolkit", "Jade", "Lisp", "LSP-pyls", "Origami", "URLEncode", "Vintage"]
ignored packages updated to: ["0_package_control_loader", "AngularJS", "ANSIescape", "ASP", "ayu", "Bootstrap 4 Autocomplete", "Bootstrap 4 Snippets", "D", "Debugger", "Erlang", "GoldenRatio", "Haskell", "IntakeToolkit", "Jade", "Lisp", "LSP-pyls", "Origami", "URLEncode", "Vintage"]

Sublime Text is doing this on its own: https://github.com/sublimehq/sublime_text/issues/2132 Randomly reverting back the ignored_packages setting on batch operations

Increasing the delay as done on https://github.com/wbond/package_control/commit/fd5d782db27ebbe084ca3d77f7309a27c22de7df avoids this bevior and was the solution I forced in my scripts.