Index Workers Threads stuck while indexing at 89%

[evandrocoan]

Summary

Expected behavior

Kill the threads when they are stuck and not eat 100% of processor for ever and ever.

Actual behavior

The indexer threads were stuck, but not kill and kept eat all my CPU 100%.

Steps to reproduce

I do not know how to create the steps, It was just indexing my project. It happens every time I open Sublime Text, and it is always stuck with the same file, presented on the screenshot just above.

The file can be found on: https://github.com/webmozart/assert

But I put on that file on a vanilla install and it does not stuck. May be it is because of the path.

Full console:

``` DPI scale: 1 startup, version: 3141 windows x32 channel: dev executable: /D/User/Dropbox/Applications/SoftwareVersioning/SublimeText/sublime_text.exe working dir: /C/ProgramData/Microsoft/Windows/Start Menu/Atalhos packages path: /D/User/Dropbox/Applications/SoftwareVersioning/SublimeText/Data/Packages state path: /D/User/Dropbox/Applications/SoftwareVersioning/SublimeText/Data/Local zip path: /D/User/Dropbox/Applications/SoftwareVersioning/SublimeText/Packages zip path: /D/User/Dropbox/Applications/SoftwareVersioning/SublimeText/Data/Installed Packages ignored_packages: ["All Autocomplete", "Anaconda", "ApplySyntax", "Boxy Theme", "BracketHighlighter", "C++ Completions", "C++ Snippets", "C++ Starting Kit", "ColorHelper", "DictionaryAutoComplete", "FileManager", "Find++", "Gist", "Git", "GitGutter", "Javatar", "Local History", "Matlab Completions", "MatlabFilenameAutoComplete", "MySQL Snippets", "Project Specific Syntax Settings", "Qt Completions for C++", "ScopeAlways", "Select All Spelling Errors", "SQL Keyword Uppercase", "SQLExec", "SQLTools", "Sublime Tutor", "sublime-text-2-buildview", "SublimeCodeIntel", "SyncedSideBar", "SyntaxManager", "Vintage", "WordHighlight"] Unable to parse binding {keys: [ctrl+alt+w]} Unable to parse binding {keys: [ctrl+shift+w]} pre session restore time: 1.16081 loading dictionary Packages/User/MultiLingual Dictionary/EN_PT.dic startup time: 3.57881 first paint time: 3.58381 Error loading scope:source.css.less: Unable to find syntax file for scope source.css.less in Packages/MarkdownLight/MarkdownLight.sublime-syntax rule scope:source.rust##anon_struct-tuple_0 has a scope name, but is unreachable, so the name will never be used rule scope:source.scala#xml-comments has a scope name, but is unreachable, so the name will never be used reloading plugin Default.auto_indent_tag reloading plugin Default.block reloading plugin Default.comment reloading plugin Default.convert_syntax reloading plugin Default.copy_path reloading plugin Default.delete_word reloading plugin Default.detect_indentation reloading plugin Default.duplicate_line reloading plugin Default.echo reloading plugin Default.exec reloading plugin Default.fold reloading plugin Default.font reloading plugin Default.goto_line reloading plugin Default.history_list reloading plugin Default.indentation reloading plugin Default.install_package_control reloading plugin Default.kill_ring reloading plugin Default.mark reloading plugin Default.new_templates reloading plugin Default.open_context_url reloading plugin Default.open_in_browser reloading plugin Default.pane reloading plugin Default.paragraph reloading plugin Default.paste_from_history reloading plugin Default.profile reloading plugin Default.quick_panel reloading plugin Default.run_syntax_tests reloading plugin Default.save_on_focus_lost reloading plugin Default.scroll reloading plugin Default.set_unsaved_view_name reloading plugin Default.settings reloading plugin Default.show_scope_name reloading plugin Default.side_bar reloading plugin Default.sort reloading plugin Default.swap_line reloading plugin Default.switch_file reloading plugin Default.symbol reloading plugin Default.transform reloading plugin Default.transpose reloading plugin Default.trim_trailing_white_space reloading plugin Default.ui reloading plugin CSS.css_completions reloading plugin Diff.diff reloading plugin HTML.encode_html_entities reloading plugin HTML.html_completions reloading plugin 0_package_control_loader.00-package_control reloading plugin 0_package_control_loader.01-package_setting_context reloading plugin 0_package_control_loader.01-pygments reloading plugin 0_package_control_loader.50-backrefs reloading plugin 0_package_control_loader.50-markupsafe reloading plugin 0_package_control_loader.50-python-markdown reloading plugin 0_package_control_loader.50-pyyaml reloading plugin 0_package_control_loader.50-regex reloading plugin 0_package_control_loader.51-python-jinja2 reloading plugin 0_package_control_loader.55-mdpopups reloading plugin A File Icon.A File Icon reloading plugin Add Folder To Project.AddFolderToProject reloading plugin AdvancedNewFile.AdvancedNewFile reloading plugin Alignment.Alignment reloading plugin AlignTab.aligner reloading plugin AlignTab.aligntab reloading plugin AlignTab.hist reloading plugin AlignTab.parser reloading plugin AlignTab.table reloading plugin AlignTab.wclen reloading plugin Auto Refresh.AutoRefresh reloading plugin Better CoffeeScript.CoffeeScript reloading plugin Case Conversion.case_conversion reloading plugin Case Conversion.case_parse reloading plugin Chain of Command.chain reloading plugin ChangeQuotes.change_quotes reloading plugin ChannelRepositoryTools.ordereddict reloading plugin ChannelRepositoryTools.tests reloading plugin ChannelRepositoryTools.upgrade reloading plugin Clickable URLs.clickable_urls reloading plugin Color Highlighter.ColorHighlighter reloading plugin Color Highlighter.colors reloading plugin ColorSchemeEditor.ColorSchemeEditor-ST2 reloading plugin Column Select.column_select reloading plugin Compare Side-By-Side.sbs_compare reloading plugin Copy Filepath With Line Numbers.CopyFilepathWithLineNumbers reloading plugin CopyWithLineNumbersReloaded.copy_with_line_numbers reloading plugin Delete Current File.DeleteCurrentFile reloading plugin DistractionFreeWindow.distraction_free_window reloading plugin DocBlockr.jsdocs reloading plugin DuplicateSelections.duplicate_selections reloading plugin Edit Preferences.__init__ reloading plugin Edit Preferences.commands_base reloading plugin Edit Preferences.edit_package_files reloading plugin Edit Preferences.extract_snippets reloading plugin Edit Preferences.helper_commands reloading plugin Edit Preferences.helpers reloading plugin Edit Preferences.insert_binding_repr reloading plugin Edit Preferences.jsonix reloading plugin Edit Preferences.list_commands reloading plugin Edit Preferences.list_menu_bindings reloading plugin Edit Preferences.list_settings reloading plugin Edit Preferences.list_shortcut_keys reloading plugin Edit Preferences.list_theme_selectors reloading plugin Edit Preferences.package_resources reloading plugin Edit Preferences.quick_panel_cols reloading plugin Edit Preferences.scheduler reloading plugin Emmet.emmet-plugin reloading plugin ExportHtml.ExportBbcode reloading plugin ExportHtml.ExportHtml reloading plugin ExportHtml.HtmlAnnotations reloading plugin ExportHtml.support reloading plugin ExtendedTabSwitcher.ExtendedSwitcher reloading plugin Extract Text to File.extract_to_file reloading plugin File History.file_history reloading plugin File Rename.file_rename reloading plugin FileDiffs.file_diffs reloading plugin FindKeyConflicts.find_key_conflicts reloading plugin FuzzyFileNav.fuzzy_file_nav reloading plugin FuzzyFileNav.multiconf reloading plugin FuzzyFileNav.notify reloading plugin Glue.__init__ reloading plugin Glue.Glue reloading plugin Glue.GlueBrowser reloading plugin Glue.GlueCmds reloading plugin Glue.GlueIO reloading plugin Glue.GlueSidebarOpener reloading plugin Google Spell Check.google-spell-check reloading plugin Highlight Build Errors.HighlightBuildErrors reloading plugin HighlightWords.HighlightWords reloading plugin Horizontal Scroll.scroll_width reloading plugin Hungry Backspace.hungry_backspace reloading plugin Increment Selection.IncrementSelection reloading plugin Incrementor.Incrementor reloading plugin Indent Size.IndentSizeCommand reloading plugin Invert Selection.invertselection reloading plugin LaTeX Word Count.WordCount reloading plugin LESS.less_completions reloading plugin Line Endings Unify.LineEndingsUnify reloading plugin Markdown Preview.helper reloading plugin Markdown Preview.markdown_settings reloading plugin Markdown Preview.markdown_wrapper reloading plugin Markdown Preview.MarkdownPreview reloading plugin MaxPane.layouts_doc reloading plugin MaxPane.max_pane reloading plugin MoveText.move_text reloading plugin MultiEditUtils.MultiEditUtils reloading plugin MultiEditUtils.selection_fields reloading plugin Non Text Files.NonTextFiles reloading plugin Number King.king reloading plugin Origami.origami reloading plugin OverrideAudit.override_audit reloading plugin OverrideAudit.settings_proxy reloading plugin PackageResourceViewer.package_resource_viewer reloading plugin PackageResourceViewer.package_resources reloading plugin Path Translator.PathTranslator reloading plugin RecentActiveFiles.RecentActiveFiles reloading plugin ReIndent.ReIndent reloading plugin ScopeHunter.scope_hunter reloading plugin ScopeHunter.scope_hunter_notify reloading plugin ScopeHunter.support reloading plugin StickySearch.StickySearch reloading plugin Sync View Scroll.syncscroll reloading plugin TabsExtra.support reloading plugin TabsExtra.tab_menu reloading plugin TabsExtra.tab_sort_helper reloading plugin TabsExtra.tabs_extra reloading plugin Text Pastry.text_pastry reloading plugin Text Pastry.text_pastry_addons reloading plugin Text Pastry.text_pastry_clipboard reloading plugin Text Pastry.text_pastry_selection reloading plugin Trimmer.Trimmer reloading plugin Whitespace.Whitespace reloading plugin Wrap Plus.wrap_plus reloading plugin Active View Jump Back.active_view_jump_back reloading plugin Advanced CSV.csvplugin === NumPy disabled, using TinyNumPy instead === To enable cell evaluation using the full NumPy, download NumPy from: https://pypi.python.org/pypi/numpy and install it into Sublime Text's Packages directory. For information on the features and limitations of TinyNumPy, visit: https://github.com/wadetb/tinynumpy ====================== reloading plugin amxmodx.AMXXEditor reloading plugin ANSIescape.ansi reloading plugin AutoFileName.autofilename reloading plugin AutoFileName.getimageinfo reloading plugin BetterFindBuffer.find_results reloading plugin BufferScroll.BufferScroll reloading plugin Clear Cursors Carets.clear_cursors_carets reloading plugin Clipboard Scope Copy.clipboard_scope_copy reloading plugin ConvertToUTF8.ConvertToUTF8 reloading plugin Default Syntax.default_syntax reloading plugin DocBlockr.test_runner reloading plugin Fix Project Switch Restart Bug.fix_project_switch_restart_bug reloading plugin Fix Selection After Indent.fix_selection_after_indent reloading plugin Fixed Selections Clear.fixed_clear_selections reloading plugin Force Rewrite Sublime Settings.force_rewrite_sublime_settings reloading plugin FuzzyFilePath.command_goto_file reloading plugin FuzzyFilePath.command_insert_path reloading plugin FuzzyFilePath.command_rebuild_cache reloading plugin FuzzyFilePath.command_replace_region reloading plugin FuzzyFilePath.command_show_context reloading plugin FuzzyFilePath.command_show_current_settings reloading plugin FuzzyFilePath.command_show_info reloading plugin FuzzyFilePath.completion reloading plugin FuzzyFilePath.controller reloading plugin FuzzyFilePath.current_state reloading plugin FuzzyFilePath.expression reloading plugin FuzzyFilePath.ProjectListener reloading plugin FuzzyFilePath.query reloading plugin FuzzyFilePath.QueryCompletionListener reloading plugin FuzzyFilePath.TestRunner reloading plugin FuzzyFilePath.ViewListener reloading plugin GotoLastEditEnhanced.commands reloading plugin Indent and braces.indent-and-braces reloading plugin Jedi - Python autocompletion.__init__ reloading plugin LaTeXTools.01_reload_submodules reloading plugin LaTeXTools.02_temp_file_cleanup reloading plugin LaTeXTools.03_reset_phantoms reloading plugin LaTeXTools.auto_label reloading plugin LaTeXTools.biblatex_crossref_completions reloading plugin LaTeXTools.biblatex_field_name_completions reloading plugin LaTeXTools.biblatex_name_completions reloading plugin LaTeXTools.biblatex_snippet_completions reloading plugin LaTeXTools.biblatex_syntax_listener reloading plugin LaTeXTools.change_environment reloading plugin LaTeXTools.create_mousemap reloading plugin LaTeXTools.delete_temp_files reloading plugin LaTeXTools.detect_spellcheck reloading plugin LaTeXTools.getTeXRoot reloading plugin LaTeXTools.jumpto_anywhere reloading plugin LaTeXTools.jumpto_tex_file reloading plugin LaTeXTools.jumpToPDF reloading plugin LaTeXTools.kpsewhich reloading plugin LaTeXTools.latex_cite_completions reloading plugin LaTeXTools.latex_cwl_completions reloading plugin LaTeXTools.latex_directive_completions reloading plugin LaTeXTools.latex_env_completions reloading plugin LaTeXTools.latex_fill_all reloading plugin LaTeXTools.latex_glossary_completions reloading plugin LaTeXTools.latex_input_completions reloading plugin LaTeXTools.latex_installed_packages reloading plugin LaTeXTools.latex_own_command_completions reloading plugin LaTeXTools.latex_ref_completions reloading plugin LaTeXTools.latexCommand reloading plugin LaTeXTools.latexDocumentationViewer reloading plugin LaTeXTools.latexEnvCloser reloading plugin LaTeXTools.latexEnvironment reloading plugin LaTeXTools.latextools_cache_listener reloading plugin LaTeXTools.latextools_default_settings reloading plugin LaTeXTools.latextools_plugin reloading plugin LaTeXTools.latextools_sublime_version_listener reloading plugin LaTeXTools.makePDF reloading plugin LaTeXTools.migrate reloading plugin LaTeXTools.open_detexify reloading plugin LaTeXTools.parseTeXlog reloading plugin LaTeXTools.reveal_folders reloading plugin LaTeXTools.search_commands reloading plugin LaTeXTools.smart_paste reloading plugin LaTeXTools.system_check reloading plugin LaTeXTools.texcount reloading plugin LaTeXTools.texMacro reloading plugin LaTeXTools.texSections reloading plugin LaTeXTools.texSyntaxListener reloading plugin LaTeXTools.toc_quickpanel reloading plugin LaTeXTools.toggle_settings reloading plugin Markdown to BB Code Converter.MarkdownToBBCode reloading plugin Open Auto Completion.open_auto_completion reloading plugin Override Unpacked Packages.override_unpacked_packages reloading plugin Overwrite Commit Completion.overwrite_commit_completion reloading plugin Package Control.1_reloader reloading plugin Package Control.2_bootstrap reloading plugin Package Control.Package Control reloading plugin package_setting_context.__init__ reloading plugin PackageDev.build_sys_dev reloading plugin PackageDev.commands_file_dev reloading plugin PackageDev.completions_dev reloading plugin PackageDev.file_conversion reloading plugin PackageDev.settings_dev reloading plugin PackageDev.snippet_dev reloading plugin PackageDev.syntax-test_dev reloading plugin PackageDev.syntax_dev reloading plugin PackageDev.syntax_dev_legacy reloading plugin PackageDev.xml_hidden_extensions_hotfix reloading plugin PowerCursors.power_cursors reloading plugin Preferences Editor.Preferences Editor reloading plugin Random Everything.__init__ reloading plugin Random Everything.random reloading plugin Side-by-Side Settings.sxs_settings reloading plugin SideBarEnhancements.SideBar reloading plugin SideBarEnhancements.SideBarAPI reloading plugin SideBarEnhancements.SideBarDefaultDisable reloading plugin Sublime Text Studio.sublime_text_studio reloading plugin Sublime Text Studio.synced_side_bar_watcher reloading plugin SublimeREPL.__init__ reloading plugin SublimeREPL.completions reloading plugin SublimeREPL.lang_integration reloading plugin SublimeREPL.run_existing_command reloading plugin SublimeREPL.sublimerepl reloading plugin SublimeREPL.sublimerepl_build_system_hack reloading plugin SublimeREPL.text_transfer reloading plugin Terminal.Terminal reloading plugin View Settings Freely.view_settings_freely reloading plugin Wrap Plus.wrap_lines_enhancement plugins loaded OverrideAudit: Sublime version is unchanged; skipping automatic report Loaded LaTeXTools plugins ['new_bibliography', 'traditional_bibliography'] from path D:\User\Dropbox\Applications\SoftwareVersioning\SublimeText\Data\Packages\LaTeXTools\bibliography_plugins syncScroll starting Loading LaTeXTools plugins... Emmet: No need to update PyV8 Loaded LaTeXTools plugins ['pdf_builder'] from path D:\User\Dropbox\Applications\SoftwareVersioning\SublimeText\Data\Packages\LaTeXTools\builders\pdfBuilder.py Loaded LaTeXTools plugins ['simple_builder', 'traditional_builder', 'basic_builder', 'script_builder'] from path D:\User\Dropbox\Applications\SoftwareVersioning\SublimeText\Data\Packages\LaTeXTools\builders Loaded LaTeXTools plugins ['base_viewer'] from path D:\User\Dropbox\Applications\SoftwareVersioning\SublimeText\Data\Packages\LaTeXTools\viewers\base_viewer.py Loaded LaTeXTools plugins ['skim_viewer', 'preview_viewer', 'command_viewer', 'okular_viewer', 'sumatra_viewer', 'evince_viewer', 'zathura_viewer'] from path D:\User\Dropbox\Applications\SoftwareVersioning\SublimeText\Data\Packages\LaTeXTools\viewers Loaded LaTeXTools plugins [] from path D:\User\Dropbox\Applications\SoftwareVersioning\SublimeText\Data\Packages\LaTeXTools\bibliography_plugins worker 14756 appears stuck while processing file /F/VSCodeStudio/extensions/felixfbecker.php-intellisense-1.4.3/vendor/webmozart/assert/README.md, killing process Package Control: No updated packages worker 14756 appears stuck while processing file /F/VSCodeStudio/extensions/felixfbecker.php-intellisense-1.4.3/vendor/webmozart/assert/README.md, killing process FuzzyFilePath cached 12588 files in F:/VSCodeStudio worker 14756 appears stuck while processing file /F/VSCodeStudio/extensions/felixfbecker.php-intellisense-1.4.3/vendor/webmozart/assert/README.md, killing process FuzzyFilePath cached 12588 files in F:/VSCodeStudio worker 13300 appears stuck while processing file /F/VSCodeStudio/extensions/felixfbecker.php-intellisense-1.4.3/vendor/webmozart/assert/README.md, killing process worker 14756 appears stuck while processing file /F/VSCodeStudio/extensions/felixfbecker.php-intellisense-1.4.3/vendor/webmozart/assert/README.md, killing process worker 13300 appears stuck while processing file /F/VSCodeStudio/extensions/felixfbecker.php-intellisense-1.4.3/vendor/webmozart/assert/README.md, killing process worker 14756 appears stuck while processing file /F/VSCodeStudio/extensions/felixfbecker.php-intellisense-1.4.3/vendor/webmozart/assert/README.md, killing process worker 13300 appears stuck while processing file /F/VSCodeStudio/extensions/felixfbecker.php-intellisense-1.4.3/vendor/webmozart/assert/README.md, killing process worker 14756 appears stuck while processing file /F/VSCodeStudio/extensions/felixfbecker.php-intellisense-1.4.3/vendor/webmozart/assert/README.md, killing process worker 13300 appears stuck while processing file /F/VSCodeStudio/extensions/felixfbecker.php-intellisense-1.4.3/vendor/webmozart/assert/README.md, killing process worker 14756 appears stuck while processing file /F/VSCodeStudio/extensions/felixfbecker.php-intellisense-1.4.3/vendor/webmozart/assert/README.md, killing process ```

### Environment * Operating system and version: * Windows 10 build 15063 * Mac OS ... * Linux ... * Monitor: * Resolution 1920x1080 * `dpi_scale` used in ST 1.0 * Sublime Text: * Build 3141 * 32 bit

[evandrocoan]

I find out how to reproduce the problem.

1. Install the package: MarkdownLight.zip

2. Unzip the project F.zip on your C:\ drive.

3. Open the project C:\F\VSCodeStudio\VSCodeStudio.sublime-project

And it will get stuck about 84%.

Sublime Text vanilla install also cannot open the README.md file inside the F.zip file while the MarkdownLight.zip syntax is installed.

[wbond]

Most likely the third-party syntax you have installed has regexes that cause catastrophic backtracking in that markdown file, causing Oniguruma to get stuck trying to run the regex.

It seems the bug is that the indexer processes are not properly dying on your machine. I will have to look into this to see if it is general problem, or something config-specific.

[deathaxe]

I recently saw the indexer stuck while it tried to index Mardown files in ST 3184 x64 on Windows.

The files itself were quite small (LICENSE.md). It looked to me like ST opened the markdown the very first time (vanilla install) and hence had to create the syntax definition cache as well, which seems to take longer than the timeout defined to detect the stuck.

[BenjaminSchaaf]

This should be fixed since build 4135.

[evandrocoan]

I can confirm, using the files and reproduction steps on https://github.com/sublimehq/sublime_text/issues/1824#issuecomment-316786643 I could reproduce it on build 3141, but not on build 4160. On the latest version, it shows these errors on the console instead of hanging and not opening the README.md:

error in regex: retry-limit-in-match over regex: ^\s*+([-_*])\s*+(?:\1\s*){2,}+$
error in regex: retry-limit-in-match over regex: ^\s*+([-_*])\s*+(?:\1\s*){2,}+$
error in regex: retry-limit-in-match over regex: ^\s*+([-_*])\s*+(?:\1\s*){2,}+$
error in regex: retry-limit-in-match over regex: ^\s*+([-_*])\s*+(?:\1\s*){2,}+$
error in regex: retry-limit-in-match over regex: ^\s*+([-_*])\s*+(?:\1\s*){2,}+$
error in regex: retry-limit-in-match over regex: ^\s*+([-_*])\s*+(?:\1\s*){2,}+$
error in regex: retry-limit-in-match over regex: ^\s*+([-_*])\s*+(?:\1\s*){2,}+$

[deathaxe]

This error is the result of a malformed pattern in MarkdownLite syntax. The + sign in \s*+ and {2,}+are invalid or at least not supported by ST's syntax engine.

[evandrocoan]

Thanks for explaining. I looked now at https://regex101.com/, and it looks like some regex feature: