search

sublimehq / sublime_text

Issue tracker for Sublime Text
https://www.sublimetext.com
807 stars 39 forks source link

Hard lock when using minihtml with Markdown Typescript code blocks #6029

Open trevorsmith opened 1 year ago

trevorsmith commented 1 year ago

Description of the bug

I've been experiencing periodic issues with Sublime Text 4 locking up when showing LSP popups, forcing me to kill and reopen the application. I've narrowed it down to minihtml trying to render the Typescript code block within the Markdown response from the Typescript language server.

It appears to be catastrophic backtracking from a regular expression in the syntax highlighter. Here's a snippet from the stack trace:

  Thread 0x8fc904    DispatchQueue "com.apple.main-thread"(1)    15 samples (1-15)    priority 47 (base 47)    cpu time 1.396s (4.4G cycles, 21.6G instructions, 0.20c/i)
  <process frontmost, thread QoS user interactive (requested user interactive), process unclamped, process received importance donation from WindowServer [164], IO tier 0>
  15  start + 2236 (dyld + 24360) [0x18519bf28] 1-15
    15  main + 8360 (sublime_text + 26456) [0x1028f2758] 1-15
      15  px_run_event_loop() + 188 (sublime_text + 1572992) [0x102a6c080] 1-15
        15  -[NSApplication run] + 464 (AppKit + 181060) [0x1887e5344] 1-15
          15  -[NSApplication(NSEvent) _nextEventMatchingEventMask:untilDate:inMode:dequeue:] + 716 (AppKit + 229088) [0x1887f0ee0] 1-15
            15  _DPSNextEvent + 636 (AppKit + 232772) [0x1887f1d44] 1-15
              15  _BlockUntilNextEventMatchingListInModeWithFilter + 76 (HIToolbox + 198612) [0x18ee1a7d4] 1-15
                15  ReceiveNextEventCommon + 648 (HIToolbox + 199292) [0x18ee1aa7c] 1-15
                  15  RunCurrentEventLoopInMode + 292 (HIToolbox + 199744) [0x18ee1ac40] 1-15
                    15  CFRunLoopRunSpecific + 612 (CoreFoundation + 513208) [0x1855d04b8] 1-15
                      15  __CFRunLoopRun + 828 (CoreFoundation + 515912) [0x1855d0f48] 1-15
                        15  __CFRunLoopDoSources0 + 244 (CoreFoundation + 521024) [0x1855d2340] 1-15
                          15  __CFRunLoopDoSource0 + 176 (CoreFoundation + 521680) [0x1855d25d0] 1-15
                            15  __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 28 (CoreFoundation + 521788) [0x1855d263c] 1-15
                              15  __NSThreadPerformPerform + 264 (Foundation + 498788) [0x186566c64] 1-15
                                15  -[WorkQueueCallback processItems:] + 128 (sublime_text + 1543836) [0x102a64e9c] 1-15
                                  15  std::__1::__function::__func<api_helper::api_set_timeout(object_id, int)::'lambda'(), std::__1::allocator<api_helper::api_set_timeout(object_id, int)::'lambda'()>, void ()>::operator()() + 28 (sublime_text + 657180) [0x10298c71c] 1-15
                                    15  int send_to_host<int, object_id>(unsigned long, unsigned int, object_id) + 244 (sublime_text + 624420) [0x102984724] 1-15
                                      15  send_helper<int>::wait_for_reply() + 124 (sublime_text + 619724) [0x1029834cc] 1-15
                                        15  handle_protocol_message(unsigned long, string_buffer const&) + 4336 (sublime_text + 526840) [0x10296c9f8] 1-15
                                          15  void decode_message<api_helper, int, unsigned int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, value const&>(unsigned long, unsigned int, int (api_helper::*)(unsigned int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, value const&), binary_input_stream&) + 124 (sublime_text + 559776) [0x102974aa0] 1-15
                                            15  api_helper::api_view_run_command(unsigned int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, value const&) + 264 (sublime_text + 565452) [0x1029760cc] 1-15
                                              15  text_command::run(value const&) + 152 (sublime_text + 3144308) [0x102beba74] 1-15
                                                15  insert_command::run(SP<TextBufferView>, value const&) + 108 (sublime_text + 3181908) [0x102bf4d54] 1-15
                                                  15  translateViewToShowSelectionSet(SP<TextBufferView> const&, SelectionSet const&, unsigned int) + 96 (sublime_text + 2888936) [0x102bad4e8] 1-15
                                                    15  TextBufferView::pointToView(long long) + 24 (sublime_text + 2937708) [0x102bb936c] 1-15
                                                      15  TokenStorage::pointToView(unsigned int, long long) + 52 (sublime_text + 4117876) [0x102cd9574] 1-15
                                                        2   TokenStorage::updateLexerState(long long) + 1788 (sublime_text + 4007380) [0x102cbe5d4] 1-2
                                                          1   lex(lex_working_memory*, LexerState*, u32substring, std::__1::vector<TokenInfo, std::__1::allocator<TokenInfo>>&, long long, backtracking_state*, lexer_state_extractor*) + 8760 (sublime_text + 3946548) [0x102caf834] (running) 1
                                                          1   lex(lex_working_memory*, LexerState*, u32substring, std::__1::vector<TokenInfo, std::__1::allocator<TokenInfo>>&, long long, backtracking_state*, lexer_state_extractor*) + 816 (sublime_text + 3938604) [0x102cad92c] 2
                                                            1   MultiMatcher::search(oniguruma::wcmatch&, sregex::working_memory&, unsigned int, u32substring, u32substring, u32substring&, std::__1::vector<u32substring, std::__1::allocator<u32substring>>&, std::__1::vector<unsigned int, std::__1::allocator<unsigned int>> const&, unsigned int const*) const + 1112 (sublime_text + 3976964) [0x102cb6f04] 2
                                                              1   bool sregex::parallel_search_with_registers<utf32_decoder, unsigned int const*>(sregex::working_memory*, unsigned char const*, char_set const*, sregex::register_list<unsigned int const*>*, unsigned int, unsigned int, unsigned int, unsigned int, unsigned int, unsigned short, unsigned int const*, unsigned int const*, unsigned int const*, unsigned int const*, unsigned int const*, sregex::match_data<unsigned int const*>*) + 3976 (sublime_text + 3696236) [0x102c7266c] (running) 2
                                                        1   TokenStorage::updateLexerState(long long) + 2668 (sublime_text + 4008260) [0x102cbe944] 3
                                                          1   TokenStorage::insertTokens(BucketContainer<tree_list<TokenBuffer, TokenBuffer::token_list_ops>, TokenStorage::TokenBufferFactory>::iterator, long long, TokenInfo const*, unsigned long) + 1044 (sublime_text + 3997380) [0x102cbbec4] 3
                                                            1   void TokenBuffer::insert<Token**>(Token**, Token**, Token**) + 236 (sublime_text + 3999764) [0x102cbc814] (running) 3
                                                        1   TokenStorage::updateLexerState(long long) + 1788 (sublime_text + 4007380) [0x102cbe5d4] 4
                                                          1   lex(lex_working_memory*, LexerState*, u32substring, std::__1::vector<TokenInfo, std::__1::allocator<TokenInfo>>&, long long, backtracking_state*, lexer_state_extractor*) + 816 (sublime_text + 3938604) [0x102cad92c] 4
                                                            1   MultiMatcher::search(oniguruma::wcmatch&, sregex::working_memory&, unsigned int, u32substring, u32substring, u32substring&, std::__1::vector<u32substring, std::__1::allocator<u32substring>>&, std::__1::vector<unsigned int, std::__1::allocator<unsigned int>> const&, unsigned int const*) const + 1112 (sublime_text + 3976964) [0x102cb6f04] 4
                                                              1   bool sregex::parallel_search_with_registers<utf32_decoder, unsigned int const*>(sregex::working_memory*, unsigned char const*, char_set const*, sregex::register_list<unsigned int const*>*, unsigned int, unsigned int, unsigned int, unsigned int, unsigned int, unsigned short, unsigned int const*, unsigned int const*, unsigned int const*, unsigned int const*, unsigned int const*, sregex::match_data<unsigned int const*>*) + 220 (sublime_text + 3692480) [0x102c717c0] (running) 4
                                                        1   TokenStorage::updateLexerState(long long) + 2668 (sublime_text + 4008260) [0x102cbe944] 5
                                                          1   TokenStorage::insertTokens(BucketContainer<tree_list<TokenBuffer, TokenBuffer::token_list_ops>, TokenStorage::TokenBufferFactory>::iterator, long long, TokenInfo const*, unsigned long) + 1044 (sublime_text + 3997380) [0x102cbbec4] 5
                                                            1   void TokenBuffer::insert<Token**>(Token**, Token**, Token**) + 272 (sublime_text + 3999800) [0x102cbc838] (running) 5
                                                        3   TokenStorage::updateLexerState(long long) + 1788 (sublime_text + 4007380) [0x102cbe5d4] 6-8
                                                          3   lex(lex_working_memory*, LexerState*, u32substring, std::__1::vector<TokenInfo, std::__1::allocator<TokenInfo>>&, long long, backtracking_state*, lexer_state_extractor*) + 816 (sublime_text + 3938604) [0x102cad92c] 6-8
                                                            3   MultiMatcher::search(oniguruma::wcmatch&, sregex::working_memory&, unsigned int, u32substring, u32substring, u32substring&, std::__1::vector<u32substring, std::__1::allocator<u32substring>>&, std::__1::vector<unsigned int, std::__1::allocator<unsigned int>> const&, unsigned int const*) const + 1112 (sublime_text + 3976964) [0x102cb6f04] 6-8
                                                              1   bool sregex::parallel_search_with_registers<utf32_decoder, unsigned int const*>(sregex::working_memory*, unsigned char const*, char_set const*, sregex::register_list<unsigned int const*>*, unsigned int, unsigned int, unsigned int, unsigned int, unsigned int, unsigned short, unsigned int const*, unsigned int const*, unsigned int const*, unsigned int const*, unsigned int const*, sregex::match_data<unsigned int const*>*) + 3976 (sublime_text + 3696236) [0x102c7266c] (running) 6
                                                              1   bool sregex::parallel_search_with_registers<utf32_decoder, unsigned int const*>(sregex::working_memory*, unsigned char const*, char_set const*, sregex::register_list<unsigned int const*>*, unsigned int, unsigned int, unsigned int, unsigned int, unsigned int, unsigned short, unsigned int const*, unsigned int const*, unsigned int const*, unsigned int const*, unsigned int const*, sregex::match_data<unsigned int const*>*) + 4648 (sublime_text + 3696908) [0x102c7290c] (running) 7
                                                              1   bool sregex::parallel_search_with_registers<utf32_decoder, unsigned int const*>(sregex::working_memory*, unsigned char const*, char_set const*, sregex::register_list<unsigned int const*>*, unsigned int, unsigned int, unsigned int, unsigned int, unsigned int, unsigned short, unsigned int const*, unsigned int const*, unsigned int const*, unsigned int const*, unsigned int const*, sregex::match_data<unsigned int const*>*) + 1636 (sublime_text + 3693896) [0x102c71d48] (running) 8
                                                        1   TokenStorage::updateLexerState(long long) + 2668 (sublime_text + 4008260) [0x102cbe944] 9
                                                          1   TokenStorage::insertTokens(BucketContainer<tree_list<TokenBuffer, TokenBuffer::token_list_ops>, TokenStorage::TokenBufferFactory>::iterator, long long, TokenInfo const*, unsigned long) + 1044 (sublime_text + 3997380) [0x102cbbec4] 9
                                                            1   TokenBuffer::erase(Token**, Token**) + 136 (sublime_text + 3981700) [0x102cb8184] (running) 9
                                                        5   TokenStorage::updateLexerState(long long) + 1788 (sublime_text + 4007380) [0x102cbe5d4] 10-14
                                                          5   lex(lex_working_memory*, LexerState*, u32substring, std::__1::vector<TokenInfo, std::__1::allocator<TokenInfo>>&, long long, backtracking_state*, lexer_state_extractor*) + 816 (sublime_text + 3938604) [0x102cad92c] 10-14
                                                            1   std::__1::enable_if<__is_cpp17_forward_iterator<u32substring*>::value && is_constructible<u32substring, std::__1::iterator_traits<u32substring*>::reference>::value, void>::type std::__1::vector<u32substring, std::__1::allocator<u32substring>>::assign<u32substring*>(u32substring*, u32substring*) + 300 (sublime_text + 2774336) [0x102b91540] (running) 10
                                                            4   MultiMatcher::search(oniguruma::wcmatch&, sregex::working_memory&, unsigned int, u32substring, u32substring, u32substring&, std::__1::vector<u32substring, std::__1::allocator<u32substring>>&, std::__1::vector<unsigned int, std::__1::allocator<unsigned int>> const&, unsigned int const*) const + 1112 (sublime_text + 3976964) [0x102cb6f04] 11-14
                                                              1   bool sregex::parallel_search_with_registers<utf32_decoder, unsigned int const*>(sregex::working_memory*, unsigned char const*, char_set const*, sregex::register_list<unsigned int const*>*, unsigned int, unsigned int, unsigned int, unsigned int, unsigned int, unsigned short, unsigned int const*, unsigned int const*, unsigned int const*, unsigned int const*, unsigned int const*, sregex::match_data<unsigned int const*>*) + 1068 (sublime_text + 3693328) [0x102c71b10] (running) 11
                                                              1   bool sregex::parallel_search_with_registers<utf32_decoder, unsigned int const*>(sregex::working_memory*, unsigned char const*, char_set const*, sregex::register_list<unsigned int const*>*, unsigned int, unsigned int, unsigned int, unsigned int, unsigned int, unsigned short, unsigned int const*, unsigned int const*, unsigned int const*, unsigned int const*, unsigned int const*, sregex::match_data<unsigned int const*>*) + 1240 (sublime_text + 3693500) [0x102c71bbc] (running) 12
                                                              1   bool sregex::parallel_search_with_registers<utf32_decoder, unsigned int const*>(sregex::working_memory*, unsigned char const*, char_set const*, sregex::register_list<unsigned int const*>*, unsigned int, unsigned int, unsigned int, unsigned int, unsigned int, unsigned short, unsigned int const*, unsigned int const*, unsigned int const*, unsigned int const*, unsigned int const*, sregex::match_data<unsigned int const*>*) + 3976 (sublime_text + 3696236) [0x102c7266c] (running) 13
                                                              1   bool sregex::parallel_search_with_registers<utf32_decoder, unsigned int const*>(sregex::working_memory*, unsigned char const*, char_set const*, sregex::register_list<unsigned int const*>*, unsigned int, unsigned int, unsigned int, unsigned int, unsigned int, unsigned short, unsigned int const*, unsigned int const*, unsigned int const*, unsigned int const*, unsigned int const*, sregex::match_data<unsigned int const*>*) + 3956 (sublime_text + 3696216) [0x102c72658] (running) 14
                                                        1   TokenStorage::updateLexerState(long long) + 2668 (sublime_text + 4008260) [0x102cbe944] 15
                                                          1   TokenStorage::insertTokens(BucketContainer<tree_list<TokenBuffer, TokenBuffer::token_list_ops>, TokenStorage::TokenBufferFactory>::iterator, long long, TokenInfo const*, unsigned long) + 872 (sublime_text + 3997208) [0x102cbbe18] 15
                                                            1   TokenStorage::createToken(TokenInfo const&, bool) + 308 (sublime_text + 3997728) [0x102cbc020] 15
                                                              1   mx3_hash_bytes(void const*, unsigned long, unsigned long long) + 120 (sublime_text + 933424) [0x1029cfe30] (running) 15

Here is the code that causes the hang:

allowed_formats = FORMAT_MARKED_STRING | FORMAT_MARKUP_CONTENT
content = {'value': '\n```typescript\ntype Foo = (({ children }: {\n    children: any;\n}) =>\n```\n', 'kind': 'markdown'}
minihtml(self.view, content, allowed_formats, None)

The => at the end of the content string triggers the catastrophic backtracking.

I've also run it with isolated FORMAT_MARKED_STRING and FORMAT_MARKUP_CONTENT flags, and both trigger a freeze. I also explored possible undocumented flags (1, 8 and 16) which don't freeze but also don't output anything.

Here is the complete content returned by tssserver:

content = {'value': '\n```typescript\ntype Foo = (({ children }: {\n    children: any;\n}) => React.JSX.Element) | (({ children }: {\n    children: React.ReactNode;\n}) => React.JSX.Element) | (({ children }: {\n    children: React.ReactNode;\n}) => React.JSX.Element) | ... 63 more ... | React.LazyExoticComponent<...>\n```\n', 'kind': 'markdown'}

Steps to reproduce

  1. Create an ST plugin.
  2. Add the following imports: 
    from .core.views import FORMAT_MARKED_STRING
from .core.views import FORMAT_MARKUP_CONTENT
from .core.views import minihtml
  3. Add this to a function within the plugin: 
    allowed_formats = FORMAT_MARKED_STRING | FORMAT_MARKUP_CONTENT
content = {'value': '\n```typescript\ntype Foo = (({ children }: {\n    children: any;\n}) =>\n```\n', 'kind': 'markdown'}
minihtml(self.view, content, allowed_formats, None)
  4. Trigger the function.

Expected behavior

Sublime Text should not lock up.

Possible solutions:

Actual behavior

Sublime Text completely freezes forcing it to be killed.

Sublime Text build number

4143

Operating system & version

macOS Ventura 13.4

(Linux) Desktop environment and/or window manager

No response

Additional information

No response

OpenGL context information

OpenGL Context Information:
  GL API Version: 4.1 Metal - 83.1
  GLSL Version: 4.10
  Vendor: Apple
  Renderer: Apple M1 Max
deathaxe commented 1 year ago

Recent dev builds have received several backtracking bugfixes which have been known to cause lockups especially in TypeScript. Does it still happen with ST4150?