search

ikatyang / tree-sitter-markdown

Markdown grammar for tree-sitter
https://ikatyang.github.io/tree-sitter-markdown
MIT License
183 stars 29 forks source link

fix oom #39

Open dimbleby opened 3 years ago

dimbleby commented 3 years ago

This addresses - but does not exactly fix - the out-of-memory reported in #32.

I have removed a block of code which, I suppose, is somehow mis-handling this case. This isn't a clean fix, we now go on to hit the assertion that follows: but with TREE_SITTER_MARKDOWN_AVOID_CRASH we catch this error and seem to recover.

The input:

[foo](*https://example.com
[foo](*https://example.com

now parses as:

(document [0, 0] - [1, 26]
  (paragraph [0, 0] - [1, 26]
    (link [0, 0] - [0, 5]
      (link_text [0, 1] - [0, 4]
        (text [0, 1] - [0, 4])))
    (text [0, 5] - [0, 7])
    (uri_autolink [0, 7] - [0, 26]
      (text [0, 7] - [0, 26]))
    (soft_line_break [0, 26] - [1, 0])
    (link [1, 0] - [1, 5]
      (link_text [1, 1] - [1, 4]
        (text [1, 1] - [1, 4])))
    (text [1, 5] - [1, 7])
    (uri_autolink [1, 7] - [1, 26]
      (text [1, 7] - [1, 26]))))

I was expecting an error! But this doesn't look stupid, and certainly is a lot better than looping and consuming all available RAM.

dimbleby commented 3 years ago

@ikatyang I see that this project has been quiet for a few months. Is it still alive?

With my fixes / workarounds here and at #36, the parser is currently holding strong against the fuzzer. (It has only been an hour, so too soon to declare victory - but that's definite progress!)

If that continues to be the case, I imagine that eg neovim might want to revive their interest in using this. So it would be good to get a sign as to whether this is a going concern or not.

NullVoxPopuli commented 3 years ago

I do a lot of markdown editing and would love to be able to have syntax highlighting!!

pappasam commented 3 years ago

@dimbleby after 17 days, does it still appear to be holding strong? If yes, @ikatyang , it would be nice if you could consider these PR's. We're all pretty excited about the efforts being pushed forward by @dimbleby

dimbleby commented 3 years ago

I stopped after a few hours. Certainly that's too soon to promise that there's nothing left to find: but it's considerably longer than the fuzzer has previously taken to find issues.

Lunarequest commented 3 years ago

it looks like the project maintainer @ikatyang isn't active anymore. Maybe we can fork and include prs?

Hmgov commented 2 years ago

IMG_20220502_112211

Default Packages

2017_OFCCP_Pre-Award_Ref_Guide_Inter_5-17-17-Final_508COMP.pdf

Hmgov commented 2 years ago

the finel todays github imjiniaring world war ll