Poor parsing performance

[brucehsu]

Example:

Thread.new do
    lines = IO.readlines '../words_1.txt'
    cnt = {'cat'=>0, 'dog'=>0, 'cow'=>0, 'tiger'=>0, 'lion'=>0, 'wolf'=>0, 'fish'=>0}
    lines.each do |l|
        words = l.split(' ')
        words.each do |word|
            cnt[word] = cnt[word] + 1
        end
    end
    p cnt
end

which is significantly slower than following example:

lines = IO.readlines '../words_1.txt'
cnt = {'cat'=>0, 'dog'=>0, 'cow'=>0, 'tiger'=>0, 'lion'=>0, 'wolf'=>0, 'fish'=>0}
lines.each do |l|
    words = l.split(' ')
    words.each do |word|
        cnt[word] = cnt[word] + 1
    end
end
p cnt

[brucehsu]

It seems like the parser performs poorly on nested blocks. Initial investigation shows it restarts from the last successful syntax rule upon failed rule.

[brucehsu]

Might be worthy to investigate goyacc and new lexer from sourcegraph. https://sourcegraph.com/blog/multi-language-lexer-and-scanner-for-go