crash when syntax file is not complete?

[xirishpvp]

Description of the problem or steps to reproduce

open a syntax file with micro, then vsplit and open a test.lua file in the same directory as the syntax file. the bug happens even when you only open test.lua so im not sure if its regex crashing it or a bug.

example:

filetype: lua

detect:
    filename: "\\.lua&"

rules:
    - identifier.var:
        start: ""
        end: ""
        rules: []

test = require("hello.lua")

a = 5

if

Specifications

Commit hash: 68d88b57 OS: arch linux Terminal: konsole

[dmaluka]

I can reproduce this on the newest master. It crashes with a stack overflow due to an infinite recursion in the highlighter:

runtime: goroutine stack exceeds 1000000000-byte limit
runtime: sp=0xc020980368 stack=[0xc020980000, 0xc040980000]
fatal error: stack overflow

runtime stack:
runtime.throw({0x9ea364?, 0x7fdbaaffccf0?})
    runtime/panic.go:1077 +0x5c fp=0x7fdbaaffcca0 sp=0x7fdbaaffcc70 pc=0x43c07c
runtime.newstack()
    runtime/stack.go:1107 +0x5ac fp=0x7fdbaaffce50 sp=0x7fdbaaffcca0 pc=0x45674c
runtime.morestack()
    runtime/asm_amd64.s:593 +0x8f fp=0x7fdbaaffce58 sp=0x7fdbaaffce50 pc=0x46dfaf

goroutine 12 [running]:
runtime.mallocgc(0x10?, 0x0?, 0x0?)
    runtime/malloc.go:952 +0x825 fp=0xc020980378 sp=0xc020980370 pc=0x412865
runtime.growslice(0xf9c560, 0xc000903080?, 0xb36c38?, 0xc0009030e8?, 0x0?)
    runtime/slice.go:266 +0x4cf fp=0xc0209803e8 sp=0xc020980378 pc=0x45468f
regexp.(*Regexp).backtrack(0xc000472000, {0xc00021a3c0, 0x1b, 0x1c}, {0x0, 0x0}, 0x0, 0xc000903080?, {0xf9c560, 0x0, ...})
    regexp/backtrack.go:364 +0x376 fp=0xc020980468 sp=0xc0209803e8 pc=0x574cd6
regexp.(*Regexp).doExecute(0xc0031f82a0?, {0x0?, 0x0}, {0xc00021a3c0, 0x1b, 0x1c}, {0x0, 0x0}, 0x1b?, 0x2, ...)
    regexp/exec.go:535 +0x27f fp=0xc020980518 sp=0xc020980468 pc=0x576d7f
regexp.(*Regexp).FindIndex(...)
    regexp/regexp.go:838
github.com/zyedidia/micro/v2/pkg/highlight.findIndex(0xc00021a3c0?, 0x1?, {0xc00021a3c0, 0x1b, 0x1c})
    github.com/zyedidia/micro/v2/pkg/highlight/highlighter.go:112 +0xfd fp=0xc0209805e0 sp=0xc020980518 pc=0x85949d
github.com/zyedidia/micro/v2/pkg/highlight.(*Highlighter).highlightEmptyRegion(0xc000805c40, 0x1b?, 0x0, 0x1, 0x1c?, {0xc00021a3c0, 0x1b, 0x1c}, 0x1)
    github.com/zyedidia/micro/v2/pkg/highlight/highlighter.go:235 +0x187 fp=0xc0209806c8 sp=0xc0209805e0 pc=0x85a547
github.com/zyedidia/micro/v2/pkg/highlight.(*Highlighter).highlightRegion(0xc000805c40, 0x1c?, 0x0, 0x1, 0xc00021a3c0?, {0xc00021a3c0, 0x1b, 0x1c}, 0xc00060d3e0, 0x1)
    github.com/zyedidia/micro/v2/pkg/highlight/highlighter.go:206 +0x5ed fp=0xc0209807c0 sp=0xc0209806c8 pc=0x859f4d
github.com/zyedidia/micro/v2/pkg/highlight.(*Highlighter).highlightEmptyRegion(0xc000805c40, 0x1b?, 0x0, 0x1, 0x1c?, {0xc00021a3c0, 0x1b, 0x1c}, 0x1)
    github.com/zyedidia/micro/v2/pkg/highlight/highlighter.go:248 +0x352 fp=0xc0209808a8 sp=0xc0209807c0 pc=0x85a712
github.com/zyedidia/micro/v2/pkg/highlight.(*Highlighter).highlightRegion(0xc000805c40, 0x1c?, 0x0, 0x1, 0xc00021a3c0?, {0xc00021a3c0, 0x1b, 0x1c}, 0xc00060d3e0, 0x1)
    github.com/zyedidia/micro/v2/pkg/highlight/highlighter.go:206 +0x5ed fp=0xc0209809a0 sp=0xc0209808a8 pc=0x859f4d
github.com/zyedidia/micro/v2/pkg/highlight.(*Highlighter).highlightEmptyRegion(0xc000805c40, 0x1b?, 0x0, 0x1, 0x1c?, {0xc00021a3c0, 0x1b, 0x1c}, 0x1)
    github.com/zyedidia/micro/v2/pkg/highlight/highlighter.go:248 +0x352 fp=0xc020980a88 sp=0xc0209809a0 pc=0x85a712
github.com/zyedidia/micro/v2/pkg/highlight.(*Highlighter).highlightRegion(0xc000805c40, 0x1c?, 0x0, 0x1, 0xc00021a3c0?, {0xc00021a3c0, 0x1b, 0x1c}, 0xc00060d3e0, 0x1)
    github.com/zyedidia/micro/v2/pkg/highlight/highlighter.go:206 +0x5ed fp=0xc020980b80 sp=0xc020980a88 pc=0x859f4d
github.com/zyedidia/micro/v2/pkg/highlight.(*Highlighter).highlightEmptyRegion(0xc000805c40, 0x1b?, 0x0, 0x1, 0x1c?, {0xc00021a3c0, 0x1b, 0x1c}, 0x1)
    github.com/zyedidia/micro/v2/pkg/highlight/highlighter.go:248 +0x352 fp=0xc020980c68 sp=0xc020980b80 pc=0x85a712
...

The good news is that it is not observed with @JoeKar's PR #3127. (This PR is still under review, but hopefully we will merge it sooner than later.)

With #3127 it is highlighted in a bizarre way:

@JoeKar FYI. Is this the expected highlighting in this case? Looks like every odd character in a line is highlighted as identifier.var and every even character is not highlighted. Perhaps, instead, every character be highlighted as identifier.var in this case, or perhaps no characters should be highlighted at all? (Or perhaps it doesn't matter much, since this is a pathological case anyway?)

EDIT: if I replace this "empty region" definition with a simple empty regex pattern:

    - identifier.var: ""

then no characters are highlighted. So it seems that at least "for consistency" it's better not to highlight any characters in the "empty region" case either?

[JoeKar]

@JoeKar FYI. Is this the expected highlighting in this case? [...] or perhaps no characters should be highlighted at all?

...in combination with...

EDIT: if I replace this "empty region" definition with a simple empty regex pattern: So it seems that at least "for consistency" it's better not to highlight any characters in the "empty region" case either?

...brings me to the same conclusion. Thanks a lot, the empty pattern test helps to do at least a consistent decision. With a start and end of "" nothing really useful can be identified. Currently I only overthink, if scenarios could exist, in which at least one of both could be empty, but I've doubts.

[Andriamanitra]

I think empty start and end should give an error. Silently ignoring the issue would be almost worse than crashing as you won't even realize you've done a mistake when creating a syntax file.

In fact is there any case where an empty string makes sense in a syntax file? We could point out all such errors when a syntax file is loaded, similar to the errors that pop up when there's an invalid regex (eg. "[") or unexpected type (eg. [] where a regex string was expected) in the yaml.

[JoeKar]

I think empty start and end should give an error. Silently ignoring the issue would be almost worse than crashing as you won't even realize you've done a mistake when creating a syntax file.

Good point! Then we catch that in the parser already and don't need to check it in the highlighter again. :+1: