purpleidea
commented
6 years ago Confirmed, this must be a bug. Here's a simple reproducer:
/[ \t\n]/ { /* skip over whitespace */ }
/[a-z]([a-z0-9:]*[a-z0-9]+)?/
{
s := yylex.Text()
fmt.Printf("TEXT: %s\n", s)
}
/#[^\n]*/
{
s := yylex.Text()
fmt.Printf("COMMENT: %s\n", s)
}
/./ {
s := yylex.Text()
fmt.Printf("ERROR: %s\n", s)
}
//
package main
import ("fmt";"os")
type yySymType struct { l, c int }
func main() {
v := new(yySymType)
NewLexer(os.Stdin).Lex(v)
//fmt.Printf("%d %d\n", v.l, v.c)
}Put that code in bug.nex, then run nex bug.nex
Then make an input file to test. I used:
# comment 1
hello42:
# comment 2
a
blah:42xThen I run:
cat input | go run bug.nn.go
and the output is:
COMMENT: # comment 1
TEXT: hello42:
COMMENT: # comment 2
TEXT: a
TEXT: blah:42xAs you can see it matches the hello42: string, which should NOT match. To prove it, here's that regexp in pure golang:
https://play.golang.org/p/0xbSLawtmgw
package main
import (
"fmt"
"regexp"
)
func main() {
// Compile the expression once, usually at init time.
// Use raw strings to avoid having to quote the backslashes.
var validID = regexp.MustCompile(`[a-z]([a-z0-9:]*[a-z0-9]+)?`)
fmt.Println(validID.FindString("a")) // true
fmt.Println(validID.FindString("hello")) // true
fmt.Println(validID.FindString("hello42")) // true
fmt.Println(validID.FindString("hello42:")) // false
fmt.Println(validID.FindString("hello42@")) // false
fmt.Println(validID.FindString("hello42:x")) // true
}
So there's a very serious bug in the nex lexer. Understanding the code base is a bit more difficult than I was hoping, so with any chance, someone can take a look at this. @blynn @wolfkdy @drewwells
Thanks
blynn
I have the following regex:
/[a-z]([a-z0-9:]*[a-z0-9]+)?/This is intended to match:
gorhey42orhey:therebut NOT:blah:trailing colons should not be allowed. Yet it does.I need more extensive testing, but AFAIK this is pretty serious matching bug!
I also tested
/[a-z]+([a-z0-9:]*[a-z0-9]+)?/Which has the same problem :/
Digging deeper... Help welcome =D