z7zmey / php-parser

PHP parser written in Go
https://php-parser.com
MIT License
941 stars 63 forks source link

scanner doesn't support \x80-\xff identifiers #91

Closed ganlvtech closed 5 years ago

ganlvtech commented 5 years ago

On dev branch.

https://github.com/z7zmey/php-parser/blob/a7e37add48979d4804d99e996cf997bc3a6963f8/scanner/scanner.rl#L61

There is \x7f-\xff in .rl file.

https://github.com/z7zmey/php-parser/blob/a7e37add48979d4804d99e996cf997bc3a6963f8/scanner/scanner.go#L4932-L4952

But there isn't code like

if lex.data[(lex.p)] >= 127 {
    goto st128
}

By the way, what I found on php.net says, as a regular expression, it would be expressed thus: ^[a-zA-Z_\x80-\xff][a-zA-Z0-9_\x80-\xff]*$. See php variables basics.

Test script:

main.go

package main

import (
    "fmt"
    "os"

    "github.com/z7zmey/php-parser/php5"
    "github.com/z7zmey/php-parser/visitor"
)

func main() {
    bytes := []byte("<?php $\x80=1;echo $\x80;")

    parser := php5.NewParser(bytes)
    parser.Parse()

    for _, e := range parser.GetErrors() {
        fmt.Println(e, e.Pos)
    }

    visitor := visitor.Dumper{
        Writer: os.Stdout,
        Indent: "",
    }

    rootNode := parser.GetRootNode()
    rootNode.Walk(&visitor)
}

test.php

z7zmey commented 5 years ago

Thank you for the pull request. I have also added a test for this case.