search

Andrewsville / PHP-Token-Reflection

Library emulating the PHP internal reflection using just the tokenized source code
http://andrewsville.github.com/PHP-Token-Reflection/
Other
190 stars 44 forks source link

Unexpected token found. #46

Closed thunderdox closed 11 years ago

thunderdox commented 12 years ago

The cause of the exception "Unexpected token found." was the T_GOTO token (line 127) in following part of the source code:

114: 115: $this->goto = TEMPLATES . '/404.html'; 116: //throw new FwException('Unrecognized request [' . User::get_real_ip() . ']: ' . 117: // ov($_SERVER['REDIRECT_URL']), E_USER_NOTICE); 118: } 119: } 120: } 121:
122: /* 123: * Include appropriate document 124: 125: * @return void 126: / 127: public function goto() 128: { 129: try { 130: HttpStatus::header(200); 131: if(!is_link($this->goto)) { 132: require_once

The exception backtrace is following:

0 C:\Users\brano\tools\php\PEAR\TokenReflection\ReflectionMethod.php(695): TokenReflection\ReflectionFunctionBase->parseReturnsReference(Object(TokenReflection\Stream\StringStream))

1 C:\Users\brano\tools\php\PEAR\TokenReflection\ReflectionElement.php(121): TokenReflection\ReflectionMethod->parse(Object(TokenReflection\Stream\StringStream), Object(TokenReflection\ReflectionClass))

2 C:\Users\brano\tools\php\PEAR\TokenReflection\ReflectionBase.php(73): TokenReflection\ReflectionElement->parseStream(Object(TokenReflection\Stream\StringStream), Object(TokenReflection\ReflectionClass))

3 C:\Users\brano\tools\php\PEAR\TokenReflection\ReflectionElement.php(104): TokenReflection\ReflectionBase->__construct(Object(TokenReflection\Stream\StringStream), Object(TokenReflection\Broker), Object(TokenReflection\ReflectionClass))

4 C:\Users\brano\tools\php\PEAR\TokenReflection\ReflectionClass.php(1839): TokenReflection\ReflectionElement->__construct(Object(TokenReflection\Stream\StringStream), Object(TokenReflection\Broker), Object(TokenReflection\ReflectionClass))

5 C:\Users\brano\tools\php\PEAR\TokenReflection\ReflectionElement.php(122): TokenReflection\ReflectionClass->parseChildren(Object(TokenReflection\Stream\StringStream), Object(TokenReflection\ReflectionFileNamespace))

6 C:\Users\brano\tools\php\PEAR\TokenReflection\ReflectionBase.php(73): TokenReflection\ReflectionElement->parseStream(Object(TokenReflection\Stream\StringStream), Object(TokenReflection\ReflectionFileNamespace))

7 C:\Users\brano\tools\php\PEAR\TokenReflection\ReflectionElement.php(104): TokenReflection\ReflectionBase->__construct(Object(TokenReflection\Stream\StringStream), Object(TokenReflection\Broker), Object(TokenReflection\ReflectionFileNamespace))

8 C:\Users\brano\tools\php\PEAR\TokenReflection\ReflectionFileNamespace.php(299): TokenReflection\ReflectionElement->__construct(Object(TokenReflection\Stream\StringStream), Object(TokenReflection\Broker), Object(TokenReflection\ReflectionFileNamespace))

9 C:\Users\brano\tools\php\PEAR\TokenReflection\ReflectionElement.php(122): TokenReflection\ReflectionFileNamespace->parseChildren(Object(TokenReflection\Stream\StringStream), Object(TokenReflection\ReflectionFile))

10 C:\Users\brano\tools\php\PEAR\TokenReflection\ReflectionBase.php(73): TokenReflection\ReflectionElement->parseStream(Object(TokenReflection\Stream\StringStream), Object(TokenReflection\ReflectionFile))

11 C:\Users\brano\tools\php\PEAR\TokenReflection\ReflectionElement.php(104): TokenReflection\ReflectionBase->__construct(Object(TokenReflection\Stream\StringStream), Object(TokenReflection\Broker), Object(TokenReflection\ReflectionFile))

12 C:\Users\brano\tools\php\PEAR\TokenReflection\ReflectionFile.php(121): TokenReflection\ReflectionElement->__construct(Object(TokenReflection\Stream\StringStream), Object(TokenReflection\Broker), Object(TokenReflection\ReflectionFile))

13 C:\Users\brano\tools\php\PEAR\TokenReflection\ReflectionBase.php(73): TokenReflection\ReflectionFile->parseStream(Object(TokenReflection\Stream\StringStream), NULL)

14 C:\Users\brano\tools\php\PEAR\TokenReflection\Broker.php(168): TokenReflection\ReflectionBase->__construct(Object(TokenReflection\Stream\StringStream), Object(TokenReflection\Broker))

15 C:\Users\brano\tools\php\PEAR\ApiGen\Generator.php(235): TokenReflection\Broker->processString('<?php?/*? \ Ge...', 'C:\Users\brano...')

16 C:\Users\brano\tools\php\apigen(130): ApiGen\Generator->parse()

17 {main}

Andrewsville commented 12 years ago

It seems like you are parsing a pre PHP 5.3 source code. Since 5.3 goto is a keyword and such source code IMO would not run (parse error).

I am thinking about an optional customizable filter, that would take the token stream and preprocess it before it gets parsed. It would allow a transparent support of legacy codes like this.

tristanpemble commented 11 years ago

Having a similar issue:

The cause of the exception "Unexpected token found." was the T_LIST token (line 17) in following part of the source code:

....
 15: class Vps extends Module
 16: {
*17:     private function list() {
....

This is perfectly valid PHP code. It seems like it is assuming that we can't define a function within a class that has the same name as any core PHP function, in this case list()

The exception backtrace is following:

#0 /Applications/MAMP/bin/php/php5.3.14/lib/php/TokenReflection/ReflectionMethod.php(695): TokenReflection\ReflectionFunctionBase->parseReturnsReference(Object(TokenReflection\Stream\StringStream))
#1 /Applications/MAMP/bin/php/php5.3.14/lib/php/TokenReflection/ReflectionElement.php(121): TokenReflection\ReflectionMethod->parse(Object(TokenReflection\Stream\StringStream), Object(TokenReflection\ReflectionClass))
#2 /Applications/MAMP/bin/php/php5.3.14/lib/php/TokenReflection/ReflectionBase.php(73): TokenReflection\ReflectionElement->parseStream(Object(TokenReflection\Stream\StringStream), Object(TokenReflection\ReflectionClass))
#3 /Applications/MAMP/bin/php/php5.3.14/lib/php/TokenReflection/ReflectionElement.php(104): TokenReflection\ReflectionBase->__construct(Object(TokenReflection\Stream\StringStream), Object(TokenReflection\Broker), Object(TokenReflection\ReflectionClass))
#4 /Applications/MAMP/bin/php/php5.3.14/lib/php/TokenReflection/ReflectionClass.php(1839): TokenReflection\ReflectionElement->__construct(Object(TokenReflection\Stream\StringStream), Object(TokenReflection\Broker), Object(TokenReflection\ReflectionClass))
#5 /Applications/MAMP/bin/php/php5.3.14/lib/php/TokenReflection/ReflectionElement.php(122): TokenReflection\ReflectionClass->parseChildren(Object(TokenReflection\Stream\StringStream), Object(TokenReflection\ReflectionFileNamespace))
#6 /Applications/MAMP/bin/php/php5.3.14/lib/php/TokenReflection/ReflectionBase.php(73): TokenReflection\ReflectionElement->parseStream(Object(TokenReflection\Stream\StringStream), Object(TokenReflection\ReflectionFileNamespace))
#7 /Applications/MAMP/bin/php/php5.3.14/lib/php/TokenReflection/ReflectionElement.php(104): TokenReflection\ReflectionBase->__construct(Object(TokenReflection\Stream\StringStream), Object(TokenReflection\Broker), Object(TokenReflection\ReflectionFileNamespace))
#8 /Applications/MAMP/bin/php/php5.3.14/lib/php/TokenReflection/ReflectionFileNamespace.php(299): TokenReflection\ReflectionElement->__construct(Object(TokenReflection\Stream\StringStream), Object(TokenReflection\Broker), Object(TokenReflection\ReflectionFileNamespace))
#9 /Applications/MAMP/bin/php/php5.3.14/lib/php/TokenReflection/ReflectionElement.php(122): TokenReflection\ReflectionFileNamespace->parseChildren(Object(TokenReflection\Stream\StringStream), Object(TokenReflection\ReflectionFile))
#10 /Applications/MAMP/bin/php/php5.3.14/lib/php/TokenReflection/ReflectionBase.php(73): TokenReflection\ReflectionElement->parseStream(Object(TokenReflection\Stream\StringStream), Object(TokenReflection\ReflectionFile))
#11 /Applications/MAMP/bin/php/php5.3.14/lib/php/TokenReflection/ReflectionElement.php(104): TokenReflection\ReflectionBase->__construct(Object(TokenReflection\Stream\StringStream), Object(TokenReflection\Broker), Object(TokenReflection\ReflectionFile))
#12 /Applications/MAMP/bin/php/php5.3.14/lib/php/TokenReflection/ReflectionFile.php(130): TokenReflection\ReflectionElement->__construct(Object(TokenReflection\Stream\StringStream), Object(TokenReflection\Broker), Object(TokenReflection\ReflectionFile))
#13 /Applications/MAMP/bin/php/php5.3.14/lib/php/TokenReflection/ReflectionBase.php(73): TokenReflection\ReflectionFile->parseStream(Object(TokenReflection\Stream\StringStream), NULL)
#14 /Applications/MAMP/bin/php/php5.3.14/lib/php/TokenReflection/Broker.php(168): TokenReflection\ReflectionBase->__construct(Object(TokenReflection\Stream\StringStream), Object(TokenReflection\Broker))
#15 /Applications/MAMP/bin/php/php5.3.14/lib/php/ApiGen/Generator.php(235): TokenReflection\Broker->processString('<?php?/**? * VP...', '/Users/tristanp...')
#16 /Applications/MAMP/bin/php/php5.3.14/bin/apigen(130): ApiGen\Generator->parse()
#17 {main}
Andrewsville commented 11 years ago

Sorry for the late reply and sorry for closing, but the source codes that cause these exceptions are not valid.