While probably exceedingly rare to be found in actual codebases, the PHP tokenizer apparently allows for whitespace between the <<< and the heredoc/nowdoc identifier. See: https://3v4l.org/NUHZd
Both spaces as well as tabs are allowed. New lines are not allowed. Comments are also not allowed. See: https://3v4l.org/7PIEK
The PHPCS Tokenizer did not execute tab replacement on these tokens leading to unexpected 'content' and incorrect 'length' values in the File::$tokens array, which in turn could lead to incorrect sniff results and incorrect fixes.
This commit adds the T_START_HEREDOC/T_START_NOWDOC tokens to the array of tokens for which to do tab replacement to make them more consistent with the rest of PHPCS.
Includes unit tests safeguarding this change.
Suggested changelog entry
Tokenizer not applying tab replacement to heredoc/nowdoc openers
Description
You learn something new every day :wink:
While probably exceedingly rare to be found in actual codebases, the PHP tokenizer apparently allows for whitespace between the
<<<
and the heredoc/nowdoc identifier. See: https://3v4l.org/NUHZdBoth spaces as well as tabs are allowed. New lines are not allowed. Comments are also not allowed. See: https://3v4l.org/7PIEK
The PHPCS
Tokenizer
did not execute tab replacement on these tokens leading to unexpected'content'
and incorrect'length'
values in theFile::$tokens
array, which in turn could lead to incorrect sniff results and incorrect fixes.This commit adds the
T_START_HEREDOC
/T_START_NOWDOC
tokens to the array of tokens for which to do tab replacement to make them more consistent with the rest of PHPCS.Includes unit tests safeguarding this change.
Suggested changelog entry
Tokenizer not applying tab replacement to heredoc/nowdoc openers
Related issues/external references
Ref: https://externals.io/message/124462#124518
Loosely related to: https://github.com/squizlabs/PHP_CodeSniffer/pull/3639 in which a similar change was made for heredoc/nowdoc closers (in the context of PHP 7.3+ flexible heredocs/nowdocs).
Types of changes