PHPCompatInfo stops on empty/broken files during analyze

[yuri-ccp]

Bug report

OS

Host operating system and version: CloudLinux release 7.9 PHP CompatInfo version: 6.1.0 PHP version and extensions:

Reference List
==============

 Description                  Type   Name       Provided Installed Comment 
 The bcmath PHP extension     bundle bcmath     8.1.1    7.4.27            
 The bz2 PHP extension        bundle bz2        8.1.1    7.4.27            
 The calendar PHP extension   bundle calendar   8.1.1    7.4.27            
 The Core PHP extension       bundle core       8.1.1    7.4.27            
 The ctype PHP extension      bundle ctype      8.1.1    7.4.27            
 The curl PHP extension       bundle curl       8.1.1    7.4.27            
 The date PHP extension       bundle date       8.1.1    7.4.27            
 The dom PHP extension        bundle dom        8.1.1    20031129          
 The enchant PHP extension    bundle enchant    8.1.1    7.4.27            
 The exif PHP extension       bundle exif       8.1.1    7.4.27            
 The fileinfo PHP extension   bundle fileinfo   8.1.1    7.4.27            
 The filter PHP extension     bundle filter     8.1.1    7.4.27            
 The ftp PHP extension        bundle ftp        8.1.1    7.4.27            
 The gd PHP extension         bundle gd         8.1.1    7.4.27            
 The gettext PHP extension    bundle gettext    8.1.1    7.4.27            
 The gmp PHP extension        bundle gmp        8.1.1    7.4.27            
 The hash PHP extension       bundle hash       8.1.1    7.4.27            
 The iconv PHP extension      bundle iconv      8.1.1    7.4.27            
 The imagick PHP extension    pecl   imagick    3.6.0    3.4.4             
 The imap PHP extension       bundle imap       8.1.1    7.4.27            
 The intl PHP extension       bundle intl       8.1.1    7.4.27            
 The json PHP extension       bundle json       8.1.1    7.4.27            
 The ldap PHP extension       bundle ldap       8.1.1    7.4.27            
 The libxml PHP extension     bundle libxml     8.1.1    7.4.27            
 The mbstring PHP extension   bundle mbstring   8.1.1    7.4.27            
 The mysqli PHP extension     bundle mysqli     8.1.1    7.4.27            
 The odbc PHP extension       bundle odbc       8.1.1    7.4.27            
 The OPcache PHP extension    bundle opcache    8.1.1    7.4.27            
 The openssl PHP extension    bundle openssl    8.1.1    7.4.27            
 The pcntl PHP extension      bundle pcntl      8.1.1    7.4.27            
 The pcre PHP extension       bundle pcre       8.1.1    7.4.27            
 The PDO PHP extension        bundle pdo        8.1.1    7.4.27            
 The pgsql PHP extension      bundle pgsql      8.1.1    7.4.27            
 The phar PHP extension       bundle phar       8.1.1    7.4.27            
 The posix PHP extension      bundle posix      8.1.1    7.4.27            
 The readline PHP extension   bundle readline   8.1.1    7.4.27            
 The Reflection PHP extension bundle reflection 8.1.1    7.4.27            
 The session PHP extension    bundle session    8.1.1    7.4.27            
 The shmop PHP extension      bundle shmop      8.1.1    7.4.27            
 The SimpleXML PHP extension  bundle simplexml  8.1.1    7.4.27            
 The soap PHP extension       bundle soap       8.1.1    7.4.27            
 The sockets PHP extension    bundle sockets    8.1.1    7.4.27            
 The spl PHP extension        bundle spl        8.1.1    7.4.27            
 The sqlite3 PHP extension    bundle sqlite3    8.1.1    7.4.27            
 The standard PHP extension   bundle standard   8.1.1    7.4.27            
 The sysvmsg PHP extension    bundle sysvmsg    8.1.1    7.4.27            
 The sysvsem PHP extension    bundle sysvsem    8.1.1    7.4.27            
 The sysvshm PHP extension    bundle sysvshm    8.1.1    7.4.27            
 The tidy PHP extension       bundle tidy       8.1.1    7.4.27            
 The tokenizer PHP extension  bundle tokenizer  8.1.1    7.4.27            
 The xml PHP extension        bundle xml        8.1.1    7.4.27            
 The xmlreader PHP extension  bundle xmlreader  8.1.1    7.4.27            
 The xmlrpc PHP extension     pecl   xmlrpc     1.0.0RC2 7.4.27            
 The xmlwriter PHP extension  bundle xmlwriter  8.1.1    7.4.27            
 The xsl PHP extension        bundle xsl        8.1.1    7.4.27            
 The zip PHP extension        bundle zip        1.20.0   1.15.6            
 The zlib PHP extension       bundle zlib       8.1.1    7.4.27            
 Total [57]                                                                

 ! [NOTE] Platform "PHP Interpreter 7.4.27" built 2022-01-17T10:34:36-03:00

Summary

PHPCompatinfo stops when finds syntax errors or empty files. Isn't uncommon that some people forget broken files or purposely give away some empty files to avoid some index listing. So in order to be capable to analyze pages and systems with such files I think that phpcompatinfo needs to ignore them instead of return an error.

Expected behaviour

Ignore broken files

Actual behaviour

Gives fatal errors like these:

[ERROR] Handling "Bartlett\CompatInfo\Application\Query\Analyser\Compatibility\GetCompatibilityQuery" failed: File has 
         no contents on line 1

 [ERROR] Handling "Bartlett\CompatInfo\Application\Query\Analyser\Compatibility\GetCompatibilityQuery" failed: Syntax   
         error, unexpected ',' on line 91 

[llaville]

This issue affect only versions 6.x Old version 5.5 is not affected by this bug.

Here is an example (of expected behavior) with version : bin/phpcompatinfo --version
phpCompatInfo version 5.5.6 DB version 3.16.1

• bin/phpcompatinfo analyser:run tests/fixtures/emptyDir/
• bin/phpcompatinfo analyser:run tests/fixtures/emptyFile.php with emptyFile.php equivalent to <?php

Data Source Analysed

Directories                                          1
Files                                                1
Errors                                               1
Errors found

 > File has no contents on line 1 in file /shared/backups/github/compatinfo-55/tests/fixtures/emptyFile.php

No extension found

No namespace found

No interface found

No trait found

No class found

No generator found

No function found

No constant found

No condition found

Requires PHP 4.0.0 (min)

[llaville]

I won't have time to fix it today, but @yuri-ccp, if you want to give a try with the following quick fix, your feedback will we welcome !

Issue came from line https://github.com/llaville/php-compatinfo/blob/6.1/src/Presentation/Console/Command/AnalyserCommand.php#L61

Here is the patch

diff --git a/src/Presentation/Console/Command/AnalyserCommand.php b/src/Presentation/Console/Command/AnalyserCommand.php
index 0786b098..3c7aea88 100644
--- a/src/Presentation/Console/Command/AnalyserCommand.php
+++ b/src/Presentation/Console/Command/AnalyserCommand.php
@@ -73,7 +73,7 @@ final class AnalyserCommand extends AbstractCommand implements CommandInterface
         $compatibilityQuery = new GetCompatibilityQuery(
             $input->getArgument('source'),
             $input->getOption('exclude'),
-            $input->hasOption('stop-on-failure')
+            $input->getOption('stop-on-failure')
         );

         try {

[llaville]

Forget the notice error

Notice: Undefined offset: 0 in /shared/backups/github/compatinfo-60/src/Application/PhpParser/NodeVisitor/ParentContextVisitor.php on line 61

Patch to fix it is :

diff --git a/src/Application/PhpParser/NodeVisitor/ParentContextVisitor.php b/src/Application/PhpParser/NodeVisitor/ParentContextVisitor.php
index 4ad956de..e275322d 100644
--- a/src/Application/PhpParser/NodeVisitor/ParentContextVisitor.php
+++ b/src/Application/PhpParser/NodeVisitor/ParentContextVisitor.php
@@ -58,7 +58,7 @@ final class ParentContextVisitor extends NodeVisitorAbstract
             /** @var Node\Stmt[] $newNodes */
             $newNodes = $nodes;
             // global namespace is not explicitly specified in source code ... we will add it
-            if ($nodes[0] instanceof Node\Stmt\Declare_) {
+            if (count($nodes) && $nodes[0] instanceof Node\Stmt\Declare_) {
                 $declare = array_shift($newNodes);
                 $newNodes = [$declare, new Node\Stmt\Namespace_(null, $newNodes)];
             } else {

[llaville]

Here is a preview (of expected behavior)

[yuri-ccp]

Thanks, I will apply test here

[yuri-ccp]

I tested here and now gives me this error:

 [ERROR] Handling "Bartlett\CompatInfo\Application\Query\Analyser\Compatibility\GetCompatibilityQuery" failed: Argument 
         1 passed to PhpParser\NodeTraverser::traverse() must be of the type array, null given, called in               
         /usr/local/vendor/bartlett/php-compatinfo/src/Application/PhpParser/Parser.php on line 148

I don't know if this error is related.

[llaville]

If your source code is public, please provide url, and specify what file is origin of this error

[llaville]

@yuri-ccp I really want to know what is your use case. If you can't provide an repository URL, at least POST the file that raise this error, to let me a change to see what was wrong. Otherwise I'll consider that the fixes I've provided is enough to fix the regression !

[llaville]

Unit tests for regression was added and fix (corresponding to previous patches) was applied on branch 6.0

[llaville]

Branch 6.1 is now up-to-date (sync with fixes on branch 6.0)

[llaville]

Branch 6.2 is now up-to-date (sync with fixes on branch 6.1)

[llaville]

Releases 6.0.4 and 6.1.1 are now on way !

[yuri-ccp]

@yuri-ccp I really want to know what is your use case. If you can't provide an repository URL, at least POST the file that raise this error, to let me a change to see what was wrong. Otherwise I'll consider that the fixes I've provided is enough to fix the regression !

Sorry @llaville I was offline when u respond.

The system giving that error in an old plugin for SPIP Core 3.1 named MediaBox (https://plugins.spip.net/mediabox.html?lang=en) this error happens in versions 1.0.4 and 1.2.0 when the compatinfo analyses the file mediabox_pipelines.php.

The error keeps happening in version 6.1.1. But I think is maybe unrelated to this bug.

Follow this file code:

<?php

if (!defined('_ECRIRE_INC_VERSION')) {
        return;
}

function mediabox_config($public = null) {
        include_spip('inc/filtres');
        include_spip('inc/config');
        $config = lire_config('mediabox', array());

        $config = array_merge(array(
                'active' => 'oui',
                'traiter_toutes_images' => 'oui',
                'selecteur_galerie' => '#documents_portfolio a[type=\'image/jpeg\'],#documents_portfolio a[type=\'image/png\'],#documents_portfolio a[type=\'image/gif\']',
                'selecteur_commun' => '.mediabox',
                'splash_url' => '',
                'splash_width' => '600px',
                'splash_height' => '90%',
                'skin' => 'black-striped',
                'transition' => 'elastic',
                'speed' => '200',
                'maxWidth' => '90%',
                'maxHeight' => '90%',
                'minWidth' => '400px',
                'minHeight' => '',
                'slideshow_speed' => '2500',
                'opacite' => '0.9',
        ), $config);

        if ((is_null($public) and test_espace_prive()) or $public === false) {
                $config = array_merge($config, array(
                        'active' => 'oui',
                        'selecteur_galerie' => '#portfolios a[type^=\'image/\']',
                        'selecteur_commun' => '.mediabox, .iconifier a[href$=jpg],.iconifier a[href$=png],.iconifier a[href$=gif]',
                        'splash_url' => '',
                        'skin' => 'white-shadow',
                        'maxWidth' => '90%',
                        'maxHeight' => '95%',
                        'minWidth' => '600px',
                        'minHeight' => '300px',
                        'opacite' => '0.9',
                ));
        }

        // Gerer aussi les liens internes de SPIP
        if (!test_espace_prive() and $config['splash_url']) {
                include_spip('inc/filtres_ecrire');
                $config['splash_url'] = url_absolue(extraire_attribut(lien_article_virtuel($config['splash_url']), 'href'));
        }

        // charger la config du theme uniquement dans le public
        if (!test_espace_prive() and include_spip('colorbox/' . $config['skin'] . '/mediabox_config_theme')) {
                $config_theme = mediabox_config_theme();
                $config = array_merge($config, $config_theme);
        }

        return $config;
}

function mediabox_insert_head_css($flux) {
        $config = mediabox_config();
        if ($config['active'] == 'oui'
                and $f = find_in_path((test_espace_prive() ? 'prive/' : '') . 'colorbox/' . $config['skin'] . '/colorbox.css')) {
                $flux .= '<link rel="stylesheet" href="' . direction_css($f) . '" type="text/css" media="all" />';
                /**
                 * Initialiser la config de la mediabox
                 */
                $configmediabox = '<script type="text/javascript">/* <![CDATA[ */
var box_settings = {tt_img:' . ($config['traiter_toutes_images'] == 'oui' ? 'true' : 'false')
                        . ',sel_g:"' . $config['selecteur_galerie']
                        . '",sel_c:"' . $config['selecteur_commun']
                        . '",trans:"' . $config['transition']
                        . '",speed:"' . $config['speed']
                        . '",ssSpeed:"' . $config['slideshow_speed']
                        . '",maxW:"' . $config['maxWidth']
                        . '",maxH:"' . $config['maxHeight']
                        . '",minW:"' . $config['minWidth']
                        . '",minH:"' . $config['minHeight']
                        . '",opa:"' . $config['opacite']
                        . '",str_ssStart:"' . unicode2charset(html2unicode(_T('mediabox:boxstr_slideshowStart')))
                        . '",str_ssStop:"' . unicode2charset(html2unicode(_T('mediabox:boxstr_slideshowStop')))
                        . '",str_cur:"' . _T('mediabox:boxstr_current', array('current' => '{current}', 'total' => '{total}'))
                        . '",str_prev:"' . _T('mediabox:boxstr_previous')
                        . '",str_next:"' . _T('mediabox:boxstr_next')
                        . '",str_close:"' . _T('mediabox:boxstr_close')
                        . '",splash_url:"' . $config['splash_url']
                        . '"};' . "\n";
                // Si c'est une image, on la chargera avec une redimentionnement automatique
                // Sinon, chargement dans une iframe
                $extension = pathinfo($config['splash_url'], PATHINFO_EXTENSION);
                if (match($extension, 'gif|png|jpg|jpeg')) {
                        $configmediabox .= 'var box_settings_iframe = false;' . "\n";
                } else {
                        $configmediabox .= 'var box_settings_splash_width = "' . $config['splash_width'] . '";
var box_settings_splash_height = "' . $config['splash_height'] . '";' . "\n";
                        $configmediabox .= 'var box_settings_iframe = true;' . "\n";
                }
                $flux = $configmediabox . '/* ]]> */</script>' . "\n" . $flux;
        }

        return $flux;
}

function mediabox_timestamp($fichier) {
        if ($m = filemtime($fichier)) {
                return "$fichier?$m";
        }

        return $fichier;
}

function mediabox_insert_head($flux) {
        $config = mediabox_config();
        if ($config['active'] == 'oui') {
                $flux .= '
        <script src="' . mediabox_timestamp(find_in_path('javascript/jquery.colorbox.js')) . '" type="text/javascript"></script>
        <script src="' . mediabox_timestamp(find_in_path('javascript/spip.mediabox.js')) . '" type="text/javascript"></script>';
                if ($config['splash_url']) {
                        $flux .= '<script src="' . mediabox_timestamp(find_in_path('javascript/splash.mediabox.js')) . '" type="text/javascript"></script>';
                }
        }

        return $flux;
}

function mediabox_jquery_plugins($plugins) {
        $config = mediabox_config();
        if ($config['splash_url']) {
                $plugins[] = 'javascript/js.cookie.js';
        }

        return $plugins;
}

[llaville]

I would have liked to get this information before release versions 6.0.4 and 6.1.1.

Effectively, under some condition nikic/php-parser may returns null rather than an empty array for statements on recovery mode. See https://github.com/nikic/PHP-Parser/blob/master/bin/php-parse#L66

Confirmed by command: vendor/bin/php-parse -r -d tests/mediabox_pipelines.php that give following output

====> File tests/mediabox_pipelines.php:
Syntax error, unexpected ',' on line 92
Syntax error, unexpected T_ELSE, expecting '}' on line 94
Syntax error, unexpected '}', expecting EOF on line 103

To have such behaviour with CompatInfo 6.0.x or 6.1.x, here is the quick patch to apply

diff --git a/src/Application/PhpParser/Parser.php b/src/Application/PhpParser/Parser.php
index da8a7a76..f99bd781 100644
--- a/src/Application/PhpParser/Parser.php
+++ b/src/Application/PhpParser/Parser.php
@@ -136,6 +136,7 @@ final class Parser
             $this->errorHandler
         );
         if (empty($stmts)) {
+            $stmts = [];
             $this->errorHandler->handleError(
                 new Error('File has no contents', ['startLine' => 1])
             );

And CompatInfo, is able to return a result (something like)

No bugfixes release will follow for 6.0 and 6.1 to catch this condition (very special). But It will be include for upcoming 6.2

[llaville]

Screenshot of results obtained with commit 96597e3