wodby / drupal-php

PHP docker container image for Drupal
https://wodby.com/stacks/drupal
MIT License
60 stars 80 forks source link

Using 8.2-dev-4.50.0 (gRPC extension) causes PHPCS to hang if `--parallel` is used #102

Closed mbomb007 closed 5 months ago

mbomb007 commented 6 months ago

For some reason, using wodby/drupal-php:8.2-dev-4.49.0 works, but images with wodby/drupal-php:8.2-dev-4.50.0 or newer cause PHPCS to hang, but only if PHPCS is run without verbose flag, or with the single-level verbosity -v. If PHPCS is run with -vv or -vvv, PHPCS runs successfully. The same problem occurs if I use PHP 8.3 instead of 8.2 and use the newest image and stability tag.

Further debugging revealed that this is because extra-verbose output means that sniffs cannot be run in parallel. Removing the --parallel option or setting --parallel=1 will also fix the hanging issue. Using a new image along with the --parallel option greater than 1 is what causes execution to hang.

My goal is to be able to use 4.51.0 or higher, since PHP 8.3 support was added.

Debugging Details and Steps to Reproduce

Click to view The PHPCS command: ```bash vendor/bin/phpcs -p -w --colors --parallel=2 --standard=custom_ruleset.xml --extensions=engine,inc,install,module,php,profile,test,theme,yaml,yml --ignore=*/features/* ../web/modules/custom ``` **custom_ruleset.xml** ```xml ``` **composer.json** ```json { "require-dev": { "dealerdirect/phpcodesniffer-composer-installer": "^1.0", "drupal/coder": "^8.3", "phpcompatibility/php-compatibility": "^9", "roave/security-advisories": "dev-latest", "squizlabs/php_codesniffer": "^3.8.1" }, "prefer-stable": true, "scripts": { "install-codestandards": [ "PHPCSStandards\\Composer\\Plugin\\Installers\\PHPCodeSniffer\\Plugin::run" ], "post-install-cmd": [ "@install-codestandards" ], "post-update-cmd": [ "@install-codestandards" ] }, "config": { "sort-packages": true, "allow-plugins": { "dealerdirect/phpcodesniffer-composer-installer": true } } } ``` The output when running hangs and the PHPCS command adds the first level of verbosity with `-v`: ``` Registering sniffs in the Drupal_Custom standard... DONE (172 sniffs registered) Creating file list... DONE (13 files in queue) ``` Then the output hangs. [**web_modules_custom.zip**](https://github.com/wodby/drupal-php/files/13968606/web_modules_custom.zip) This is almost the exact contents of the folder PHPCS is being run on. I excluded a composer.json file and changed a module's name. If I run on an image with stability tag 4.49.0 or earlier, the same PHPCS command and composer.json runs successfully, regardless of the verbosity `-v` flag level included. This is the expected output: ``` Installing dependencies from lock file (including require-dev) Verifying lock file contents can be installed on current platform. Package operations: 10 installs, 0 updates, 0 removals - Downloading squizlabs/php_codesniffer (3.8.1) - Downloading dealerdirect/phpcodesniffer-composer-installer (v1.0.0) - Downloading symfony/polyfill-ctype (v1.28.0) - Downloading symfony/yaml (v7.0.0) - Downloading phpstan/phpdoc-parser (1.25.0) - Downloading slevomat/coding-standard (8.14.1) - Downloading sirbrillig/phpcs-variable-analysis (v2.11.17) - Downloading drupal/coder (8.3.22) - Downloading phpcompatibility/php-compatibility (9.3.5) - Installing squizlabs/php_codesniffer (3.8.1): Extracting archive - Installing dealerdirect/phpcodesniffer-composer-installer (v1.0.0): Extracting archive - Installing symfony/polyfill-ctype (v1.28.0): Extracting archive - Installing symfony/yaml (v7.0.0): Extracting archive - Installing phpstan/phpdoc-parser (1.25.0): Extracting archive - Installing slevomat/coding-standard (8.14.1): Extracting archive - Installing sirbrillig/phpcs-variable-analysis (v2.11.17): Extracting archive - Installing drupal/coder (8.3.22): Extracting archive - Installing phpcompatibility/php-compatibility (9.3.5): Extracting archive - Installing roave/security-advisories (dev-latest b7a13fc) Generating autoload files 5 packages you are using are looking for funding. Use the `composer fund` command to find out more! PHP CodeSniffer Config installed_paths set to ../../drupal/coder/coder_sniffer,../../phpcompatibility/php-compatibility,../../sirbrillig/phpcs-variable-analysis,../../slevomat/coding-standard > PHPCSStandards\Composer\Plugin\Installers\PHPCodeSniffer\Plugin::run vendor/bin/phpcs -v -p -w --colors --parallel=2 --standard=custom_ruleset.xml --extensions=engine,inc,install,module,php,profile,test,theme,yaml,yml --ignore=*/features/* ../web/modules/custom Registering sniffs in the Drupal_Custom standard... DONE (172 sniffs registered) Creating file list... DONE (13 files in queue) Changing into directory /builds/web/html/web/modules/custom/vite Processing drush.services.yml [PHP => 7 tokens in 7 lines]... DONE in 9ms (0 errors, 0 warnings) Processing vite.info.yml [PHP => 6 tokens in 6 lines]... DONE in 5ms (0 errors, 0 warnings) Processing vite.libraries.yml [PHP => 5 tokens in 5 lines]... DONE in 0ms (0 errors, 0 warnings) Processing vite.links.menu.yml [PHP => 6 tokens in 6 lines]... DONE in 0ms (0 errors, 0 warnings) Processing vite.module [PHP => 922 tokens in 120 lines]... DONE in 53ms (0 errors, 0 warnings) Processing vite.routing.yml [PHP => 7 tokens in 7 lines]... DONE in 0ms (0 errors, 0 warnings) Changing into directory /builds/web/html/web/modules/custom/disable_user_edit/src/Services Processing RouteSubscriber.php [PHP => 172 tokens in 30 lines]... DONE in 36ms (0 errors, 0 warnings) Changing into directory /builds/web/html/web/modules/custom/disable_user_edit Processing disable_user_edit.info.yml [PHP => 5 tokens in 5 lines]... DONE in 6ms (0 errors, 0 warnings) Processing disable_user_edit.services.yml [PHP => 5 tokens in 5 lines]... DONE in 0ms (0 errors, 0 warnings) Changing into directory /builds/web/html/web/modules/custom/vite/config/optional Processing vite.settings.yml [PHP => 2 tokens in 2 lines]... DONE in 0ms (0 errors, 0 warnings) Changing into directory /builds/web/html/web/modules/custom/vite/src/Commands Processing ViteCommands.php [PHP => 500 tokens in 78 lines]... DONE in 20ms (0 errors, 0 warnings) Changing into directory /builds/web/html/web/modules/custom/vite/src/Form Processing ViteSettingsForm.php [PHP => 405 tokens in 64 lines]... DONE in 15ms (0 errors, 0 warnings) Changing into directory /builds/web/html/web/modules/custom/vite/src/Helper Processing Vite.php [PHP => 127 tokens in 31 lines]... DONE in 2ms (0 errors, 0 warnings) Time: 202ms; Memory: 8MB ```

Debugging Update & Workaround

After trying a couple more things, I found that either removing --parallel from the PHPCS command, or setting its value to 1, will fix issue on the newer images. This also makes sense with the verbosity flags since, I assume, when verbose is enabled at a high level then sniffing cannot be parallelized.

So, this means that the newer images break parallel execution of sniffs for some reason. Removing the --parallel option isn't ideal, obviously, as this increases the execution time. So hopefully this information can point towards the cause of the issue and a fix to follow.

mbomb007 commented 6 months ago

comment was moved into issue summary above

mbomb007 commented 6 months ago

After trying a couple more things, I found that either removing --parallel from the PHPCS command, or setting its value to 1, will fix issue on the newer images. This also makes sense with the verbosity flags since, I assume, when verbose is enabled at a high level then sniffing cannot be parallelized.

csandanov commented 6 months ago

My guess the difference is Alpine version and maybe phpcs depended on some packages that were missing or updated in 3.19. Could you try 4.52.0 where we added a few missing imagemagick- packages

mbomb007 commented 6 months ago

I tried just now and the newer stability tag didn't fix the problem.

The only dependencies PHPCS needs are:

    "require": {
        "php": ">=5.4.0",
        "ext-simplexml": "*",
        "ext-tokenizer": "*",
        "ext-xmlwriter": "*"
    },
mbomb007 commented 6 months ago

I found the cause of the issue. https://github.com/grpc/grpc/issues/20250#issuecomment-531321945

Debug code

# Check php.ini values.

test=$(php -r "echo ini_get('pcre.jit');")
echo "pcre.jit: $test"
# Can PCRE JIT be disabled? PHPCS does this in its code.
test=$(php -r "ini_set('pcre.jit', false);echo ini_get('pcre.jit');")
echo "pcre.jit: $test"

test=$(php -r "echo ini_get('grpc.enable_fork_support');")
echo "grpc.enable_fork_support: $test"
test=$(php -r "echo ini_get('grpc.poll_strategy');")
echo "grpc.poll_strategy: $test"

# Will it work if grpc values are set properly?
echo "grpc.enable_fork_support = 1" >> /usr/local/etc/php/conf.d/docker-php-ext-grpc.ini
echo "grpc.poll_strategy = epoll1" >> /usr/local/etc/php/conf.d/docker-php-ext-grpc.ini

test=$(php -r "echo ini_get('grpc.enable_fork_support');")
echo "grpc.enable_fork_support: $test"
test=$(php -r "echo ini_get('grpc.poll_strategy');")
echo "grpc.poll_strategy: $test"

Output:

pcre.jit: 1
pcre.jit: 
grpc.enable_fork_support: 0
grpc.poll_strategy: 
grpc.enable_fork_support: 1
grpc.poll_strategy: epoll1

Explanation

From my tests, PHPCS should be able to disable pcre.jit properly. So that part is fine.

If the gRPC extension is enabled, then these options must be set for parallel to work:

grpc.enable_fork_support = 1
grpc.poll_strategy = epoll1

They aren't set, though. The initial/default values are as follows, which causes the parallel execution to fail:

grpc.enable_fork_support = 0
grpc.poll_strategy = 

Further explanation of why it causes it to fail is here.

Setting them to the necessary values from Bash prior to running PHPCS led the commands to succeed again.

Could these values be set in the Dockerfile, similar to what was done here?

mbomb007 commented 6 months ago

While I have an issue open for PHPCS to hopefully set the ini values in its code, it would still be ideal if they could be set in the Dockerfile for this project (or wodby/php upstream) as well.

csandanov commented 5 months ago

Hi, please try 4.53.0, it now has env vars to configure GRPC extension and updated default values (fork support and poll strategy) https://github.com/wodby/php/blob/master/8/templates/docker-php-ext-grpc.ini.tmpl

mbomb007 commented 5 months ago

Yes, that fixed it. Thanks.