search

mage2tv / magento-cache-clean

A faster drop in replacement for bin/magento cache:clean with file watcher
BSD 3-Clause "New" or "Revised" License
532 stars 63 forks source link

More Issues When Switching Branches / Stashing #58

Closed sprankhub closed 4 years ago

sprankhub commented 4 years ago

This is kind of a follow-up of #54.

When I switch branches / git stash / git stash pop, I got the following error:

PHP Warning:  require(/var/www/shop/vendor/composer/../namespace/module/registration.php): failed to open stream: No such file or directory in /var/www/shop/vendor/composer/autoload_real.php on line 70
PHP Fatal error:  require(): Failed opening required '/var/www/shop/vendor/composer/../namespace/module/registration.php' (include_path='/var/www/shop/vendor/magento/zendframework1/library:.:/usr/share/php:/var/www/shop/vendor/deployer/recipes:/var/www/shop/vendor/jalogut/magento2-deployer-plus') in /var/www/shop/vendor/composer/autoload_real.php on line 70
11:18:58 ERROR: failed shelling out to php for reading the module list.
11:18:58 ERROR Details: Command failed: php -r "require '/var/www/shop/vendor/autoload.php'; \$bp = strlen(dirname(dirname(realpath('/var/www/shop/vendor/autoload.php')))) + 1; foreach ((new \Magento\Framework\Component\ComponentRegistrar)->getPaths('module') as \$m) echo substr(\$m, \$bp).PHP_EOL;"
PHP Warning:  require(/var/www/shop/vendor/composer/../namespace/module/registration.php): failed to open stream: No such file or directory in /var/www/shop/vendor/composer/autoload_real.php on line 70
PHP Fatal error:  require(): Failed opening required '/var/www/shop/vendor/composer/../namespace/module/registration.php' (include_path='/var/www/shop/vendor/magento/zendframework1/library:.:/usr/share/php:/var/www/shop/vendor/deployer/recipes:/var/www/shop/vendor/jalogut/magento2-deployer-plus') in /var/www/shop/vendor/composer/autoload_real.php on line 70

PHP Warning:  require(/var/www/shop/vendor/composer/../namespace/module/registration.php): failed to open stream: No such file or directory in /var/www/shop/vendor/composer/autoload_real.php on line 70
PHP Fatal error:  require(): Failed opening required '/var/www/shop/vendor/composer/../namespace/module/registration.php' (include_path='/var/www/shop/vendor/magento/zendframework1/library:.:/usr/share/php:/var/www/shop/vendor/deployer/recipes:/var/www/shop/vendor/jalogut/magento2-deployer-plus') in /var/www/shop/vendor/composer/autoload_real.php on line 70
11:18:58 ERROR: failed shelling out to php for reading the theme list.
11:18:58 ERROR Details: Command failed: php -r "require '/var/www/shop/vendor/autoload.php'; \$bp = strlen(dirname(dirname(realpath('/var/www/shop/vendor/autoload.php')))) + 1; foreach ((new \Magento\Framework\Component\ComponentRegistrar)->getPaths('theme') as \$m) echo substr(\$m, \$bp).PHP_EOL;"
PHP Warning:  require(/var/www/shop/vendor/composer/../namespace/module/registration.php): failed to open stream: No such file or directory in /var/www/shop/vendor/composer/autoload_real.php on line 70
PHP Fatal error:  require(): Failed opening required '/var/www/shop/vendor/composer/../namespace/module/registration.php' (include_path='/var/www/shop/vendor/magento/zendframework1/library:.:/usr/share/php:/var/www/shop/vendor/deployer/recipes:/var/www/shop/vendor/jalogut/magento2-deployer-plus') in /var/www/shop/vendor/composer/autoload_real.php on line 70

[ERROR] ENOENT: no such file or directory, watch '/var/www/shop/pub/static/frontend'

Process finished with exit code 1

I see two isses here:

  1. The error when switching branches seems to be not fully fixed.
  2. There is an error when pub/static/frontend is no directory.
    1. The process should not crash if the directory is missing.
    2. Not sure why this directory is missing. I do not delete it myself. So maybe in some situations the cache clean tool deletes the whole directory?
Vinai commented 4 years ago

The PHP issue seems to be that the composer autoloader is referencing registration.php files that no longer exist after switching the branch. The message ERROR: failed shelling out to php for reading the module list. indicates that the process was able to recover. That is possible due to the fix in https://github.com/mage2tv/magento-cache-clean/commit/b0f2cc12379ca04cd86b24d96c67728f88999ab3#diff-78779ff4827a41650bf6bbbf32e3b7e8R55-R66 that closed the issue you referenced.

I do not know why pub/static/frontend is missing - the cache clean tool doesn't remove it. It does monitor all pub/static/frontend/*/*/*/ directories in order to be able to clean the full page cache when a compiled requirejs-config.js file is removed. Some action related to that seems to cause the process to die.

There are the "clean static content area" hotkeys F for pub/static/frontend and A for pub/static/adminhtml that remove those directories, but according to your output that didn't happen.

Vinai commented 4 years ago

I've made a few changes aimed at making the watcher more robust. Could you please try installing the branch-switching git branch and testing how that works for you?

To install in a project use

composer remove --dev mage2tv/magento-cache-clean
composer require --dev mage2tv/magento-cache-clean:dev-branch-switching

To install globally, use composer global instead.

Thank you

Vinai commented 4 years ago

@sprankhub any chance you check it out?

sprankhub commented 4 years ago

Sorry for not coming back to you, @Vinai! Will do. Does it make sense to first merge master into the branch-switching branch?

Vinai commented 4 years ago

Good idea - done. Thanks for being willing to try it out. Curious to hear your report.

Vinai commented 4 years ago

Quick note that I'm continuing to tune the code in the master branch. I'll won't merge the latest changes into the branch-switching branch, because I think the current state there should be good enough to check if the issue is resolved.

sprankhub commented 4 years ago

Tested it shortly and it indeed seems to fix the issue. Additionally, I found out that the pub/static/frontend directory is removed during n98-magerun2.phar setup:upgrade.

sprankhub commented 4 years ago

Thanks again by the way!

Vinai commented 4 years ago

Thanks for checking! The branch is merged into master and included in the 1.0.15 release.

sprankhub commented 4 years ago

Sorry @Vinai, but even though my manual test seemed to work, I now got a related error again with the 1.0.15 release. When switching branches:

/usr/local/bin/node /var/www/shop/vendor/bin/cache-clean.js --watch
Release 1.0.15 sponsored by https://www.mage2.tv

Hot-keys for manual cache cleaning:
[c]onfig [b]lock_html [l]ayout [t]ranslate [f]ull_page [v]iew [a]ll

Hot-key for cleaning all generated code: [G]
Hot-keys for cleaning static content areas: [F]rontend [A]dminhtml

Watcher initialized (Ctrl-C to quit)
14:16:36 Cleaning cache type(s) block_html full_page
14:16:36 Cleaning cache type(s) translate full_page
internal/fs/utils.js:220
    throw err;
    ^

Error: ENOENT: no such file or directory, scandir '/var/www/shop/var/view_preprocessed/pub/static/frontend'
    at Object.readdirSync (fs.js:854:3)
    at xi (/var/www/shop/vendor/mage2tv/magento-cache-clean/bin/cache-clean.js:343:476)
    at /var/www/shop/vendor/mage2tv/magento-cache-clean/bin/cache-clean.js:344:104
    at /var/www/shop/vendor/mage2tv/magento-cache-clean/bin/cache-clean.js:184:298
    at /var/www/shop/vendor/mage2tv/magento-cache-clean/bin/cache-clean.js:184:311
    at Yd (/var/www/shop/vendor/mage2tv/magento-cache-clean/bin/cache-clean.js:123:429)
    at Xd.f.N (/var/www/shop/vendor/mage2tv/magento-cache-clean/bin/cache-clean.js:126:224)
    at M (/var/www/shop/vendor/mage2tv/magento-cache-clean/bin/cache-clean.js:61:83)
    at de (/var/www/shop/vendor/mage2tv/magento-cache-clean/bin/cache-clean.js:132:137)
    at ne (/var/www/shop/vendor/mage2tv/magento-cache-clean/bin/cache-clean.js:146:38) {
  errno: -2,
  syscall: 'scandir',
  code: 'ENOENT',
  path: '/var/www/shop/var/view_preprocessed/pub/static/frontend'
}

Process finished with exit code 1

Could you re-open and double check?

Vinai commented 4 years ago

Hi Simon, thanks for the info - sure, I'll check!

Vinai commented 4 years ago

I think I was able to figure out how that ls on a non-existent directory is happening. It was a new issue. Thank you for uncovering it! Fixed in 1.0.16... can you please check this again? :)

sprankhub commented 4 years ago

Seems to work. Thanks again @Vinai!

sprankhub commented 4 years ago

Sorry, but I found another one :)

/usr/local/bin/node /var/www/shop/vendor/bin/cache-clean.js --watch
Release 1.0.17 sponsored by https://www.mage2.tv

[...]

16:32:46 Cleaning cache type(s) block_html full_page
PHP Warning:  require(/var/www/shop/vendor/composer/../shop/module-sth/registration.php): failed to open stream: No such file or directory in /var/www/shop/vendor/composer/autoload_real.php on line 70
PHP Fatal error:  require(): Failed opening required '/var/www/shop/vendor/composer/../shop/module-sth/registration.php' (include_path='/var/www/shop/vendor/magento/zendframework1/library:.:/usr/share/php:/var/www/shop/vendor/deployer/recipes:/var/www/shop/vendor/jalogut/magento2-deployer-plus') in /var/www/shop/vendor/composer/autoload_real.php on line 70
16:32:46 ERROR: failed shelling out to php for reading the module list.
16:32:46 ERROR Details: Command failed: php -r "require '/var/www/shop/vendor/autoload.php'; \$bp = strlen(dirname(dirname(realpath('/var/www/shop/vendor/autoload.php')))) + 1; foreach ((new \Magento\Framework\Component\ComponentRegistrar)->getPaths('module') as \$m) echo substr(\$m, \$bp).PHP_EOL;"

Mind having a look at that one as well? It happened when I was on a branch including module-sth and then switched to the master branch, which does not have module-sth.

Thanks!

sprankhub commented 4 years ago

Addition: Although I saw the errors / warnings, the process at least did not crash :) But maybe it would be nice to suppress these errors / warnings?

Vinai commented 4 years ago

Thanks for reopening the issue. That's actually how it's supposed to be right now, i.e. display the error but recover gracefully. I'll have a look at suppressing the error in notice (default) and info log levels, and only show it in debug mode. I doubt that I'll get around to it before the end of the year though - family time!

Vinai commented 4 years ago

I've partially shortened the error message by default now (in upcoming release 1.18). Still need to spend some time on this to make it nicer. Now that the process doesn't die anymore the issue has a lower priority currently, sorry :)

sprankhub commented 4 years ago

Oh sure, I totally get that - no worries and thanks again!

Vinai commented 4 years ago

I'm having a hard time reproducing the issue. So far I'm doing

  1. composer dump --optimize --no-dev --classmap-authoritative
  2. cache-clean.js -w
  3. rm vendor/magento/module-xxx/registration.php
  4. Edit a file in vendor/magento/module-xxx/etc/

Can you help me find a simple way to trigger the error maybe?

Vinai commented 4 years ago

Please ignore my previous comment, all works. I've spent a little time cleaning up PHP error message display.

  1. Remove the failed PHP cli command from the output.
  2. Only display failure notice in normal log level.
  3. In info log level (-v) display failure notice and first line of PHP error.
  4. In debug log level (-vv) also print full PHP stack trace.

Tested with missing registration.php file and with syntax error in registration.php error.

Released in 1.0.20

sprankhub commented 4 years ago

Thanks @Vinai!