Updating from 1.6.15 to 1.6.20 failed

[drnasin]

I just ran "bin/gpm self-upgrade -y" and got:

$  bin/gpm self-upgrade -y

GPM Releases Configuration: Stable

Grav v1.6.20 is now available [release date: Mon Feb  3 18:05:05 2020].
You are currently using v1.6.15.

Preparing to upgrade to v1.6.20..
  |- Downloading upgrade [3.91M]...   100%
  |- Installing upgrade...
In Folder.php line 393:

  [RuntimeException]  

I ran it second time and got:

$  bin/gpm self-upgrade -y

Warning: require(B:\Dropbox\Ante\Programming\ServerRoot\projects\slomagic.com\vendor\composer/../../system/defines.php): failed to open stream: No such file or directory in B:\Dropbox\Ante\Programming\ServerRoot\projects\slomagic.com\vendor\composer\autoload_real.p
hp on line 67

Call Stack:
    0.2009     398768   1. {main}() B:\Dropbox\Ante\Programming\ServerRoot\projects\slomagic.com\bin\gpm:0
    0.2015     399872   2. require('B:\Dropbox\Ante\Programming\ServerRoot\projects\slomagic.com\vendor\autoload.php') B:\Dropbox\Ante\Programming\ServerRoot\projects\slomagic.com\bin\gpm:21
    0.2238     414184   3. ComposerAutoloaderInit3c504857f50693eae30a724a9742ef76::getLoader() B:\Dropbox\Ante\Programming\ServerRoot\projects\slomagic.com\vendor\autoload.php:7
    0.4051    1096928   4. composerRequire3c504857f50693eae30a724a9742ef76() B:\Dropbox\Ante\Programming\ServerRoot\projects\slomagic.com\vendor\composer\autoload_real.php:57

Fatal error: require(): Failed opening required 'B:\Dropbox\Ante\Programming\ServerRoot\projects\slomagic.com\vendor\composer/../../system/defines.php' (include_path='.;C:\php\pear') in B:\Dropbox\Ante\Programming\ServerRoot\projects\slomagic.com\vendor\composer\au
toload_real.php on line 67

Call Stack:
    0.2009     398768   1. {main}() B:\Dropbox\Ante\Programming\ServerRoot\projects\slomagic.com\bin\gpm:0
    0.2015     399872   2. require('B:\Dropbox\Ante\Programming\ServerRoot\projects\slomagic.com\vendor\autoload.php') B:\Dropbox\Ante\Programming\ServerRoot\projects\slomagic.com\bin\gpm:21
    0.2238     414184   3. ComposerAutoloaderInit3c504857f50693eae30a724a9742ef76::getLoader() B:\Dropbox\Ante\Programming\ServerRoot\projects\slomagic.com\vendor\autoload.php:7
    0.4051    1096928   4. composerRequire3c504857f50693eae30a724a9742ef76() B:\Dropbox\Ante\Programming\ServerRoot\projects\slomagic.com\vendor\composer\autoload_real.php:57

[mahagr]

Looks like the upgrade failed in halfway.

I've seen this bug being reported a few times, but I have no idea what causes it. It would be great if someone had an environment where I could play with this bug as I have never seen it myself.

[drnasin]

should I do a clean install?

[mahagr]

On the live site, you can just manually update. But what I am interested in is to have a way to be able to debug this issue and figure out what causes it.

[drnasin]

@mahagr I don't know if it helps but I managed to get the core updated but when running

bin/gpmm update -y it fails on DevTools

Preparing to install DevTools [v1.2.4] |- Downloading package... 100% |- Checking destination... ok |- Installing package... In Folder.php line 407:

[RuntimeException]

[gizmecano]

I don't know if it can help, but I just encountered a fairly similar warning when I try to upgrade on local environment, i.e.:

$ bin/gpm self-upgrade

Preparing to upgrade to v1.6.20..
  |- Downloading upgrade [3.91M]...   100%
  |- Installing upgrade...
In Folder.php line 427:

  [RuntimeException]

Maybe there is a problem with the permissions, but I couldn't find it.

Oddly, the site and the admin seem to be working properly despite this warning.

[mahagr]

Try changing the last function in src/Grav/Common/Filesystem/Folder.php to:

    /**
     * @param  string $folder
     * @param  bool   $include_target
     * @return bool
     * @internal
     */
    protected static function doDelete($folder, $include_target = true)
    {
        // Special case for symbolic links.
        if ($include_target && is_link($folder)) {
            return @unlink($folder);
        }

        // Go through all items in filesystem and recursively remove everything.
        $files = array_diff(scandir($folder, SCANDIR_SORT_NONE), array('.', '..'));
        foreach ($files as $file) {
            $path = "{$folder}/{$file}";
            if (!(is_dir($path) ? self::doDelete($path) : @unlink($path))) {
                return false;
            }
        }

        return $include_target ? @rmdir($folder) : true;
    }

It will not fix the issue, but it will likely point the file which cannot be deleted.

[mahagr]

Seems to be same as #2670

[gizmecano]

@mahagr wrote :

Seems to be same as #2670

It is indeed possible (I also use Windows). It remains to know whether this is also the case for @drnasin who opened this issue.

[mahagr]

@gizmecano Please check out the code from above. If you change the method into that, it will hopefully tell what went wrong.

[drnasin]

@mahagr I changed the function. Did an update. Core update (1.6.11 to 1.6.21) went ok. It died on Email update:

Preparing to install Email [v3.0.6]
  |- Downloading package...   100%.9%
  |- Checking destination...  ok
  |- Installing package...
In Folder.php line 407:

  [RuntimeException]  

update [-f|--force] [-d|--destination [DESTINATION]] [-y|--all-yes] [-o|--overwrite] [-p|--plugins] [-t|--themes] [--] [<package>.
..]

[drnasin]

UPDATE: I tried resinstalling Email plugin. First few times it died. Then it went through...

[drnasin]

DId one more update which failed but there was no output so I added echo $path and when updating Email it failed and I got

Preparing to install Email [v3.0.6]
  |- Downloading package...   100%.8%
  |- Checking destination...  ok
  |- Installing package...  /tmp/Grav-5e5e657392e06/getgrav-grav-plugin-email-e8e2fea//vendor/symfony/polyfill-iconv/Resources/charset
/tmp/Grav-5e5e657392e06/getgrav-grav-plugin-email-e8e2fea//vendor/symfony/polyfill-iconv/Resources
/tmp/Grav-5e5e657392e06/getgrav-grav-plugin-email-e8e2fea/vendor/symfony/polyfill-iconv
/tmp/Grav-5e5e657392e06/getgrav-grav-plugin-email-e8e2fea/vendor/symfony
/tmp/Grav-5e5e657392e06/getgrav-grav-plugin-email-e8e2fea/vendor

Hope it helps

[mahagr]

That does not really help me. It is not failing where I was thinking it was. More debugging needed. :)

[mahagr]

Try this

    /**
     * @param  string $folder
     * @param  bool   $include_target
     * @return bool
     * @internal
     */
    protected static function doDelete($folder, $include_target = true)
    {
        // Special case for symbolic links.
        if ($include_target && is_link($folder)) {
            $result = @unlink($folder);
            if (!$result) {
                echo "Unlinking link $folder failed!\n";
            }
            return $result;
        }

        // Go through all items in filesystem and recursively remove everything.
        $files = array_diff(scandir($folder, SCANDIR_SORT_NONE), array('.', '..'));
        foreach ($files as $file) {
            $path = "{$folder}/{$file}";
            if (is_dir($path)) {
                if (!self::doDelete($path)) {
                    echo "Deleting folder $folder failed!\n";
                }
            } else{
                if (!unlink($path)) {
                    echo "Deleting file $folder failed!\n";
                }
            }
        }

        if ($include_target && !@rmdir($folder)) {
            echo "Deleting target folder $folder failed!\n";
            return false;
        }

        return true;
    }

[drnasin]

OK, did an update of another grav site (1.6.19). This one went with no problem. This exception is so random.

While I was installing youtube plugin I got

Preparing to install YouTube [v3.0.1]
  |- Downloading package...   100%
  |- Checking destination...  ok
  |- Installing package...  Deleting target folder B:/Dropbox/Ante/Programming/ServerRoot/projects/drustvo-rokus.local/tmp/Grav-5e5ec0ec48dbb/getgrav-grav-plugin-youtube-afb3dca/ failed!

In Folder.php line 407:

  [RuntimeException]  

but if I run it a few times it will eventually go through

[drnasin]

while clearing cache

Deleting target folder B:/Dropbox/Ante/Programming/ServerRoot/projects/drustvo-rokus.local/cache/doctrine/3a26d1fd failed!

[mahagr]

Can you check the folder if there are hidden files or something else?

[drnasin]

Directory is there but "ls -a" says it's completely empty.

[mahagr]

That is completely odd, there should have been a file which prevented folder from being deleted.

I know that in Windows you cannot delete open files because of NTFS / FAT limitations. In Unix (Linux, OSX etc) you can delete open files as deleting just means removing one file descriptor (they can still access the deleted file). Maybe the issue is somehow related to that?

[drnasin]

Hm, @gizmecano how did you do the update? where you using terminal window in phpstorm by any chance?

[gizmecano]

where you using terminal window in phpstorm by any chance?

The comment I posted earlier was about an update I made using the terminal integrated into Visual Studio Code, not that of PHPStorm, sorry.

For now, I haven't had time to test the tips given by @mahagr in order to update to the most recent version.

[mahagr]

@gizmecano They were not tips, just some changes in order to track down what could be happening there.

I'm starting to think that this is some kind of file locking issue in Windows where you cannot delete files which are open either by Grav or in editor or file browser. If this is true, I don't think there's really any good way to fix the issue as it is a limitation in Windows.

[drnasin]

@mahagr my thinking is that while deleting files ide (at least phpstorm) triggers re-indexing and this may cause locking of directories. As I said before I'm not even sure this is Grav issue but I did have a hunch gizmecano was using terminal built in in some ide...

[gizmecano]

They were not tips

My apologies for expressing myself incorrectly. However, I think I understood the idea.

[mahagr]

@gizmecano I meant that they will not to fix the issue, but allows me to track down what could be happening.

@drnasin Can you try if the issue happens if you do not have any application potentially locking the files?

[smz]

@drnasin you probably can use the Handle utility from Sysinternals to check if any process has your files open/locked. See: https://docs.microsoft.com/en-us/sysinternals/downloads/handle

[gizmecano]

I have finally found some free time to test the changes @mahagr had proposed in order to track what could be happening.

This is what I get:

Preparing to upgrade to v1.6.25..
  |- Downloading upgrade [3.92M]...   100%
  |- Installing upgrade...    ok
  '- Success!

Fatal error: Uncaught Error: Class 'Grav\Console\Cli\ClearCacheCommand' not found in [path]\system\src\Grav\Console\ConsoleTrait.php:110
Stack trace:
#0 [path]\system\src\Grav\Console\Gpm\SelfupgradeCommand.php(176): Grav\Console\ConsoleCommand->clearCache(Array)
#1 [path]\system\src\Grav\Console\ConsoleCommand.php(30): Grav\Console\Gpm\SelfupgradeCommand->serve()
#2 [path]\vendor\symfony\console\Command\Command.php(255): Grav\Console\ConsoleCommand->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#3 [path]\vendor\symfony\console\Application.php(917): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#4 [path]\vendor\symfony\console\Application.php(269): Symfo in [path]\system\src\Grav\Console\ConsoleTrait.php on line 110

This time, nor the site or the admin interface are working properly after this update (even if it was announced as successfull) and all seems totally broken...

Therefore, I reverted to the last version (1.6.20) which was correclty working (by using git restore .)

[gizmecano]

I have continued to look for a way to update my current installation (from 1.6.20 with these change in src/Grav/Common/Filesystem/Folder.php.

This time, I tested to manually upgrade to Grav 1.6.25 using the main admin panel button.

I got a popup warning about a missing system file or folder (but I didn't have time to capture it).

After refreshing the admin panel, I obtained a kind of blank page with only:

Please run: bin/grav install

So, I did:

$ bin/grav install
Preparing to install vendor dependencies...

Loading composer repositories with package information
Installing dependencies from lock file
Package operations: 45 installs, 0 updates, 0 removals

[skip the 45 items of the list]

Generating optimized autoload files

ERROR Missing .dependencies file in user/ folder
HINT Are you trying to install Grav? Grav is already installed. You need to run this command only if you download a skeleton from GitHub directly.

But, very strangely, after refreshing the page again, the admin panel seems to work correctly now, indicating that the running version is Grav v1.6.25...

I must admit that I don't understand the problem, nor even if my installation is now correctly up to date...

🤷‍♂️

[gizmecano]

For the moment, I will consider that my project is up to date with the latest version 1.6.25, despite the upgrade issue with CLI mentioned above.

However, in anticipation of any future updates, I keep the file src/Grav/Common/Filesystem/Folder.php modified by @mahagr according to this comment.

[gizmecano]

I don't know how far this issue is resolved, but in order to switch to the recent version 1.6.26, I preferred (for maintenance reasons) to use the interface of the administration panel, and this time everything worked perfectly without any warning. Maybe i will try next time using CLI.

[ricardo118]

when all else fails you can just download the latest version from the website or github and just paste your user file into it

[gizmecano]

when all else fails you can just download the latest version from the website or github and just paste your user file into it

Indeed, this can be a solution.

But as @mahagr thought it could have come from the system (maybe because of hidden or locked files), I took the opportunity to check back this opened issue.