Update failed with `core/shipped.json is not available`

[ronnicek]

Hi,

so I finally get some time to get new version, but update failed (again :)).

1. Start updater
2. Updater failed with gateway timeout from nginx (need to tune that)
3. I push retry - it said step 7 is still running
4. I wait a while until I get endsep of 7, I pushed retry again
5. Ended here :)

Here is the log:

2017-08-15T20:09:55+0000 uilDzvyw2l [info] POST request for step "6"
2017-08-15T20:09:55+0000 uilDzvyw2l [info] startStep("6")
2017-08-15T20:09:55+0000 uilDzvyw2l [info] extractDownload()
2017-08-15T20:09:55+0000 uilDzvyw2l [info] storage location: /var/www/owncloud/data/updater-ocn0mgjslvwm/downloads/
2017-08-15T20:10:50+0000 uilDzvyw2l [info] end of extractDownload()
2017-08-15T20:10:50+0000 uilDzvyw2l [info] endStep("6")
2017-08-15T20:11:07+0000 45vb4zxvXO [info] request to updater
2017-08-15T20:11:07+0000 45vb4zxvXO [info] currentStep()
2017-08-15T20:11:07+0000 45vb4zxvXO [info] Step 6 is in state "end".
2017-08-15T20:11:07+0000 45vb4zxvXO [info] POST request for step "7"
2017-08-15T20:11:07+0000 45vb4zxvXO [info] startStep("7")
2017-08-15T20:11:07+0000 45vb4zxvXO [info] replaceEntryPoints()
2017-08-15T20:11:07+0000 45vb4zxvXO [info] replace index.php
2017-08-15T20:11:07+0000 45vb4zxvXO [info] replace status.php
2017-08-15T20:11:07+0000 45vb4zxvXO [info] replace remote.php
2017-08-15T20:11:07+0000 45vb4zxvXO [info] replace public.php
2017-08-15T20:11:07+0000 45vb4zxvXO [info] replace ocs/v1.php
2017-08-15T20:11:07+0000 45vb4zxvXO [info] replace ocs/v2.php
2017-08-15T20:11:07+0000 45vb4zxvXO [info] end of replaceEntryPoints()
2017-08-15T20:11:07+0000 45vb4zxvXO [info] endStep("7")
2017-08-15T20:11:24+0000 GZpjXPNiuX [info] request to updater
2017-08-15T20:11:24+0000 GZpjXPNiuX [info] currentStep()
2017-08-15T20:11:24+0000 GZpjXPNiuX [info] Step 7 is in state "end".
2017-08-15T20:11:24+0000 GZpjXPNiuX [info] POST request for step "8"
2017-08-15T20:11:25+0000 GZpjXPNiuX [info] startStep("8")
2017-08-15T20:11:44+0000 GZpjXPNiuX [info] deleteOldFiles()
2017-08-15T20:11:48+0000 GZpjXPNiuX [info] config sample exists
2017-08-15T20:11:48+0000 GZpjXPNiuX [info] thmes README exists
2017-08-15T20:12:57+0000 KHPcY5KwSk [info] request to updater
2017-08-15T20:12:57+0000 KHPcY5KwSk [info] currentStep()
2017-08-15T20:12:57+0000 KHPcY5KwSk [info] Step 8 is in state "start".
2017-08-15T20:13:01+0000 GZpjXPNiuX [info] end of deleteOldFiles()
2017-08-15T20:13:01+0000 GZpjXPNiuX [info] endStep("8")
2017-08-15T20:13:06+0000 hqmFdHuUot [info] request to updater
2017-08-15T20:13:06+0000 hqmFdHuUot [info] currentStep()
2017-08-15T20:13:06+0000 hqmFdHuUot [info] Step 8 is in state "end".
2017-08-15T20:13:06+0000 hqmFdHuUot [info] POST request for step "8"
2017-08-15T20:13:06+0000 hqmFdHuUot [info] startStep("8")
2017-08-15T20:13:06+0000 hqmFdHuUot [info] deleteOldFiles()
2017-08-15T20:13:06+0000 hqmFdHuUot [error] POST request failed with other exception
2017-08-15T20:13:06+0000 hqmFdHuUot [error] Exception: Exception
Message: core/shipped.json is not available
Code:0
Trace:
#0 /var/www/owncloud/updater/index.php(1138): Updater->deleteOldFiles()
#1 {main}
File:/var/www/owncloud/updater/index.php
Line:703

2017-08-15T20:13:06+0000 hqmFdHuUot [info] rollbackChanges("8")

But file is here: /owncloud/data/updater-ocn0mgjslvwm/backups/nextcloud-11.0.4.1/core.

Files in /var/www listed (don't worry about rights, it's running in docker container):

total 44
drwxr-xr-x  7 6662 6662 4096 Aug 15 22:13 .
drwxrwxr-x  3 root root 4096 Dec 11  2016 ..
drwxr-xr-x 11 6662 6662 4096 Aug 15 22:11 apps
drwxrwxr-x  2 6662 6662 4096 Aug 15 22:11 config
drwxr-xr-x  2 6662 6662 4096 Oct  4  2016 data
-rw-r--r--  1 6662 6662   57 Aug 15 22:11 index.php
-rw-r--r--  1 6662 6662   57 Aug 15 22:11 public.php
-rw-r--r--  1 6662 6662   57 Aug 15 22:11 remote.php
-rw-r--r--  1 6662 6662   57 Aug 15 22:11 status.php
drwxr-xr-x  2 6662 6662 4096 Aug 15 22:11 themes
drwxr-xr-x  2 6662 6662 4096 Mar  4 11:39 updater

[ronnicek]

Hi, since there is no answer I will revert it back to 11.0.4 tomorrow so I cannot be able to provide more info about this issue if needed :) [need my nextcloud up and running]

[ronnicek]

Hi,

Tried update to 12.0.3 and it went well.. :) so I dont know if you want to close this or not..

[juliusknorr]

My guess is that due to the timeout it already deleted the core/shipped.json and complained that it could not find it on the second run then. I guess the updater should ignore such errors on retry situations.

[ronnicek]

Could be.. timeout is biggest pain of updater I think. When I was trying run updater from commandline I didnt get any response (php7 updater.phar) and it just ended.. So I extend nginx timeout and run updater from web :)

[ITwrx]

@juliushaertl yes, that's exactly what just happened to me. upgrading from 12.0.3 to 13.0.0

devs may consider making the updater fail from various causes and see what happens then. it gets rough pretty quick. i have my own backups and am not dependent on the web gui or i might have been in trouble. it's not like the web updater is going to be able to use it's backup while it's completely borked. i had my max_execution_time set to 120 then 720 and it still got hung up in a few different stages. maybe because i have system() disabled? maybe the updater needs to check more stuff before it tries to run? i kept having to edit the .step files and restart php and the second nextcloud instance never would cooperate.

i'll be using the console updater from now on. i didn't know it was so fancy now. It worked just fine, though it complained about system() at the end.

[skyynet]

I experience this problem since years. I'm on a shared hosting environment but don't know the file system. My solution is to copy core/shipped.json to the installation folder with my FTP client, every time the updated complains, it's missing. I just don't understand, why the failure of deleting a file which is no longer present causes the updater to throw an error. Why not simply continiue?