Damaged zip file

[Nobiuss]

HI Team,

We are having sometimes troubles with our zip file.

Here the error that the terminal is giving on the composer update.

` - Installing */yith-woocommerce-checkout-manager (1.2.8): Downloading (100%)
Failed to execute unzip -qq 'wp-content/plugins/yith-woocommerce-checkout-manager//410bce36adda1299be1b03e679b9a6f5.8' -d '/Users/kimvanrijckel/Local Sites//app/public/vendor/composer/71771147'

[wp-content/plugins/yith-woocommerce-checkout-manager//410bce36adda1299be1b03e679b9a6f5.8] End-of-central-directory signature not found. Either this file is not a zipfile, or it constitutes one disk of a multi-part archive. In the latter case the central directory and zipfile comment will be found on the last disk(s) of this archive. unzip: cannot find zipfile directory in one of wp-content/plugins/yith-woocommerce-checkout-manager//410bce36adda1299be1b03e679b9a6f5.8 or wp-content/plugins/yith-woocommerce-checkout-manager//410bce36adda1299be1b03e679b9a6f5.8.zip, and cannot find wp-content/plugins/yith-woocommerce-checkout-manager//410bce36adda1299be1b03e679b9a6f5.8.ZIP, period.

The archive may contain identical file names with different capitalization (which fails on case insensitive filesystems)
Unzip with unzip command failed, falling back to ZipArchive class

[UnexpectedValueException]
'wp-content/plugins/yith-woocommerce-checkout-manager//410bce36adda1299be1b03e679b9a6f5.8' is not a zip archive.

` Here a screenshot from another plugin. You see on the file size that there is something wrong.

Kind regards, KIM

[bradyvercher]

It's hard for me to tell what's going on here. It looks like your errors are related to yith-woocommerce-checkout-manager, but the screenshot is referring to yith-woocommerce-customize-myaccount-page. Are you having problems with both of those?

Regarding yith-woocommerce-checkout-manager, have you tried downloading the zip for version 1.2.8 manually and inspecting it? Go to Settings → SatisPress and click the Packages tab to download it.

Also, do you know if the zip artifact for 1.2.8 was created from the installed source or downloaded from the update server? Usually, only the very first release you cache will be created from source, all the other releases should come directly from the vendor.

[GaryJones]

Does it happen with other plugins / zip files?

Can you share some details from your setup, so we might be able to try and reproduce the error?

[Nobiuss]

This happens sometimes. On several plugins. So the screenshot was indeed not relevant with my comment. We delete it to continue our work. Here is a screenshot with the info about the site.

We use satispress 0.3.2

Here a download link for a plugins where we noticed the same issue. link version2.1.7 link version2.1.3

API key => 'vFNl7L2NsZT8SxOh1UQg7gfCw8eeBasY' Password you will know :-)

Kind regards, KIM

[bradyvercher]

That's a lot of plugins! That screenshot says most of them are inactive, though. How do you keep them updated? That might help determine whether the zips are being corrupted when downloading an update or if there's an issue zipping the files up the plugins directory.

I checked out the two plugins from Yith and those zips appear to be working. Did you do something to update them?

I do see an issue with the Youzer plugin you linked to. Did you update that manually or through the Envato Market plugin?

[bradyvercher]

@Nobiuss Have you had any other problems related to this or can I close it out?

[koengabriels]

@bradyvercher Nobiuss asked me to look into the issue, here is what I found out so far: I activated the Envato Market plugin upon which SatisPress attempted to get the latest version for Youzer.

I added a var_dump after

$filename = $this->get_absolute_path( $release->get_file() );
$tmpfname = download_url( $release->get_source_url() );

in the Archiver class and the download returns an error:

/app/public/wp-content/plugins/satispress/src/Archiver.php:134:
object(WP_Error)[4290]
  public 'errors' => 
    array (size=1)
      'http_request_failed' => 
        array (size=1)
          0 => string 'cURL error 60: SSL certificate problem: self signed certificate' (length=63)
  public 'error_data' => 
    array (size=0)
      empty

If you need more test results on our end just leave a comment and I will provide them asap

[bradyvercher]

Thanks, @koengabriels. I would have to do some testing, but based on that, it doesn't look like SatisPress should be creating a zip file at all. Instead it should log a message and fail. When you get that error, is it getting past this check?

[koengabriels]

@bradyvercher it is indeed getting past that check, the is_wp_error does return true but it doesn't seem to be throwing the Exception as it does go through the entire function...

Not sure what might be causing that.

[bradyvercher]

Hmm, that's strange. If you have WP_DEBUG enabled, does it log an error message in debug.log? I'm not sure what would cause it to continue stepping through the function like that.

[koengabriels]

Very strange indeed, I checked the log and the only entry (other than the one I put inside of the check where it should throw the FileDownloadFailed exception is the following one, I don't see any relation to satispress though and it's just a notice:

[11-Jun-2019 08:44:58 UTC] PHP Notice:  map_meta_cap was called <strong>incorrectly</strong>. The post type scheduled-action is not registered, so it may not be reliable to check the capability "read_post" against a post of that type. Please see <a href="https://codex.wordpress.org/Debugging_in_WordPress">Debugging in WordPress</a> for more information. (This message was added in version 4.4.0.) in /app/public/wp-includes/functions.php on line 4231
[11-Jun-2019 08:44:58 UTC] PHP Stack trace:
[11-Jun-2019 08:44:58 UTC] PHP   1. {main}() /app/public/wp-admin/index.php:0
[11-Jun-2019 08:44:58 UTC] PHP   2. wp_dashboard() /app/public/wp-admin/index.php:134
[11-Jun-2019 08:44:58 UTC] PHP   3. do_meta_boxes() /app/public/wp-admin/includes/dashboard.php:218
[11-Jun-2019 08:44:58 UTC] PHP   4. wp_dashboard_site_activity() /app/public/wp-admin/includes/template.php:1215
[11-Jun-2019 08:44:58 UTC] PHP   5. wp_dashboard_recent_comments() /app/public/wp-admin/includes/dashboard.php:779
[11-Jun-2019 08:44:58 UTC] PHP   6. current_user_can() /app/public/wp-admin/includes/dashboard.php:907
[11-Jun-2019 08:44:58 UTC] PHP   7. WP_User->has_cap() /app/public/wp-includes/capabilities.php:623
[11-Jun-2019 08:44:58 UTC] PHP   8. map_meta_cap() /app/public/wp-includes/class-wp-user.php:722
[11-Jun-2019 08:44:58 UTC] PHP   9. _doing_it_wrong() /app/public/wp-includes/capabilities.php:221
[11-Jun-2019 08:44:58 UTC] PHP  10. trigger_error() /app/public/wp-includes/functions.php:4231

[Nobiuss]

Hi Team,

Any update on this?

We are still having this issue, mostly on YITH plugins.

Kind regards, Kim.

[bradyvercher]

Hi @Nobiuss, I haven't ever been able to replicate this and there haven't been any additional reports, so it's not something I've been able to look into further.

At this time, it sounds like it's either an issue with your environment or the specific plugins you're using. Based on the feedback provided by @koengabriels, it sounds like it could be an issue with the SSL certificate used by Yith to deliver updates, but I'm not sure why that's not throwing an exception in the code I linked to.

[ricsmo]

I'm seeing the exact same issue.

The plugin is downloaded from the vendor and updates fine, but the zip file in the cache is much smaller and corrupt, just like in the screenshot in the first post.

To fix I manually delete the corrupt file, disable satispress for that plugin and re-enable it so it builds the zip from the successfully extracted plugin directory. But that's more work than using GitHub repos and the scripts I used to use.

As far as environment, this server is on AWS Lightsail using the Bitnami WP image.

I also noticed the affected plugin zips are coming from s3 signed urls, if that helps at all.

[ricsmo]

I looked at the corrupted "zip" file and it's the html from wp-login.php with a redirect to the plugin settings page so this plugin doesn't seem to allow a second download or one outside of the WP update process (haven't dug into the code to find out).

I'm going to modify the satispress code a bit on my installation so it only archives from source and see if that fixes it.

src/ReleaseManager.php Line 91

// $filename = $this->archiver->archive_from_url( $release ); $filename = $this->archiver->archive_from_source( $package, $release->get_version() );`

If it solves the problem maybe adding a checkbox in the settings to always force archive from source can be added?

I'd actually prefer some extra CPU time instead of double bandwidth.

[bradyvercher]

Thanks for confirming that you're experiencing the same issue, @ricsmo.

SatisPress uses the same URL provided to the WP update process, which should be publicly accessible. It also uses the same download_url() method used by core to download packages.

Downloading the package from the vendor is preferred over zipping from source when possible. New releases can be cached as they become available, but when using the installed source the plugin has to be updated before it can zipped.

That's interesting that you're seeing the source for wp-login.php in the zip. Is that for login page local to the install where SatisPress is installed? Or is it the login screen for the vendor's site?

[ricsmo]

It's the login page for my installation that hosts SatisPress.

[bradyvercher]

Hmm, that's really strange. I'm not sure why the local wp-login.php page would ever come into play.

Somewhere in the Archiver::archive_from_url() method is the most likely place where something is going wrong. If one of y'all that are experiencing the issue could add some logging to that method to pinpoint what exactly the issue is, that'd help immensely.

[ricsmo]

I just checked _site_transient_update_plugins and the urls being stored by this vendor aren't normal.

Instead of links to the .zips, it's a dynamic link back to the plugin (I'm assuming to grab a new expiring s3 url or other overly-protective auth, but haven't checked).

I think this plugin only expects someone to update by doing it from the admin so, since there's not a logged in user when that method is called, the login page is shown with the same link as a redirect.

Seems like a really nonstandard way of doing things.

Plus, if SatusPress downloads the zips before the update is made as you said above, then I'll probably get the html as zip issue even with my small code change.

I guess it's back to a GitHub repo and manual updates, at least for the offending vendor.

[711st]

Hi! I'm seeing the same issue here. Every time a plugins is updated (from the Wordperss dashboard) satispress generates a damaged file instead of the correct one (always 6KB). If I remove the damaged file, uncheck that plugin from the satispress column in the plugins directory and re-check it again satispress generates the zip correctly.

[bradyvercher]

Hi @711st! When the zip file doesn't exist and you toggle the checkbox like that, SatisPress zips up the plugin directory that's installed locally and caches that.

However, subsequent releases should come directly from the vendor as delivered to WordPress. Try checking the update_plugins site transient to see what the URL is for the package key. It should be a direct link to a publicly accessible zip file.

[ricsmo]

@711st please try to open the "damaged" zip file (maybe after unzipping) in a text editor and report what you see. In my case it was the html for the Wordpress login page, which meant the plugin being updated wanted a logged in user, gave the login page, and SatisPress tried to zip that because it wasn't logged in.

[ricsmo]

@bradyvercher I think that an increasing number of paid plugin developers are using very nonstandard ways of putting API urls that need authentication into that transient.

[bradyvercher]

@ricsmo WordPress itself doesn't provide any way to authenticate before downloading an update, so external APIs shouldn't see a difference between a request from SatisPress or one from WordPress. They should appear identical unless individual plugins are modifying the upgrade process.

I'd be curious to look further into the plugin you're having issues with and seeing if it's something that can be resolved. Feel free to email me directly. Any additional information would be helpful.

[martyp187]

I am getting the same with all woocommerce produced plugins... its making satispress unusable. The same as previously mentioned. If you rename the .zip to .txt and open it, its html for the wordpress login page.

Maybe I miss understood, but why does Satispress not zip the files from the plugin folder and deliver those?

[bradyvercher]

@martyp187 SatisPress does zip the files from the plugin directory when a plugin is first whitelisted, but when updates become available, those are downloaded directly from the vendor. It's done that way so that new releases can be cached immediately. Otherwise, you would need to manually update a plugin for the code to exist in the plugin directory and cache a new version.

Theoretically, there's more that could go wrong while zipping from source, so the file directly from the vendor should also be more reliable.

I haven't ever been able to replicate the issue here, so someone that's experiencing it will need to debug to determine why the local login page is being cached.

[martyp187]

I understand. But as others mentioned, the creative ways developers are attempting to paywall their gpl code means this might be an issue that will not go away. So triggering automatic updates on the WordPress instance may be the best we can have.

[bradyvercher]

@martyp187 This isn't something I've been able to replicate or see for myself and no one has reported exactly what's happening, so I can't be sure it's due to some sort of custom authentication scheme. Considering WordPress itself wouldn't be able to handle anything other than a publicly accessible URL, I'm thinking it's more likely there's some sort of configuration or communication issue.

[joejordanbrown]

I just faced this exact issue.

My issue was caused when using the Envato Market plugin for updating a purchased theme/plugin.

Steps to reproduce the issue...

1. Install a purchased plugin that's not the latest version for testing purposes.
2. Install the Envato Market plugin found at https://envato.com/market-plugin/ and link with an account that has the purchased theme/plugin.
3. Run the Envato Market update plugin from https://example.com/wp-admin/admin.php?page=envato-market#plugins
4. View the source of the newly installed plugin at /uploads/satispress-**/packages/plugin-name/plugin-name-1.0.1.zip you will find the following text instead of a valid zip archive.

Archive:  plugin-name-1.0.1.zip
  End-of-central-directory signature not found.  Either this file is not
  a zipfile, or it constitutes one disk of a multi-part archive.  In the
  latter case the central directory and zipfile comment will be found on
  the last disk(s) of this archive.
unzip:  cannot find zipfile directory in one of plugin-name-1.0.1.zip or
        plugin-name-1.0.1.zip.zip, and cannot find plugin-name-1.0.1.zip.ZIP, period.

The fix to make satispress work with 3rd party update systems that use authentication for the update downloads would be to detect an invalid zip archive and then zip from local source instead. Possibly an option to also flag plugins/themes to use source instead of download too.

[Rarst]

Also just hit an invalid zip in client project.

In my case zip ended up as text file with a message about invalid API key, so I am guessing vendor returned that instead of a real download?

Might be worth adding a check if zip is valid before exposing it as package.

[visata]

The same issue here. I uploaded some initial plugin versions manually and it was archived by Satispress fine. Plugins are updated via some 3rd party plugin. After the updates were received (by 3rd party plugin), Satispress generated invalid zip file with the size of 7KB. When I opened the file via text editor, it was an index.php file.

[visata]

I double checked how Satispress works and it seems I know why I'm experience this issue.

"When an update for a managed plugin or theme becomes available, SatisPress downloads and saves it alongside previously cached releases"

If it uses the url directly from the vendor, most likely that's why I experience this issue. Vendors use different authentication methods which might not be supported by Satispress. The issue would be solved, if Satispress archived the currently (usually it's the latest) installed version instead of downloading the latest version from the vendor's url.

@bradyvercher Is this something that can be supported?

Thanks

[martyp187]

@visata This has been around a long time, and a few others above have mentioned your solution as being something that's better than nothing, but almost a year later, I don't think we are going to see the solution here. @joejordanbrown proposed solution sums it up. It's likely not reasonable to cope with all different vendors authentication possibilities, so we should just zip source when it's verified updated by the version number in the plugin file.

[sdrib]

Might be worth nothing we had the exact same issue and removing the whole vendor folder and running composer again fixed this.

[visata]

Might be worth nothing we had the exact same issue and removing the whole vendor folder and running composer again fixed this.

Yes, I did the same, it fixed the damaged release but it will generate the same damaged file with the next release.

I also did some quick code changes to the Satispress plugin. Not sure if it helps but I changed from:

satispress/src/ReleaseManager.php

 if ( ! empty( $source_url ) ) { 
    $filename = $this->archiver->archive_from_url( $release ); 
 } elseif ( $package->is_installed() && $package->is_installed_release( $release ) ) { 
    $filename = $this->archiver->archive_from_source( $package, $release->get_version() ); 

To:

 if ( ! empty( $source_url ) ) { 
    $filename = $this->archiver->archive_from_source( $package, $release->get_version() ); 
 } elseif ( $package->is_installed() && $package->is_installed_release( $release ) ) { 
    $filename = $this->archiver->archive_from_source( $package, $release->get_version() ); 

I expect it to compress the source even if the link to the plugin is not empty.

[bradyvercher]

I'm not sure which plugins folks were having issues with, but I pushed up a new branch that will allow adapters to be written for non-standard update routines. Right now it only has an adapter for the Envato Market plugin, but additional adapters shouldn't be difficult to create. If anyone would like to test it out and provide feedback, that'd be great!

[bradyvercher]

The latest release contains both validators and an adapter for plugins updated from the Envato Market plugin. I believe that should resolve most of the issues raised here. Please open a new issue if you run into any problems.

[martyp187]

Hi @bradyvercher

Just to report, I have given the latest version a go, and it fixed all the issues I had and I'm yet to have this issue again from any of my plugin sources. Thank you!

[bradyvercher]

@martyp187 Awesome! Glad to hear it's working for you. Thanks for trying it out and letting me know.