search

wp-media / wp-rocket

Performance optimization plugin for WordPress
https://wp-rocket.me
GNU General Public License v2.0
701 stars 218 forks source link

Htaccess corrupted on update #928

Closed webtrainingwheels closed 6 months ago

webtrainingwheels commented 6 years ago

A few customers report that on occasion, typically when updating, the htaccess file is regenerated with only with our rules and nothing else, creating 404 errors. It doesn't seem to happen every time and seems almost impossible to reproduce, yet we have several reports now. Is there anything we can do to make sure that at least the default WP rules are placed back into the file? HS tags: htaccess, update fail, error 404 Intercom report: https://app.intercom.io/a/apps/llnz7wr8/inbox/inbox/46787/conversations/15589780670

blogtutor commented 6 years ago

This just happened again on one of my clients' sites, updating from 3.0.5.1 to 3.1.

The .htaccess file was left with only the WP Rocket 3.1 directives, stripping out other redirects and the standard WordPress .htaccess code.

As a result, every page on her site was a 404 error, except her homepage, resulting in downtime of about two hours until it was reported to us and fixed. :(

I know this is tough to reproduce, but we really need a solution here. Maybe every time WP Rocket makes edits to the .htaccess file, it can make a local backup, and then compare the new/old files, and then take action if the code outside of the WP Rocket code has changed?

ecotechie commented 6 years ago

Same problem here. Today is the fourth time this happens to one of the sites I maintain. Same as above, I updated to 3.1.1 and all directives were removed except the WP Rocket and WordPress ones. Seems like a problem with the code updating the file from previous versions.

blogtutor commented 6 years ago

@ecotechie Are you sure the WordPress directives weren't removed? I think Cloudways may have already added that back in by the time you got to it. In the past when I've encountered this issue, only the WP Rocket code was there - everything else was gone.

ecotechie commented 6 years ago

@blogtutor that may have been the case as there were many fingers on that file when the site stopped loading correctly.

blogtutor commented 6 years ago

Ack! This just happened again to another client. Last night she updated to WP Rocket 3.1.2 - and her .htaccess file ended up with only the WP Rocket directives, and everything else stripped out.

Her site was down (except for her home page) for over 12 hours because of this...and I've also had to pull a copy of her .htaccess file from a backup in order to restore the rest of the directives that were in the file. (Re-saving the permalink settings gets the site back up, but we had other stuff in the file that was lost when WP Rocket stripped everything out.)

@Tabrisrp & @webtrainingwheels, is there any way to get this bumped up on the priority list?

blogtutor commented 6 years ago

OMG. Yet another client just ran into this issue! It's a good thing we've got backups of their .htaccess files...

This used to only happen very rarely... this latest update seems to have a much higher rate of occurrence. :(

remyperona commented 6 years ago

Are they all hosted on the same hosting provider?

The only function that write to the htaccess is this one https://github.com/wp-media/wp-rocket/blob/master/inc/functions/htaccess.php#L13

And the part that is doing a removing of content is this $ftmp = preg_replace( '/# BEGIN WP Rocket(.*)# END WP Rocket/isU', '', $ftmp );

For some reason, it might be that this regex is not working as expected, removing all content instead of only the content between the WP Rocket markers.

a good test to determine if it's indeed the issue would be to check for the regex matching against one of the htaccess affected, on a site like https://www.phpliveregex.com/ for example.

blogtutor commented 6 years ago

HI @Tabrisrp - thanks for jumping in on this.

Most (maybe all) of the sites where we've encountered this are hosted on Cloudways, either Vultr or Linode servers.

And yep, it sure does sound like that preg_replace function is stripping everything out.

Here's one of the .htaccess files that was corrupted (I had to rename it to upload here.): htaccess.txt

I tinkered with the phpliveregex.com site, and it seems to work fine on that .htaccess file: https://www.phpliveregex.com/p/p1o (gotta switch to preg_replace, but then it works)

remyperona commented 6 years ago

Yes indeed it works correctly for this htaccess file. The issue might be later in function.

Are you able to reproduce the issue everytime if you update/rollback between versions, or is it happening randomly?

blogtutor commented 6 years ago

Thanks @Tabrisrp.

We haven't attempted to reproduce the issue on any sites (since these are live sites, after all, and we just wanted to get them working again). If it happens again on another site, though, we can give that a quick try.

At this point my hunch is that it's either (1) related to a specific server setting, and/or (2) the method in which the update is run. (Batch updates page vs. plugins page -- and if other updates are run at the same time). I just emailed the two most recent clients to see if they remember exactly how they ran the updates.

Also, I just emailed some more sample .htaccess files to Vasilis, per his request. Though there's not too much customization stuff in them, and they're very similar to many other clients who haven't had issues (such as http->https redirects).

blogtutor commented 6 years ago

Heard back from one client about her upgrade process:

I did the updates from the Dashboard> Updates. I did update several plugins at once, they were: Wp Rocket, SEO by Yoast and Social Warfare (I remember these exactly). Also Social warfare pro had update but it did not work. So out of 4 plugins, 3 updated and WP Rocket messed up.

remyperona commented 6 years ago

Currently I'm leaning toward server related issue as, AFAIK, all the cases we had for this were on Cloudways.

The resulting htaccess does contain the WP Rocket rules, but the rest is stripped out, correct?

blogtutor commented 6 years ago

All of the recent cases have definitely been on Cloudways. The ones that happened weeks/months ago I don't remember for sure, but odds are good they were also Cloudways.

And yep, the resulting.htaccess contains the new WP Rocket rules (with the newest version number), and that's it.

blogtutor commented 6 years ago

This just happened again to another client. Same exact situation. She's on Cloudways hosting. She updated to 3.1.3 and ended up with an .htaccess file that had only the WP Rocket stuff -- everything else was stripped out. Attached are before/after .htaccess files.

She said: "I was on the 'All Plug-ins' page. I updated one plug-in first (I can't remember which) after checking the first box on the UpDraft pop-up. Then I updated the WP Rocket plug-in, but first unchecked the box on the pop-up so I didn't have to wait for additional backup to run."

(So it sounds like she ran plugin updates one-at-a-time, and everything updated correctly, so it seemed.)

htaccess_original.txt htaccess_broken-by-update.txt

blogtutor commented 6 years ago

@Tabrisrp The problem seems to be getting worse. Over the holiday weekend this happened again on (at least) two more client sites when they updated WP Rocket. Both are on Cloudways.

Not sure if this info helps, but: One is running:

The other site was running:

We are now desperate for a solution and/or workaround here... how can I help get this resolved?

At the very least, can WP Rocket save a backup of the .htaccess file locally, before making the edits? Then at least restoring will be much quicker/easier.

Thanks!

remyperona commented 6 years ago

I'm talking with Cloudways about this too, but they are not able to reproduce this on their side for now. I believe it would be really helpful if you also contact them, and provide them the list of websites where it's happening, because the only recent reports of this issue on our side are coming from you.

blogtutor commented 6 years ago

Thanks Remy. I will reach out to them -- with whom have you been talking? I'll try to connect to the same person.

(It also just happened on another client's site.)

Hey, might it be a file permissions issue? On Cloudways, the master user on the server is something like masterxndjwemw -- but WordPress runs under its own user (whatever the "application" username is). I run into hassles a lot when the WordPress user creates files (such as when installing a plugin from the repo) that are then owned by the application user name -- and then when I use FTP, logged in with the master account, the master_ account doesn't have sufficient permissions to edit those files. Maybe that has something to do with all this?

blogtutor commented 6 years ago

Unfortunately, this continues to be a significant problem; it happened again on another clients' site yesterday, and again this morning on a different site.

Both Cloudways support and I have not been able to replicate the problem with any consistency.

We need to find some fix, or at least a workaround, for this. This problem takes down sites completely (except for the homepage). Many of my clients are very high-traffic sites and downtime costs them hundreds of dollars an hour in lost revenue.

At this point, since we can't isolate the root cause of the problem, can you please at least add some error-checking so that in case it happens, there's an automatic solution in place?

blogtutor commented 5 years ago

@Tabrisrp - This happened yet again on another client's site this morning - she updated WP Rocket and boom, everything else in her .htaccess file was gone. Same exact issue, though I hadn't seen this site affected before. It's also on Cloudways.

At this point, since we can't isolate the root cause of the problem, can you please at least add some error-checking so that in case it happens, there's an automatic solution in place?

vmanthos commented 5 years ago

A customer has a site on Cloudways. I've updated WP Rocket from 3.1.4 to 3.2.2 without an issue.

However, when I opened the htaccess file using Notepad++ I saw this:

image

Above the NULs there were WP Rocket's htaccess rules.

I removed the NULs, saved it to a new file and diff checked the two htaccess files using Meld:

image

The original htaccess can be downloaded from this link: https://mega.nz/#!SpVUnS5S!42C5hVJlEXhL5EssO6QtDtKIc2Z_cho2UiRlykIDlK0

This may or may not be related to this particular issue. However, it's very strange and that's why I'm reporting it here.

Related ticket: https://secure.helpscout.net/conversation/711583715/86840?folderId=2135277

blogtutor commented 5 years ago

Thanks for adding this! For what it's worth, I haven't seen this particular issue before on any of our sites that have had update issues with the rest of .htaccess getting stripped out.

webtrainingwheels commented 5 years ago

Possible case: https://secure.helpscout.net/conversation/717560494/88121?folderId=273761

blogtutor commented 5 years ago

Thank you!! 😃

blogtutor commented 5 years ago

Just had another client on Cloudways with this issue. It's been happening less often, but clearly it's still a problem.

😢

blogtutor commented 5 years ago

Yep, just happened again with a different site. Client updated, and boom, the .htaccess file lost everything except the (new) WP Rocket code.

blogtutor commented 5 years ago

...and it's still happening. Another client on Cloudways updated from v3.3.3 to v3.3.3.1 ... same exact problem.

@webtrainingwheels any other ideas?

blogtutor commented 5 years ago

@webtrainingwheels @Tabrisrp Can you please reopen this ticket? It's definitely not solved, unfortunately.

blogtutor commented 5 years ago

Just happened again on another site. Also hosted on Cloudways. Updated 3.3.2 to 3.3.3.1.

Interestingly, this site was updated with Staging Pilot -- which updates on a staging server first and then pushes the updated plugin files to the live site. I think this is the first time a Staging Pilot update has run into the issue.

blogtutor commented 5 years ago

...and just happened again, on another site that was updated via Staging Pilot.

This was the third site to have this issue on the most recent 3.3.3.1 update.

ecotechie commented 5 years ago

Hi @Tabrisrp seems like the commit that closed this issue is only checking for the standard WordPress copy in the .htaccesss file and not any of the custom code that gets stripped. Is this correct? If so, it won't solve this issue. Also I just had a .htaccess file stripped on another site and it took all the directives with it except the WP Rocket ones.

vmanthos commented 5 years ago

A case which seems similar: https://secure.helpscout.net/conversation/977417384/126315/

This was on Dreamhost.

blogtutor commented 4 years ago

This is still a problem. We just updated a client from 3.4.4 to 3.5.1. Checked the site after the update and all seemed fine. A couple of hours later she emailed us because only her homepage was working. Sure enough, the .htaccess file had been stripped of everything except the WP Rocket code.

Happened on another site last week as well.

goddas commented 4 years ago

I've just had a similar issue that might be related to this. My client contacted me with the website being down and only the homepage working. I've checked the htaccess file and it was missing about bottom-500 of the 1000-something lines. So the first 500 or so lines contain the WP Rocket code and some redirects, while the other half containing more redirects and wordpress directives was missing. The website is hosted on cloudways/digital ocean server

mifrero commented 4 years ago

Related ticket: https://secure.helpscout.net/conversation/1192263109/172159/

ecotechie commented 4 years ago

Similar issue with another site. .htaccess was 636 lines, but the update stripped all but the first 134. Left behind the new WP Rocket directives and some Redirect 301 lines. This seems kinda new, since it used to strip ALL directives except the new WP Rocket ones, but this left some lines behind, like what happened to @goddas above.

Happy to email the files if wanted....

webtrainingwheels commented 4 years ago

https://secure.helpscout.net/conversation/1297510459/198847?folderId=377611 It's happened to this customer for the last 2 updates. Hosted on 02switch. It's never reproducible after the first update attempt. Rolling back and updating again will be successful. What's happening from our side only in the very first update routine for each version?

vmanthos commented 4 years ago

@blogtutor Reported this is still occurring for them: https://secure.helpscout.net/conversation/1302688547/200372?folderId=3326173

vmanthos commented 4 years ago

Possibly related ticket: https://secure.helpscout.net/conversation/1319380832/205461/ The site is hosted on OVH.

[WIP] I'm investigating a similar case that took place on OVH: https://secure.helpscout.net/conversation/1302165472/200249/#thread-3776342425

Please find the customer's comment and my findings in the note here: https://secure.helpscout.net/conversation/1319380832/205461/#thread-3804061230

vmanthos commented 4 years ago

Update: I heard back from one of the customers (ticket).

I had prevented WP Rocket from writing to the htaccess file on their site. Despite that, the file was corrupted after a few weeks. Other plugins were writing in that file.

Most likely this was related to the host, OVH in this case. If a process takes too long, it may timeout, and that can corrupt the file.

DahmaniAdame commented 3 years ago

Possibly related - https://secure.helpscout.net/conversation/1335422175/211113/

vmanthos commented 3 years ago

Possibly related: https://secure.helpscout.net/conversation/1366892923/223063/

NataliaDrause commented 3 years ago

happened for customer again when updated to 3.8: https://secure.helpscout.net/conversation/1368024563/223391

blogtutor commented 3 years ago

This is continuing to happen for our clients on Cloudways servers. I cannot emphasize enough how problematic this is! It takes the entire site (save the homepage) offline, and leads to an extremely panicked site owner, often requiring emergency response/support at off-hours.

PLEASE PLEASE PLEASE FIX THIS! I BEG YOU.

blogtutor commented 3 years ago

Also, have you actually talked with Cloudways about this? Clearly there's something unique about their setup -- even it is "just" nginx in front of apache... Their frontline support isn't going to be helpful, but perhaps you could work with one of their senior techs or developers?

I have emailed with a couple of Customer Success Managers and Associate Vice Presidents there -- happy to send along their contact info if that would be helpful.

vmanthos commented 3 years ago

Related ticket: https://secure.helpscout.net/conversation/1366892923/223063/ Host: O2switch

vmanthos commented 3 years ago

Related ticket: https://secure.helpscout.net/conversation/1374886981/225451?folderId=3326173 Host: Hetzner.

vmanthos commented 3 years ago

Related ticket: https://secure.helpscout.net/conversation/1389568405/229477?folderId=2135277 Host: Dreamhost

NataliaDrause commented 3 years ago

Related ticket: https://secure.helpscout.net/conversation/1391871951/230082?folderId=3864740 Host: WPopt, which uses Litespeed web server

NataliaDrause commented 3 years ago

Related ticket: https://secure.helpscout.net/conversation/1392031396/230125?folderId=3864740 Host: OVH

vmanthos commented 3 years ago

Potentially related ticket: https://secure.helpscout.net/conversation/1401738049/232928/