Add auto-compatibility with WordFence for our background processes

[Tabrisrp]

Describe the bug We have seen cases where WordFence is preventing our background processes from running correctly.

WordFence has a learning mode that can be used to allow the background processes. But It seems to be possible to programatically enable them by using some compatibility code. We need to explore this option to simplify our users experience.

Backlog Grooming (for WP Media dev team use only)

• [x] Reproduce the problem
• [x] Identify the root cause
• [x] Scope a solution
• [x] Estimate the effort

[iCaspar]

Grooming Notes:

Reproduce the Issue

Issue has been reproduced on multiple sites in various contexts.

Root Cause

WordFence Firewall blocks unknown requests that it considers risky, except when in "Learning Mode" during which time such requests are added to an allow list for later review by site admins. If WPRocket's background requests are not added during a Learning Mode period, the firewall blocks them.

Scope for Solution

The WordFence API includes (starting WF version 5.3.2 a wordfence::whitelistIP() method to programatically whitelist IPs used by themes and plugins. @see WordFence API Documentation.

We need a new compatibility class Engine\ThirdParty\Plugins\Security\WordFence that implements Subscriber_Interface. In this class, we need a const with our IPs to be whitelisted, and a method whitelist_wordfence_firewall_ips() that will call wordfence::whitelistIP() with each of our IPs. The class's get_subscribed_events() will hook whitelist_wordfence_firewall_ips() to all Rocket background processes needing one of the whitelisted IPs.

Will need relevant Unit and Integration tests for the new class.

Note, since this is an IP-dependent solution, we will need to update the IPs in the class constant (requiring a release) whenever our IPs are changed. It may be indicated that we should provide a filter in whitelist_wordfence_firewall_ips() in case there is a need to substitute other IPs on a temporary basis or if we have an IP change and expect a delay in releasing.

Estimted Effort

[M]

[or128]

WP-Cerber is also blocking the warmup requests. Can WP-Rocket become compatible with Cerber?

https://wpcerber.com/

[vmanthos]

This is still an issue.

On a customer's website resources weren't fetched until I activated the "Learning mode" in Wordfence.

The SaaS IP was in Wordfence's Allowlisted IPs:

Since this is about fetching the resources, Wordfence likely blocks the requests that we perform to the server the site is hosted on. So, allow-listing the IP of the SaaS won't make any difference, while doing that for the server's IP might. Unfortunately, I enabled the learning mode before trying this.

I've noticed that Wordfence has an "Allowlisted URLs" section on the options page. We can try that if they offer respective hooks.

Just a couple of observations about this. The following was already there, but WP Rocket's AJAX requests didn't appear after activating the "Learning mode":

I'm not sure if this means our requests were not allow-listed here or not.

PS: I'll ask the RUCSS team, the next time they come across such an issue to allow-list the server's IP and check if fetching proceeded. We'll report back here.

Related ticket: https://secure.helpscout.net/conversation/1697146380/307976/

[vmanthos]

On the same site👆, resources stopped being fetched when I disabled the "Learning mode". I had the chance to check if allow-listing the server's IP would resolve that, and it did.

To rule out this was random, we need to test it on other sites too.

[vmanthos]

I had another case where allowlisting the server's IP allowed the fetching of resources.

@piotrbak I'd say this is not random and we should move forward with it.

Ticket: https://secure.helpscout.net/conversation/1720845757/313287/

UPDATE

Here is a screenshot of the blocked requests in WordFence's log:

[mehedihasanziku]

tried to allow server ip like @vmanthos said, but didn't stopped logging but strange to notice even with that logging used css added but not always as i checked in multiple sites.

although it's not good to use: reason told by WF support

Note: Such issue happens only If Web Application Firewall Status "Enable & Protecting", in learning mode no issue like that but it's filling up rules list.

[piotrbak]

There's no feedback from customers regarding this one for 2 years. It's not going to be implemented in the near future. We're open to discuss and reopen the issue though.