3.16 Preload - Shouldn't fetch RSS feed or restAPI links from the Homepage

[hanna-meda]

Before submitting an issue please check that you’ve completed the following steps:

• Made sure you’re on the latest version - 5.16-alpha1 ✓
• Used the search feature to ensure that the bug hasn’t been reported before ✓

Describe the bug Currently, having internal RSS feed or restAPI links between the first 10 links on the Homepage, they will be fetched.

To Reproduce Steps to reproduce the behavior:

1. On new.rocketlabsqa.ovh set "homepage_with_rss_feed_restAPI_links" as Homepage.
2. Clear cache to trigger link fetching.
3. Check the URLs inside "Add to queue request arguments" logs in wp-rocket-debug.log.html.
4. Notice that between the links fetched are also RSS feed and restAPI links, i.e:

   https://new.rocketlabsqa.ovh/feed/ https://new.rocketlabsqa.ovh/wp-json/wp/v2/users

Expected behavior RSS feed and restAPI links should be excluded from fetching.

Screenshots

Additional context Add any other context about the problem here.

Acceptance Criteria (for WP Media team use only) Clear instructions for developers, to be added before the grooming

[Tabrisrp]

Scope a solution ✅

Engine\Media\AboveTheFold\WarmUp\Controller

• in fetch_links(), we can use the function get_rocket_cache_reject_uri() to get an array of excluded URLs from the cache (which includes the rest API and feed URLs). We probably don't want to do the optimization for excluded pages anyway, so it makes sense to remove them too
• We can use a preg_match( $reject_uri, $url ) to check if one of the pattern match with one of the URLs found
• Update the related tests

Estimate the effort ✅

Effort [S]

[Miraeld]

Looks good to me

[hanna-meda]

Related TCs 41737 & 41744.

[MathieuLamiot]

@jeawhanlee I still get the same issue when testing this branch on new.rocketlabs. Did you check that this PR is doing what is expected?

(There is also the wp-json link there).

I think the test case you added in the fixture is wrong because the wp-json and feed links you added have the new.rocketlabs host. So they are discarded in the fetch_links method as they are external links maybe? You should try to put the right hostname in the fixture maybe? If this is correct, as a way to improve and avoid this type of mistake from now on, the TDD approach would have helped: adding the new fixture first, you would have seen that the test was still passing. This would allow you to identif the fixture is not doing what you expect from it?

[jeawhanlee]

@MathieuLamiot Thanks for pointing this out, indeed the fixture contained the wrong host, As a matter of fact, the TDD approach was used here and the test failed initially, but along the line, I recopied the wrong fixture from an online html minifier.