Check the possibility to run tests on 3.16 to check that the beacon captures the right images on templates (integration vs E2E)

[MathieuLamiot]

Context

One critical aspect of 3.16 is that the beacon script should return the correct data to the database (LCP/ATF images correctly identified). It is not clear now how this is tested, and how it could be tested, other than manually. the main question is whether or not this could be covered as part of integration tests, or if Rocket-e2e is needed.

What needs to be done

• [ ] Check if there are currently integration tests for the beacon script, implemented by @Tabrisrp
• [ ] Assess if it is possible to test it in integration tests. What we would need is:
  • Run the beacon script on a specific template, for a given screen size (desktop vs. mobile).
  • Capture the data "LCP/ATF images" returned by the script to the AJAX endpoints
  • Compare this with the expected result.
• [ ] If it is possible, implement it as integration tests. Otherwise, within Rocket-E2E.
• [ ] Gather a list of templates and expected results from the QA team, and add them to the tests.
• [ ] Run the tests and validate that it is working as expected with the QA team.

[jeawhanlee]

Scope a solution ✅

• Create Gherkin steps(See below)
• Add new step definitions
  • Navigate to template page based on browser window parsed from the Gherkin
  • Get the LCP/ATF data from the database through the e2e helper plugin (We would need to update the e2e helper plugin to handle this)
  • Assert that the results is the same as expected.

Estimate the effort ✅

[M]

[jeawhanlee]

Steps in Gherkin

Background:
    Given I am logged in
    And plugin is installed 'new_release'
    And plugin is activated

Scenario: Beacon captures expected images in desktop
    Given I go to ‘page’ in ‘desktop’
    When I get the lcp from beacon
    Then it should be as expected

Scenario: Beacon captures expected images in mobile
    Given I go to ‘page’ in ‘mobile’
    When I get the lcp from beacon
    Then it should be as expected

[Miraeld]

seems ok to me

[MathieuLamiot]

As discussed with @jeawhanlee today, if we plan on moving away from the helper plugin in the mid-term, we could rely on 3rd party plugins like WP Data Access to access the DB from the front-end for now. To avoid developing something ourselves.

[Miraeld]

I forgot to let a comment in here, But if the idea is to access db data, we can use the SSH. The SSH part has already been developed previously, we just need to use it. For this we need to setup our own configuration file, wp.config.ts so we can connect and use ssh related function.

For the db queries, we should use the wp-cli, using this method https://github.com/wp-media/wp-rocket-e2e/blob/bc5885c3faa74b74a868ac55f486b46dbb68d165/utils/commands.ts#L45 we can achieve what we want.

Going through ssh has some pros and cons. If we want something easily reproducible manually, then going through the front-end of the helper plugin is kinda easier. However, going through ssh means it would be faster and more reliable.

[Mai-Saad]

Note: 1- To get the LCP image (shall work for most of templates) => visit page?nowprocket in page speed insights and get LCP 2- Any other image(s) in the viewport shall be ATF 3- LCP/ATF may different from desktop than mobile