3.17 - LRC Beacon script implementation

[Miraeld]

Description

In this issue, we will implement the RocketLrcBeacon class within the WPR Beacon script. The RocketLrcBeacon class is responsible for the Lazy Render Content (LRC) feature. The class will contain methods to handle the specific logic for the LRC process, including checking if an element is in the viewport, skipping certain elements based on specific conditions, and calculating the distance of an element from the top of the viewport. The prototype code provided will serve as a basis for the implementation. Also, this new class will use the BeaconManager to run the ajax call and pre-conditions.

Sub-tasks (feedbacks)

• [ ] https://github.com/wp-media/wp-rocket/issues/6891
• [x] https://github.com/wp-media/rocket-scripts/issues/23

Scope a solution

• Create a BeaconLrc file in here.
• This new file will contain the class BeaconLrc here.
• Implement the constructor for the RocketLrcBeacon class, which will take a config object as a parameter similar to this
• Implement the run method in the RocketLrcBeacon class. This method will contain the main process for the LRC feature.
• If not done already, implements generic helper methods in the BeaconUtils class based on the prototype code otherwise re-use them. These methods include isElementInViewPort, getDepthAndDistance, getLocationHash , skipElement
  • The isElementInViewport method will check if an element is in the viewport.
  • The getLocationHash method will get the location hash of an element.
  • The skipElement and shouldSkipElement methods will determine whether to skip certain elements based on specific conditions.
  • The getDepthAndDistance method will calculate the distance of an element from the top of the viewport.
• Remove any unnecessary code related to XPath from the prototype.
• Ensure that the RocketLrcBeacon class is properly integrated with the RocketBeacon class.
• Thoroughly test the RocketLrcBeacon class to ensure it works as expected.

Also, a change needs to be made to

if (BeaconUtils.isPageCached() && ( this.config.status.atf && generated_before.lcp )) {
            this.logger.logMessage('Bailing out because data is already available');
            return false;
        }

This condition needs to take into account that LRC can also be disabled, & when it is but LCP isn't, only LRC shouldn't run.

---

For this implementation, we should not depends on _isGeneratedBefore.lcr === true as a condition, as we won't be able to test if the AJAX endpoint hasn't been developed before. Please skip this condition.

Additional

Prototype code:

function isElementInViewport(el) {
    const rect = el.getBoundingClientRect();
    return (
        rect.top >= 0 &&
        rect.left >= 0 &&
        rect.bottom <= (window.innerHeight || document.documentElement.clientHeight) &&
        rect.right <= (window.innerWidth || document.documentElement.clientWidth)
    );
}

function getLocationHash(element) {
  // Check if the element has the 'data-rocket-location-hash' attribute
  if (element.hasAttribute('data-rocket-location-hash')) {
    // Get and return the value of the 'location-hash' attribute
    return element.getAttribute('data-rocket-location-hash');
  } else {
    // Return null or an appropriate value if the attribute doesn't exist
    return 'No hash detected';
  }
}

function getXPath(element) {
    if (element.id !== "") {
        // If the element has an ID, return the XPath with the ID
        return '//*[@id="' + element.id + '"]';
    }

    // Function to get the position of the element among its siblings of the same type
    function getElementPosition(element) {
        let pos = 1;
        let sibling = element.previousElementSibling;
        while (sibling) {
            if (sibling.nodeName === element.nodeName) {
                pos++;
            }
            sibling = sibling.previousElementSibling;
        }
        return pos;
    }

    // Recursively build the XPath string
    function getElementXPath(element) {
        if (element === document.body) {
            return '/html/body';
        }

        const position = getElementPosition(element);
        const xpath = getElementXPath(element.parentNode) + '/' + element.nodeName.toLowerCase() + '[' + position + ']';
        return xpath;
    }

    return getElementXPath(element);
}

function skipElement(element) {
    if (!element || !element.id) return false; // If element has no ID, don't skip it
    // Check if element's ID contains specific strings to skip
    const skipStrings = ['memex']; // Add more strings to skip as needed
    return skipStrings.some(str => element.id.toLowerCase().includes(str));
}

function shouldSkipElement(element, exclusions) {
    if (!element) return false;
    // Check if the element matches any of the exclusion patterns
    for (let i = 0; i < exclusions.length; i++) {
        const [attribute, pattern] = exclusions[i];
        const attributeValue = element.getAttribute(attribute);
        if (attributeValue && new RegExp(pattern, 'i').test(attributeValue)) {
            return true;
        }
    }

    return false;
}

function getDepthAndDistance(element, depth, viewportHeight) {
    if (depth > 3 || !element) return;

    const exclusions = [
        ['id', 'memex'],  // Skip elements with id containing 'memex'
        ['class', 'grecaptcha-badge']
    ];

    const tagName = element.tagName;
    if (!['BODY', 'DIV', 'MAIN', 'FOOTER', 'SECTION', 'ARTICLE', 'HEADER'].includes(tagName) || shouldSkipElement(element, exclusions)) return; // Only process some specific tags, and skip elements with specific attributes

    const rect = element.getBoundingClientRect();
    const scrollTop = window.pageYOffset || document.documentElement.scrollTop;
    let distanceFromViewportTop = rect.top + scrollTop - viewportHeight;

    // Ensure distance is non-negative
    distanceFromViewportTop = Math.max(0, distanceFromViewportTop);

    // Apply styling based on distance
    let style = '';
    if (distanceFromViewportTop > 1800) {
        style = 'color: green;';
    } else if (distanceFromViewportTop === 0) {
        style = 'color: red;';
    }

    console.log(`%c${'\t'.repeat(depth)}${tagName} (Depth: ${depth}, Distance from viewport top: ${distanceFromViewportTop}px)`, style);

    const xpath = getXPath(element);
    console.log(`%c${'\t'.repeat(depth)}Xpath: ${xpath}`, style);

    const locationhash = getLocationHash(element);
    console.log(`%c${'\t'.repeat(depth)}Location hash: ${locationhash}`, style);

    console.log(`%c${'\t'.repeat(depth)}Dimensions Client Height: ${element.clientHeight}`, style);

    // Check if parent distance exceeds 2500 pixels, then skip processing children
    if (distanceFromViewportTop <= 2500) {
        const children = Array.from(element.children);
        children.forEach(child => {
            getDepthAndDistance(child, depth + 1, viewportHeight);
        });
    }
}

window.onload = function () {

    const viewportHeight = window.innerHeight || document.documentElement.clientHeight;

    // Print elements and their distances
    getDepthAndDistance(document.body, 1, viewportHeight);
}

Additional 2 - Code not tested.

BeaconLrc should be like:

'use strict';

import BeaconUtils from "./Utils.js";
import BeaconManager from "./BeaconManager.js";

class BeaconLrc {
    constructor(config, logger) {
        this.config = config;
        this.logger = logger;
        this.lazyRenderElements = [];
    }

    async run() {
        try {
            const elementsInView = this._getLazyRenderElements();
            if (elementsInView) {
                this._processElements(elementsInView);
            }
        } catch (err) {
            this.errorCode = 'script_error';
            this.logger.logMessage('Script Error: ' + err);
        }
    }

    _getLazyRenderElements() {
        const elements = document.querySelectorAll(this.config.elements);

        if (elements.length <= 0) {
            return [];
        }

        const validElements = Array.from(elements).filter(element => !this._skipElement(element));

        return validElements.map(element => ({
            element: element,
            depth: this._getElementDepth(element),
            distance: this._getElementDistance(element)
        }));
    }

    _getElementDepth(element) {
        let depth = 0;
        let parent = element.parentElement;
        while (parent) {
            depth++;
            parent = parent.parentElement;
        }
        return depth;
    }

    _getElementDistance(element) {
        const rect = element.getBoundingClientRect();
        const scrollTop = window.pageYOffset || document.documentElement.scrollTop;
        return rect.top + scrollTop - (window.innerHeight || document.documentElement.clientHeight);
    }

    _skipElement(element) {
        const skipStrings = this.config.skipStrings || ['memex'];
        if (!element || !element.id) return false;
        return skipStrings.some(str => element.id.toLowerCase().includes(str));
    }

    _shouldSkipElement(element, exclusions) {
        if (!element) return false;
        for (let i = 0; i < exclusions.length; i++) {
            const [attribute, pattern] = exclusions[i];
            const attributeValue = element.getAttribute(attribute);
            if (attributeValue && new RegExp(pattern, 'i').test(attributeValue)) {
                return true;
            }
        }
        return false;
    }

    _processElements(elements) {
        elements.forEach(({ element, depth, distance }) => {
            if (this._shouldSkipElement(element, this.config.exclusions || [])) {
                return;
            }

            this.lazyRenderElements.push({ element, depth, distance });

            const style = distance > 1800 ? 'color: green;' : distance === 0 ? 'color: red;' : '';
            console.log(`%c${'\t'.repeat(depth)}${element.tagName} (Depth: ${depth}, Distance from viewport top: ${distance}px)`, style);

            const xpath = this._getXPath(element);
            console.log(`%c${'\t'.repeat(depth)}Xpath: ${xpath}`, style);

            const locationhash = this._getLocationHash(element);
            console.log(`%c${'\t'.repeat(depth)}Location hash: ${locationhash}`, style);

            console.log(`%c${'\t'.repeat(depth)}Dimensions Client Height: ${element.clientHeight}`, style);
        });
    }

    _getXPath(element) {
        if (element.id !== "") {
            return `//*[@id="${element.id}"]`;
        }

        function getElementPosition(element) {
            let pos = 1;
            let sibling = element.previousElementSibling;
            while (sibling) {
                if (sibling.nodeName === element.nodeName) {
                    pos++;
                }
                sibling = sibling.previousElementSibling;
            }
            return pos;
        }

        function getElementXPath(element) {
            if (element === document.body) {
                return '/html/body';
            }
            const position = getElementPosition(element);
            const xpath = `${getElementXPath(element.parentNode)}/${element.nodeName.toLowerCase()}[${position}]`;
            return xpath;
        }

        return getElementXPath(element);
    }

    _getLocationHash(element) {
        return element.hasAttribute('data-rocket-location-hash')
            ? element.getAttribute('data-rocket-location-hash')
            : 'No hash detected';
    }

    getResults() {
        return this.lazyRenderElements;
    }
}

export default BeaconLrc;

And BeaconManager should be modified to

import BeaconLcp from "./BeaconLcp.js";
import BeaconLrc from "./BeaconLrc.js";  // Import BeaconLrc
import BeaconUtils from "./Utils.js";
import Logger from "./Logger.js";

class BeaconManager {
    constructor(config) {
        this.config = config;
        this.lcpBeacon = null;
        this.lrcBeacon = null;  // Initialize lrcBeacon
        this.infiniteLoopId = null;
        this.scriptTimer = new Date();
        this.errorCode = '';
        this.logger = new Logger(this.config.debug);
    }

    async init() {
        if (!await this._isValidPreconditions()) {
            this._finalize();
            return;
        }

        this.infiniteLoopId = setTimeout(() => {
            this._handleInfiniteLoop();
        }, 10000);

        const isGeneratedBefore = await this._isGeneratedBefore();

        if (!isGeneratedBefore.lcp) {
            this.lcpBeacon = new BeaconLcp(this.config, this.logger);
            await this.lcpBeacon.run();
        }

        if (!isGeneratedBefore.lrc) {  // Check and run LRC if not generated before
            this.lrcBeacon = new BeaconLrc(this.config, this.logger);
            await this.lrcBeacon.run();
        }

        this._saveFinalResultIntoDB();
    }

    async _isValidPreconditions() {
        const threshold = {
            width: this.config.width_threshold,
            height: this.config.height_threshold
        };
        if (BeaconUtils.isNotValidScreensize(this.config.is_mobile, threshold)) {
            this.logger.logMessage('Bailing out because screen size is not acceptable');
            return false;
        }

        if (BeaconUtils.isPageCached() && await this._isGeneratedBefore()) {
            this.logger.logMessage('Bailing out because data is already available');
            return false;
        }

        return true;
    }

    async _isGeneratedBefore() {
        let data_check = new FormData();
        data_check.append('action', 'rocket_check_beacon');
        data_check.append('rocket_beacon_nonce', this.config.nonce);
        data_check.append('url', this.config.url);
        data_check.append('is_mobile', this.config.is_mobile);

        const beacon_data_response = await fetch(this.config.ajax_url, {
            method: "POST",
            credentials: 'same-origin',
            body: data_check
        }).then(data => data.json());

        return beacon_data_response.data;
    }

    _saveFinalResultIntoDB() {
        const results = {
            lcp: this.lcpBeacon ? this.lcpBeacon.getResults() : null,
            lrc: this.lrcBeacon ? this.lrcBeacon.getResults() : null  // Save LRC results
        };

        const data = new FormData();
        data.append('action', 'rocket_beacon');
        data.append('rocket_beacon_nonce', this.config.nonce);
        data.append('url', this.config.url);
        data.append('is_mobile', this.config.is_mobile);
        data.append('status', this._getFinalStatus());
        data.append('results', JSON.stringify(results));

        fetch(this.config.ajax_url, {
            method: "POST",
            credentials: 'same-origin',
            body: data,
            headers: {
                'wpr-saas-no-intercept': true
            }
        })
            .then(response => response.json())
            .then(data => {
                this.logger.logMessage(data);
            })
            .catch(error => {
                this.logger.logMessage(error);
            })
            .finally(() => {
                this._finalize();
            });
    }

    _getFinalStatus() {
        if ('' !== this.errorCode) {
            return this.errorCode;
        }

        const scriptTime = (new Date() - this.scriptTimer) / 1000;
        if (10 <= scriptTime) {
            return 'timeout';
        }

        return 'success';
    }

    _handleInfiniteLoop() {
        this._saveFinalResultIntoDB();
    }

    _finalize() {
        const beaconscript = document.querySelector('[data-name="wpr-wpr-beacon"]');
        beaconscript.setAttribute('beacon-completed', 'true');
        clearTimeout(this.infiniteLoopId);
    }
}

export default BeaconManager;

[MathieuLamiot]

@Miraeld @jeawhanlee

---

A point seems missing: what happens if the user deactivate one of the feature, let's say LCR. It would be best if the script could avoid running LCR in that case. Maybe this is the intent of config.status.atf here? In which case logic must be adapted?

        if (BeaconUtils.isPageCached() && ( this.config.status.atf && generated_before.lcp )) {
            this.logger.logMessage('Bailing out because data is already available');
            return false;
        }

Or maybe the script would get this information through the check_data? (if the feature is off, check_data returns false for this feature?). If this is indeed missing, maybe add it as a dedicated GH issue?

This needs some clarifications.

---

For AC purposes, maybe consider adding logs in _isGeneratedBefore to get the answers? Also, we might need a first version that is not depending on the result of _isGeneratedBefore for LCR. Otherwise, we won't be able to test without the AJAX endpoints implemented, which will create too much dependencies. @Miraeld & @jeawhanlee I would suggest to clearly state in this issue that LCR must run even if _isGeneratedBefore.lcr is true ; and add in the AJAX issue that this condition must be added then.

Regarding ACs, this issue does not need any others issues right? @wp-media/qa-team @wp-media/product ACs would mostly be testign against templates as per here

[Miraeld]

I've modified the grooming according to your comment to add some precision as you did. Also, this issue doesn't need anything else to work, technically. It can be tested by adding some console.log in the script itself to see outputs. However if we want something that works from end to end, we need the AJAX endpoint to be implemented and be able to save data, like you mention. Now we can skip this part (AJAX) by adding logs and forcing the value of _isGeneratedBefore.lcr for testing purpose to make it run as it would in real condition.