compucorp / uk.co.compucorp.civicrm.pivotreport

CiviCRM Pivot table reporting solution
Other
8 stars 13 forks source link

HW-433: Pivot Report rebuild cache with cron job #83

Closed coldrunKacper closed 6 years ago

coldrunKacper commented 6 years ago

This PR adds new 'rebuildcachecronjob' API action for PivotReport entity.

It doesn't require any parameters (it automatically walks through available Pivot Report entities and caches further data on each run). It uses 'lock' property so there is no overlapping when running the action before previous run is finished.

Caching is stored as 'non-active' until whole cache is built. Then the cache is marked as 'active' and is used by Pivot Reports. This way we can build the cache without harming the existing one and use Pivot Reports with previously built cache without waiting until current build process is complete.

The action can be add into CiviCRM scheduled jobs to run repeatedly. Each run caches one chunk of Pivot Report data and saves information (such as offset, index, page) needed for next run to cache further Pivot Report data.

The action can also be executed with drush or API explorer (for testing purposes) to see its results.

The action returns an array such as:

"values": [
    {
        "entity": "Contribution",
        "offset": 128,
        "multiValuesOffset": 0,
        "time": 0.71826481819153,
        "cacheBuilt": false
    }
]

'entity' is the name of the next processed entity 'offset' is the offset of the next processed entity 'multiValuesOffset' is the multiValuesOffset of the next processed entity 'time' is the time (in seconds) required to complete latest run 'cacheBuilt' says if whole cache is built after latest run

If the cache is completely built, the action response looks like below:

"values": [
    {
        "entity": null,
        "offset": 0,
        "multiValuesOffset": 0,
        "time": 0.27362394332886,
        "cacheBuilt": true
    }
]

('entity' becomes null and 'cacheBuilt' is true)

If the action is run before previously action finishes then the response is simply null:

"values": [
    null
]

After whole cache is built it is set as 'active' (so the Pivot Report is using it) and next run starts over the caching process.