Customize which queries are refreshed

[vprus]

We are using CubeJS based analytics with a number of users, and as of right now, Redis stores about 1000 saved query results. Some of these queries are very common - e.g. total orders per day - I would like them to auto-refresh. Some of them are very special - the number of orders delivered to the North Pole via albatross delivery. Nobody will run it again.

Right now, I have two options:

• Don't run a separate refresh worker, and CubeJS will refresh the queries on-demand, at the cost of latency
• Run a separate refresh worker, which, per documentation, will be refreshing all saved queries - including the queries nobody needs.

To be honest, I haven't yet tried - for fear it will be actually bring the database down and/or incur BQ costs.

I would like a way to refresh only query results that were used in the last hour, or day.

I looked whether Redis TTL can be used here, but what I see if that

• Redis TTL is either specified in the query or defaults to 24 hours, I can't specify a different default via config
• If Redis TTL is set to 24 hours, then refresh worker will refresh all saved queries each 1 hour, and potentially reset the TTL, not solving my problem at all.

[paveltiunov]

@vprus Redis is not supposed to be refreshed on a scheduled basis and Cube will never refresh data queries in Redis unless those are requested. If you're looking for a scheduled cache refresh you actually need to use pre-aggregations. Have you considered using pre-aggregations?

[vprus]

@paveltiunov : Thanks for the response! If that's the case, it seems the documentation can be improved. Right now, it says, at https://cube.dev/docs/caching:

By default, Cube.js will check and invalidate the cache in the background when in development mode. In production environments, we recommend running a Refresh Worker as a separate instance.

We recommend enabling background cache invalidation in a separate Cube.js worker for production deployments. Please consult the Production Checklist for more information.

If background refresh is disabled, Cube.js will refresh the cache during query execution. Since this could lead to delays in responding to end-users, we recommend always enabling background refresh.

These paragraphs together seem to suggest that refresh worker does background refresh (and production checking suggests to use Redis for cache). Your response say that no, refresh worker does not do background refresh of cached query results.