search

jube-home / jube

Jube is an open-source software designed for monitoring transactions and events. It offers a range of powerful features including real-time data wrangling, artificial intelligence, decision making, and case management. Jube's exceptional performance is particularly evident in its application to fraud prevention and abuse detection scenarios.
https://www.jube.io
GNU Affero General Public License v3.0
29 stars 2 forks source link

Create a catalog of fields in use in rules to suggest covered indexes and reduce fetch from database. #20

Closed richard-churchman closed 6 months ago

richard-churchman commented 11 months ago

Oftentimes the amount of data being used in rules is a small subset of the data that is presented. It follows that forcing the database to go out to Page and Tuple is quite expensive, especially when the index is likely in the buffer cache.

It is not currently possible to see the fields in use by rules that depend on cache data.

Create a function in model sync that will examine the Request XPath elements in existence in rules and create a catalogue of fields in use.

With the catalogue, only select data back from the database that is required in the rules. Limiting the select statement has a big impact on query performance and jsonb field parsing.

Optionally, build - or at a minimum suggest - indexes which cover the fields to avoid the need to go to Page and Tuple.

richard-churchman commented 6 months ago

This won't be implemented as while this functionality does now look at rules to find keys in use to reduce select over wire, covered indies on JSONB are not supported. The strategy is to move the cache to Redis in any event and only bug fixes and streamlining of existing functionality will be implemented where PostgreSQL as cache.