search

jube-home / jube

Jube is an open-source software designed for monitoring transactions and events. It offers a range of powerful features including real-time data wrangling, artificial intelligence, decision making, and case management. Jube's exceptional performance is particularly evident in its application to fraud prevention and abuse detection scenarios.
https://www.jube.io
GNU Affero General Public License v3.0
29 stars 2 forks source link

Move PostgreSQL Cache Updated and Inserts to a Single Transaction. Move Case Creation and TTL Counter Entry to Bulk Insert. #34

Closed richard-churchman closed 6 months ago

richard-churchman commented 8 months ago

During online transaction processing, rather the invoke process, there are several insert \ update interactions with the PostgreSQL database which happen inline. These interactions are expensive, but given PostgreSQL being used as cache unavoidable.

All inserts and updates should be done inside the same transaction to avoid excessive commit. It follows that it is necessary to batch up the inserts \ updates, where not possible to bulk insert (in the case of Time To Live (TTL) Counter Entries and Case Creation) and execute in a single transaction.

For inserts which are not time sensitive, such as TTL Counter Entries (used to wind back TTL counters) and Case creation, move these back bulk and \ or background processes.

richard-churchman commented 6 months ago

Given issue #31 this would not longer be possible as all IO is passed to the thread pool.