search

tempesta-tech / tempesta

All-in-one solution for high performance web content delivery and advanced protection against DDoS and web attacks
https://tempesta-tech.com/
GNU General Public License v2.0
607 stars 103 forks source link

TDBv3: cluster #828

Open krizhanovsky opened 6 years ago

krizhanovsky commented 6 years ago

There many requirement to replicate TempestaDB data among cluster nodes for clustered Web cache, TLS sessions, blocking rules and so on. The replication must be per-collection/table, e.g. to replicate or not TLS sessions, ML models, web cache and so on.

Explore applicability of Transparent Inter-process Communication (TIPC) for the task.

vankoven commented 5 years ago

Note, that some data structures we have may be easily replicated - e.g. TfwHttpSess - they store pointers to other data structures from active configuration. After Tempesta is cold-restarted that pointers in TDB data becomes invalid and must be updated. It's not wise to do bulk updates on configuration reloading, restart times will be enormously increased. Instead an effective way must be provided to update the single TDB entry before the first use.

Other issues - is data consistency after unexpected shutdowns. Cleanup routines can not be finished on previous run, some old data (as reference counters) may remain dirty in the TDB.

See https://github.com/tempesta-tech/tempesta/issues/685#issuecomment-509808595 related to storing TfwHttpSess in TDB.