Add option to use XTDB in coco/base charts

[planetf1]

Currently both charts use inmemory or the graph repo. In memory isn't persistent - fine for the first steps of a demo, but can be frustrating if developers use the environment as a springboard for further investigation. The graph repo is slow.

Meanwhile xtdb is compelling, the performance figures look good, so we could point people down a better path by including xtdb in these charts

Additionally I need xtdb to support my operator work. Whilst the charts using the operator will be rather different in terms of egeria, the setup of xtdb would follow a similar pattern

[planetf1]

Note - XTDB referenced in https://github.com/odpi/egeria/issues/6341

[cmgrote]

@planetf1 let's pick this up when we can? I can help translate configurable back-end from what we've setup in PTS and CTS charts, but would love your help / guidance on how to leverage any PV(C) patterns for the underlying persistence layer.

[planetf1]

Thanks @cmgrote . I've seen the pts/cts example, but I doubt I'll get a chance to look at this until after our upcoming community meeting so probably looking at w/c 10 Oct onwards

[dwolfson]

This would be good - but what XTDB backend should use by default? rocks? Postgres? @cmgrote - any recommendations? I would assume that this isn't production but could be a test environment.

[cmgrote]

Purely for these kinds of test purposes we could go for either XTDB's own native in-memory back-end (no persistence), or for a mixture of RocksDB and/or LMDB. Both of these can run embedded (no external dependencies) while still giving a disk-backed persistence possibility. (Postgres would require an external dependency to run the Postgres system itself, as I don't believe it can run "embedded", at least not with XTDB.)

In fact the config I use when running the CTS and PTS charts is:

• LMDB as the index store
• RocksDB as the document store and transaction log

[planetf1]

And for ref - supported list: https://docs.xtdb.com/administration/configuring/#_storage

I think we should have 2 options a) inmemory b) persistent

At some point we need to move to seperating out the persistence - arguably it's easier, and will be needed if/when we do replication.

The logical stages might be

• add xtdb/in memory

• add a simple xtdb/persisted configuration using rocksdb (we already have storage allocated in a persistent configuration for each container)

  • seperate out to external stores - this could be postgres+rocks or whatever we think!

  but preferably in rapid progression :-)

[dwolfson]

Indeed - and we are also discussing adding a Postgres option in the Egeria deployment for other purposes anyway..

[planetf1]

@dwolfson Some of this has been done, but you may wish to add more?