✨ add AWS S3 backend for immudb

[edulix]

Preflight checklist

• [X] I could not find a solution in the existing issues, docs, nor discussions.

Suggestion

add AWS S3 backend for immudb

Possible Implementation

No response

Tasks

Screenshots

```[tasklist] # Tasks - [x] Create the required infrastructure - [x] Adapt the immudb Helm Chart - [x] Enhance Bucket Policy (Versioning) - [x] Prepare IAM Role for the future ``` ```[tasklist] # Main PRs - [ ] https://github.com/sequentech/gitops/pull/218 - [ ] https://github.com/sequentech/infrastructure/pull/42 ``` ```[tasklist] # Stable PRs - [ ] PR 1 ``` ### Additional context _No response_

[angelAtSequent]

• https://docs.immudb.io/master/production/replication.html#replicator-tool
• https://docs.immudb.io/master/production/s3-storage.html
• https://github.com/codenotary/immudb/issues/1094

[angelAtSequent]

Hit: https://github.com/codenotary/immudb/pull/1712/files

Pending: Include lifecycle policy on S3 to delete old versions from the immudb state bucket

[angelAtSequent]

Hit another issue: https://github.com/codenotary/immudb/issues/1877 I'll try to workaround it by pushing the "expected" file upfront (on bucket creation).

[angelAtSequent]

Pending:

• [x] Include lifecycle policy on S3 (versioning was required to enable replication)
• [x] Workaround issue: https://github.com/codenotary/immudb/issues/1877 by pushing an identifier file upfront. (Run the old version 1.5 with both a disk and S3, check what the identifier contains, add it to terraform)

[angelAtSequent]

I'll try to run some tests on staging with the new immudb configuration today

[angelAtSequent]

I'm creating the IAM role just to be ready for: https://github.com/codenotary/immudb/issues/1094

[angelAtSequent]

Once approved, we would need to apply the changes eventually.

[Findeton]

@angelAtSequent What I don't understand is that the s3 buckets (like staging-immudb-lasting-sloth) seem to be empty (looking in the aws page). Shouldn't it be filled with new data? immudb in staging doesn't seem to be using s3 but I may be wrong.

[angelAtSequent]

@angelAtSequent What I don't understand is that the s3 buckets (like staging-immudb-lasting-sloth) seem to be empty (looking in the aws page). Shouldn't it be filled with new data? immudb in staging doesn't seem to be using s3 but I may be wrong.

TL;DR: https://github.com/codenotary/immudb/discussions/1140

immudb uses a multi-file approach. Files are created with exactly the same size as they are filled up. Once a file is fully written and a new file is created for the next writes, the previous one will be automatically uploaded to the S3 server. This upload also takes place when gracefully closing immudb server

if you want to force immudb to write all local data to S3, you need to gracefully shutdown immudb

[angelAtSequent]

After restarting the immudb instance (currently located in the tests-3 namespace, it puts all the files to S3.

[edulix]

Closing it since it was moved to done, please reopen it if it's a mistake to close it.