How to sync to s3?

[spirobel]

what is the commandline option to sync to s3? I see --sync-from s3 but it seems like the docs for sync to s3 are missing.

Kind Regards, spirobel

[losfair]

Hi,

Please check the docs for setting up sync to S3.

[spirobel]

@losfair how to do it with the self hosted version?

[tionis]

@spirobel There is no built-in way, you can however use litestream to replicate to s3/sftp with or without encryption. I implemented this using the following Dockerfile:

FROM litestream/litestream AS litestream

FROM ghcr.io/denoland/denokv:latest AS denokv

FROM debian:latest
COPY --from=litestream /usr/local/bin/litestream /usr/local/bin/litestream
COPY --from=denokv /usr/local/bin/denokv /usr/local/bin/denokv

RUN apt update && apt upgrade -y
RUN apt install openssh-client -y

ENV DENO_KV_SQLITE_PATH="/data/denokv.sqlite3"

RUN mkdir -p ~/.ssh
RUN ssh-keyscan zh2587s2.rsync.net >> ~/.ssh/known_hosts

COPY litestream.yml /etc/litestream.yml
COPY start.sh /usr/local/bin/start.sh

ENTRYPOINT ["/usr/local/bin/start.sh"]

and start.sh

#!/bin/sh
echo "$SSH_PRIV_KEY" | base64 -d >~/.ssh/id_ed25519
echo "$SSH_PUB_KEY" | base64 -d >~/.ssh/id_ed25519.pub
if ! test -d /data; then
    mkdir /data
fi
if ! test -f /data/denokv.sqlite3; then
    litestream restore /data/denokv.sqlite3
fi
# acquite some distributed lock here?
litestream replicate -exec '/usr/local/bin/denokv serve'

and successfully deployed it on fly.io and have been using it since then.

(The docker container also automatically restores the database from backup if it does not exist, the operator has to ensure that there are no two versions of the container running at any time.)

[tionis]

Oh, and my litestream config looks like this:

dbs:
  - path: /data/denokv.sqlite3
    replicas:
      - type: sftp
        host: zh2587s2.rsync.net:22
        user: zh2587s2
        path: litestream/denokv
        key-path: ${HOME}/.ssh/id_ed25519
        age:
          identities:
            - ${AGE_PRIVATE_KEY}
          recipients:
            - ${AGE_PUBLIC_KEY}