search

riotkit-org / backup-repository

Backup storage for E2E GPG-encrypted files, with multi-user, quotas, versioning, using a object storage (S3/Min.io/GCS etc.) and deployed on Kubernetes or standalone.
Apache License 2.0
25 stars 4 forks source link
amazon-s3 anarchist backup backup-repository golang image k3s kubernetes lightweight microservice minio openshift repository riotkit s3 service storage storage-api

readme

Backup Repository

NOTICE: Sadly, we no longer have power to maintain Backup Repository ecosystem as it grew up pretty big and we are not a full-time employeed development team. Feel free to maintain a fork.

Coverage Status Test Artifact Hub

Cloud-native, zero-knowledge, multi-tenant, compliance-strict, security-first backup storage with minimal footprint.

TLDR; Primitive backup storage for E2E GPG-encrypted files, with multi-user, quotas, versioning, using a object storage (S3/Min.io/GCS etc.) and deployed on Kubernetes or standalone. No fancy stuff included, lightweight and stable as much as possible is the project target.

Natively supports:

Notice:

Technology stack:

Requirements:

Support:

Difference between other backups systems

Selecting a best tool depends on specific use case. Most common way on Kubernetes is to perform cloud-native volume snapshotting, there Velero project is the most recognized solution that integrates with cloud provides like AWS, Google Cloud or Azure and uses API calls to ask cloud provider for a snapshot.

Backup Repository approach uses application-native and more traditional method of performing backups - using tar, pg_dump, mysqldump and other application-native tools for Backup and Restore. This selected approach have pros and cons as following:

Pros:

Cons:

Maturity

Notice: This software is currently in a pre-prod stage. We do not plan breaking changes to the functionality, but the API interface may still change. We recommend our official Backup Maker client that will be always up-to-date with API changes.

Star a repo, subscribe for releases to get informed.

Security/Compliance demo

Are my backups created in specific time?

Every Backup Collection has HTTP health check endpoint you can monitor and trigger alerts in case when expected backup was not submitted or is invalid.

Attacker got my Kubernetes cluster and wants to overwrite remote backups

Attacker got my Backup Repository credentials from target environment

Attacker wants to upload a terabyte file to generate cloud costs or exhaust disk space

Backup Repository operates on disk quotas. Every incoming byte stream is calculated on the fly and cancelled, when the limit is exhausted.

Storage of my Backup Repository server leaked!

End-To-End backup encryption makes your backup unreadable for people not having your GPG private key.

Running

Application is written in GO and distributed as a single-binary file. Recommended way is to run it within a docker image on a Kubernetes cluster.

Running standalone

export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
export AWS_SECRET_ACCESS_KEY=wJaFuCKtnFEMI/CApItaliSM/bPxRfiCYEXAMPLEKEY

backup-repository \
    --db-password=postgres \
    --db-user=postgres \
    --db-password=postgres \
    --db-name=postgres \
    --jwt-secret-key="secret key" \
    --storage-url="s3://mybucket?endpoint=localhost:9000&disableSSL=true&s3ForcePathStyle=true&region=eu-central-1"

Installing via Helm

helm repo add riotkit-org https://riotkit-org.github.io/helm-of-revolution/
helm install backups riotkit-org/backup-repository-server -n backup-repository # --values ...

Documentation

For documentation please look into ./docs directory

NOTICE: You are probably reading documentation at main branch, please consider selecting a versioned tag from branch/tag selector.

Ecosystem

Find more projects in the Github Community.

Security

Argon2Config{
    time:    1,
    memory:  64 * 1024,
    threads: 4,
    keyLen:  32,
}

RBAC

Objects of type kind: BackupUser (users that can login to Backup Repository server) have a list of global roles. Global roles are granting access to all objects of given type in the system.

If somebody has a collectionManager in its profile, then in all collections that person is a manager which means browsing, deleting, editing, creating.

---
apiVersion: backups.riotkit.org/v1alpha1
kind: BackupUser
# ...
spec:
    # ...
    roles:
        - collectionManager

Scoped RBAC

Most of the object types implements accessControl to specify permissions for given users in scope of this object.

---
apiVersion: backups.riotkit.org/v1alpha1
kind: BackupCollection
# ...
spec:
    # ...
    accessControl:
        - name: admin
          roles:
              - collectionManager

RBAC in code

Domain objects should implement a logic that checks given Actor if it can act specifically in context of this object.

func (u User) CanViewMyProfile(actor User) bool {
    // rbac
    if actor.GetRoles().HasRole(security.RoleUserManager) {
        return true
    }

    // user can view self info
    return u.Spec.Email == actor.Spec.Email
}

ACL in code

func (c Collection) CanUploadToMe(user *users.User) bool {
    if user.GetRoles().HasRole(security.RoleBackupUploader) {
        return true
    }

    for _, permitted := range c.Spec.AccessControl {
        if permitted.UserName == user.Metadata.Name && permitted.Roles.HasRole(security.RoleBackupUploader) {
            return true
        }
    }

    return false
}

Backup Windows

Good practice is to limit how often versions can be submitted. Attacker would need to be very patient to overwrite your past backups with malicious ones.

In emergency cases System Administrator or person with uploadsAnytime role can upload backups between backup windows. Be careful! Do not set up automated backups with administrator account or with account that has uploadsAnytime role.

---
apiVersion: backups.riotkit.org/v1alpha1
kind: BackupCollection
# ...
spec:
    # ...
    window:
        # allow to send backups only everyday starting from 00:30 to 01:30
        - from: "00 30 * * *"
          duration: 1h

Quota

System administrator can create a collection with specified storage limits on single file, whole collection, select a rotation strategy.

Concept is simple - there can be stored X versions of Y size in given collection.

Additionally, there is such thing as extra space which allows to upload a file that exceeds the limit to not break the backup pipeline. Such situation is immediately reported in a collection health check as a warning.

---
apiVersion: backups.riotkit.org/v1alpha1
kind: BackupCollection
# ...
spec:
    # ...
    maxBackupsCount: 5
    maxOneVersionSize: 1M
    maxCollectionSize: 5M

Extra space

The following example allows uploading files of 1 MB size normally, but optionally allows uploading larger files that could in summary take additional 5MB. For example one of uploaded versions can be a 5MB file, or there could be two versions of 2,5MB file each - both exceeding the soft limit of maxOneVersionSize. The maxCollectionSize is a hard limit.

maxBackupsCount = 5
maxOneVersionSize = 1MB
maxCollectionSize = 10MB

estimatedCollectionSize = maxBackupsCount * maxOneVersionSize = 5 * 1MB = 5MB
extraSpace = maxCollectionSize - estimatedCollectionSize = 10MB - 5MB
---
apiVersion: backups.riotkit.org/v1alpha1
kind: BackupCollection
# ...
spec:
    # ...
    maxBackupsCount: 5
    maxOneVersionSize: 1M
    maxCollectionSize: 10M

Rotation

Rotation Strategies gives control over backup versioning.

fifo

First in first out. When adding a new version deletes oldest.

---
apiVersion: backups.riotkit.org/v1alpha1
kind: BackupCollection
# ...
spec:
    # ...
    strategyName: fifo

Contributing

This software is developed with GoLand licensed for open source development. Special thanks for the support.

Security policy