Moving sfs forward requires us to change the database and directory data store layout occasionally.
We already have a mechanism to version the database, run migration code and mark incompatible versions.
What we don't have is automated testing that runs a current version (e.g PR, nightly, etc) with a previous data store version that it claims to support. Due to this migration bugs and unintended incompatibilities went into main and releases.
Possible Solution
As part of every PR CI run start s3gw with the latest released version and run basic functional tests.
A possible implementation would be:
During PR CI start s3gw:latest with a named docker volume.
Then start s3gw:PR with that named volume.
Starts ok?
If so, run basic functional test
An alternative would be to create golden images of the data store with well defined content (buckets, objects, users) and make that available to the CI process.
During PR CI we could try to start with that image and property check it via S3
Problem Statement
Moving sfs forward requires us to change the database and directory data store layout occasionally. We already have a mechanism to version the database, run migration code and mark incompatible versions.
What we don't have is automated testing that runs a current version (e.g PR, nightly, etc) with a previous data store version that it claims to support. Due to this migration bugs and unintended incompatibilities went into main and releases.
Possible Solution
As part of every PR CI run start s3gw with the latest released version and run basic functional tests.
A possible implementation would be: During PR CI start s3gw:latest with a named docker volume. Then start s3gw:PR with that named volume. Starts ok? If so, run basic functional test
An alternative would be to create golden images of the data store with well defined content (buckets, objects, users) and make that available to the CI process. During PR CI we could try to start with that image and property check it via S3