compasjs / compas

Unified backend tooling
https://compasjs.com
MIT License
12 stars 8 forks source link

store(minio): switch to @aws-sdk/client-s3 #1906

Closed dirkdev98 closed 2 years ago

dirkdev98 commented 2 years ago

This is a smaller SDK, is more popular than minio and is most likely expected to be used by our users. I am also not happy with the current functions that declare the minio parameter, so we may want to revisit those api's.

This is a big breaking change, and compatibility with S3 like deployments should be reverified on usage.

dirkdev98 commented 2 years ago

Since the @aws-sdk/client-s3 has almost daily updates, most of the time it being only a version bump, we may want to disable this in our dependabot config or figure some way of auto ignoring these updates when nothing has changed for the s3 client

dirkdev98 commented 2 years ago

Current functions accepting the Minio client all in @compas/store

Quick draft for new api's;

Uses objectStorage for bucket only operations. Otherwise uses the file prefix.

function objectStorageGetDevelopmentConfig(): S3ClientConfig;
function objectStorageCreateClient(config?: S3ClientConfig): S3Client;
function objectStorageEnsureBucket(S3Client, bucketName, opts: { timeout?: number }): Promise<void>;
// -> Should use `waitUntilBucketExists` to check if the bucket exists with a small timeout, else create it
function objectStorageRemoveBucket(S3Client, bucketName, opts: { includeObjects: boolean });
function objectStorageListObjects(S3Client): Promise<*[]>;
function objectStorageGetFileStream(S3Client, file);

async function fileCreateOrUpdate(sql, objectStorage, ...);
async function fileSyncDeletedWithObjectStorage(sql, objectStorage, ...);
async function fileFormatMetadata(sql, objectStorage, ...);
async function fileSendFile(sql, objectStorage, KoaContext, ...);
async function fileSendTransformedImage(sql, objectStorage, KoaContext, ...);