supabase / edge-runtime

A server based on Deno runtime, capable of running JavaScript, TypeScript, and WASM services.
MIT License
689 stars 62 forks source link

feat: partial enable s3 and tmp file system #429

Closed nyannyacha closed 2 weeks ago

nyannyacha commented 4 weeks ago

What kind of change does this PR introduce?

Feature, Enhancement

Description

This PR implements s3 file system and tmp file system. Access to them is supported via some of the Deno APIs below.

// Note that it will not be available for APIs with the sync suffix.
open
stat
realPath
create
remove
writeFile
writeTextFile
readFile
readTextFile

To access each filesystem, the path must be preceded by a prefix. Below are the prefixes required to access the file system.

File System Prefix
S3 /s3
Tmp /tmp

Note that for the s3 file system, the path segment that appears right after /s3 is treated as a bucket name. Below is a snippet with a detailed description of the s3 path.

await Deno.readFile("/s3/meowmeow/example.json");                 
//                     │     │       │                             
//                     │     │       │                             
//                     │     │       └──object key
//                     │     └──────────bucket name                
//                     └────────────────prefix to access s3 storage

The following is a sample code snippet that attempts to access the file system in the import statement. Note that accessing the file system in this way is not currently allowed (but may be reconsidered in the future).

await Deno.writeTextFile("/tmp/meowmeow.ts", `
    import { meow } from "/tmp/meowmeow2.ts";
    export { meow }
`);

await Deno.writeTextFile("/tmp/meowmeow2.ts", `
    export function meow() {
        return "meowmeow"
    }
`);

const module = await import("/tmp/meowmeow.ts");

console.log(module);
console.log(module.meow());

You can also pass configurations for these file systems in the main worker script.

...
const s3FsConfig = {
  // optional
  appName: "meowmeow",
  // optional
  endpointUrl: "https://***REDACTED***.supabase.red/storage/v1/s3/",
  // optional
  region: "ap-southeast-1",
  // optional
  forcePathStyle: true,
  // required
  credentials: {
    // required
    accessKeyId: "***REDACTED***",
    // required
    secretAccessKey: "***REDACTED***",
    // optional
    // expiresAfter: <UNIX_TIMESTAMP_SEC>
  },
  // optional
  retry_config: {
    // optional, "standard" | "adaptive"
    mode: "standard",
    // optional
    maxAttempts: 3,
    // optional
    initialBackoffSec: 1,
    // optional
    maxBackoffSec: 20,
    // optional, "reconnect_on_transient_error" | "reuse_all_connections"
    reconnect_mode: "reconnect_on_transient_error",
    // optional
    use_static_exponential_base: false
  }
};

// tmp file system is enabled by default in the user worker.
// if no extra configuration is required, you do not need to pass it to the function.
const tmpFsConfig = {
  // optional, the base path where the temp directory will be created
  base: <PATH>,
  // optional, prefix to be used for temp directory name
  prefix: "meow",
  // optional, suffix to be used for temp directory name
  suffix: "meow",
  // optional, number of random bytes to be used for temp directory name
  random_len: 6,
  // optional, the quota limit for the file system.
  // if this is not specified, files can be written to the file system unlimited.
  quota: <BYTES>
};

return await EdgeRuntime.userWorkers.create({
  ...
  s3FsConfig,
  tmpFsConfig
});
github-actions[bot] commented 2 weeks ago

:tada: This PR is included in version 1.61.0 :tada:

The release is available on GitHub release

Your semantic-release bot :package::rocket: