search

storj / gateway-st

Single-tenant, S3-compatible server to interact with the Storj network
Apache License 2.0
71 stars 19 forks source link

Provide an option to perform in-memory encoding on upload #36

Open amwolff opened 2 years ago

amwolff commented 2 years ago

We added the hacktoberfest label to issues we considered a good fit for 2021's Hacktoberfest. If you want to work on this issue, let us know in the discussion below, so we can remove the up for grabs label and avoid having people working on the same issue independently, making sure no one's work goes in vain. Please also consider discussing how you would like to approach the problem, if appropriate. Thank you for contributing!

The uplink library that Gateway-MT and Gateway-ST use internally has an option to perform file encoding (upload) with an in-memory buffer instead of a disk buffer. The type of buffer should be a config option for both gateways. Currently, only Gateway-MT uses it:

  1. pkg/server/middleware/inmemory.go
  2. cmd/gateway-mt/main.go#L45

It would be best to port this feature into Gateway-ST as well. The context value that tells libuplink to perform in-memory encoding can be set in Gateway-ST's object layer directly for Put... functions. Since Gateway-MT imports Gateway-ST, this functionality can be essentially moved from Gateway-MT to here and re-imported.

amwolff commented 7 months ago

oh, oops, shouldn't have closed it