Storj offers the ability override file encryption beneath a path prefix. The S3 Gateway-MT
currently does not show files it cannot decrypt. Thus while an admin may have rights
to delete files via their access grant, they lack the functional ability to delete them via S3.
One solution is to allow S3 to show encrypted files. The core use case could be handled
by simply allowing files to be deleted beneath a path prefix, regardless of whether they
can be decrypted or not.
What is the problem/pain point?
The Storj native APIs allow for overriding the encryption key for all files beneath a prefix.
The S3 client used in Edge will only display files which it can "see" - files with the same
encryption key. Thus S3 users may see that a path exists, but it cannot see files beneath
that path. In cases where they are authorized to delete these files they cannot, because
they cannot see the object names.
Who is impacted?
This problem is most likely to happen with businesses who want to position themselves as
secure. In this case, they would allow their end users to override the encryption key for
files beneath a prefix. If these businesses need to delete the data of these users, they are
currently limited to using Uplink or similar Go code. Thus only non-Go users are impacted.
What is the impact?
This would allow businesses who want to offer their end users privacy a complete solution
while taking advantage of S3's benefits.
Why now?
Key Storj customers are actively requesting a solution.
Requirements
User Story
As an S3 user, I want to be able to delete files that I have permission to delete, but not
permission to decrypt, so that I can delete files that have end-user specified encryption.
Acceptance Criteria
Given a path prefix in a bucket where the encryption is overridden, a user with S3 credentials
that have delete access to the path but not the prefix encryption key should be able to:
1) Discover the existence of a prefix at which encryption becomes overridden using AWS APIs
2) Delete all files beneath the prefix using standard AWS APIs with custom HTTP headers
Success Metrics
Allow new code developers using existing AWS S3 APIs to see Storj prefixes at which custom
encryption is set, and delete all files beneath that encryption key using standard AWS APIs and
custom code.
amwolff
commented
11 months ago
Background
Storj offers the ability override file encryption beneath a path prefix. The S3 Gateway-MT currently does not show files it cannot decrypt. Thus while an admin may have rights to delete files via their access grant, they lack the functional ability to delete them via S3.
One solution is to allow S3 to show encrypted files. The core use case could be handled by simply allowing files to be deleted beneath a path prefix, regardless of whether they can be decrypted or not.
What is the problem/pain point?
The Storj native APIs allow for overriding the encryption key for all files beneath a prefix. The S3 client used in Edge will only display files which it can "see" - files with the same encryption key. Thus S3 users may see that a path exists, but it cannot see files beneath that path. In cases where they are authorized to delete these files they cannot, because they cannot see the object names.
Who is impacted?
This problem is most likely to happen with businesses who want to position themselves as secure. In this case, they would allow their end users to override the encryption key for files beneath a prefix. If these businesses need to delete the data of these users, they are currently limited to using Uplink or similar Go code. Thus only non-Go users are impacted.
What is the impact?
This would allow businesses who want to offer their end users privacy a complete solution while taking advantage of S3's benefits.
Why now?
Key Storj customers are actively requesting a solution.
Requirements
User Story
As an S3 user, I want to be able to delete files that I have permission to delete, but not permission to decrypt, so that I can delete files that have end-user specified encryption.
Acceptance Criteria
Given a path prefix in a bucket where the encryption is overridden, a user with S3 credentials that have delete access to the path but not the prefix encryption key should be able to: 1) Discover the existence of a prefix at which encryption becomes overridden using AWS APIs 2) Delete all files beneath the prefix using standard AWS APIs with custom HTTP headers
Success Metrics
Allow new code developers using existing AWS S3 APIs to see Storj prefixes at which custom encryption is set, and delete all files beneath that encryption key using standard AWS APIs and custom code.