new feature: List deleted objects that haven't been permanently removed

[meteorgan]

Feature Description

When performing a List request, list all object versions, including the deleted ones.

Problem and Solution

Add a deleted parameter to the List operation, when set to true, return all active objects along with deleted objects

Additional Context

In S3, when versioning is enabled, a simple DELETE cannot permanently remove an object. We can still retrieve the object using the version parameter. Therefore, it's useful if a List request can return all object versions.

Are you willing to contribute to the development of this feature?

• [X] Yes, I am willing to contribute to the development of this feature.

[meteorgan]

There is a problem in https://github.com/apache/opendal/pull/5106. when performing a simple DELETE in S3, a delete marker is inserted as the current version. In the response from ListObjectVersions, the delete markers are placed in DeleteMarker container, while other versions of the object are in the Version container. so currently, the List_with(path).version(true) will return all object versions excluding delete marker.

[Xuanwo]

Hi, I believe listing deleted objects is an orthogonal feature compared to listing versioned objects. Some services that don't enable versioning can still have features like "soft delete," such as Azure Blob Soft Delete.

So all the following cases are possible:

• List active files: list_with(path)
• List active files along with non-current files: list_with(path).version(true)
• List active files along with deleted files: list_with(path).deleted(true)
• List all files (active, non-current, deleted): list_with(path).version(true).deleted(true).

[meteorgan]

Hi, I believe listing deleted objects is an orthogonal feature compared to listing versioned objects. Some services that don't enable versioning can still have features like "soft delete," such as Azure Blob Soft Delete.

So all the following cases are possible:

• List active files: list_with(path)
• List active files along with non-current files: list_with(path).version(true)
• List active files along with deleted files: list_with(path).deleted(true)
• List all files (active, non-current, deleted): list_with(path).version(true).deleted(true).

Understood. I'll revise this issue accordingly.