pipe: support append

[dormanze]

Community Contribution License

All community contributions in this pull request are licensed to the project maintainers under the terms of the Apache 2 license. By creating this pull request I represent that I have the right to license the contributions to the project maintainers under the Apache 2 license.

Description

As the title.

Motivation and Context

How to test this PR?

1. creat file

   echo "test1" | mc pipe --append myminio/bucket/test.file

2. upload new content with append

   echo "test2" | mc pipe --append myminio/bucket/test.file

3. cat test.file

   mc cat myminio/bucket/test.file

Types of changes

• [ ] Bug fix (non-breaking change which fixes an issue)
• [ ] New feature (non-breaking change which adds functionality)
• [x] Optimization (provides speedup with no functional changes)
• [ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

• [ ] Fixes a regression (If yes, please add commit-id or PR # here)
• [ ] Unit tests added/updated
• [ ] Internal documentation updated
• [ ] Create a documentation update request here

[klauspost]

I am not against the feature per se - but please make it very clear in the documentation that this works by reading back the existing object.

Users will expect to be able to append to a 1GB file instantly - and it should be noted everywhere that this will not be the case. (and even so users will still complain).

[harshavardhana]

I think this may not be what we want to encourage as a practice @dormanze it is better called as mc compose or something where you create a new object with existing 2 or more objects.

[dormanze]

I think this may not be what we want to encourage as a practice @dormanze it is better called as mc compose or something where you create a new object with existing 2 or more objects.

That's a good idea. I'll put it on my to-do list. However, I added the append functionality initially because Linux streaming supports appending. Do you think this is necessary?

[dormanze]

I am not against the feature per se - but please make it very clear in the documentation that this works by reading back the existing object.

Users will expect to be able to append to a 1GB file instantly - and it should be noted everywhere that this will not be the case. (and even so users will still complain).

I'll add some notes to the documentation.Online documentation also needs attention.