When setting the -PartSize variable for Write-S3Object TransferUtility is using the default part size of 16MB

[ajmunson]

Describe the bug

When running this command:

Write-S3Object -BucketName $BucketName -Key "$fileKey" -File "$filePath" -PartSize $PartSize

The part size that is used for the transfer of files above 16MB is still 16MB (default).

Expected Behavior

I expected the new specified PartSize to be respected and not only for the new part size to be used for files with a size over 16MB, but for files with a size under the new PartSize to not be multi-parted at all.

Current Behavior

As of now, when -PartSize is declared it really only sets the PartSize in MetaData.

Reproduction Steps

$bucketName = {BucketName} $fileKey = {S3ObjectKey} $filePath = {PathToFileOver16MB} $partSize = {PartSizeLargerThenSizeOfFile} Write-S3Object -BucketName $bucketName -Key "$fileKey" -File "$filePath" -PartSize $partSize $fileHash = (Get-FileHash -Path $filePath -Algorithm MD5).Hash Get-S3ObjectMetadata -BucketName $bucketName -Key "$fileKey" -EtagToMatch "$fileHash".ToLower()

Possible Solution

Add this code at line 646 of Write-S3Object-Cmdlet.cs:

if (cmdletContext.PartSize.HasValue) transferUtilityConfig.MinSizeBeforePartUpload = cmdletContext.PartSize.Value;

API Documentation

Additional Information/Context

No response

AWS Tools for PowerShell version used

AWS.Tools.S3 4.1.263

PowerShell version used

Name Value

---

PSVersion 5.1.17763.1007
PSEdition Desktop
PSCompatibleVersions {1.0, 2.0, 3.0, 4.0...}
BuildVersion 10.0.17763.1007
CLRVersion 4.0.30319.42000
WSManStackVersion 3.0
PSRemotingProtocolVersion 2.3
SerializationVersion 1.1.0.1

Operating System and version

Windows Server 2019

[ashishdhingra]

@ajmunson Thanks for submitting the issue and corresponding PR. Just to be clear, the PartSize and MinSizeBeforePartUpload are separate properties. Per TransferUtilityConfig AP as you linked to as well, the MinSizeBeforePartUpload (it is not MinPartSizeBeforeUpload) is the Gets or sets the minimum size required (in bytes) to enable multi-part upload. So if the file being uploaded using Write-S3Object is at least (default) 16MB in size, it would use multi-part upload, else it would use simple upload. For multipart upload (for files greater than or equal to 16MB default), it would use the part size set by -PartSize parameter (refer Amazon S3 multipart upload limits) for more details.

Canceling the PR and closing this issue.

Thanks, Ashish

[github-actions[bot]]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.