Support Tagging in PostPolicy upload functionality

[orientalperil]

NOTE

I'm trying to upload a file using a presigned POST URL and have the file tagged. The code works on S3 but not Minio.

Expected Behavior

Object in the bucket should be tagged with the key and value {"retention": "expire_fast"}

Current Behavior

Object was not tagged at all

Steps to Reproduce (for bugs)

I used this code which uploads the file but doesn't tag:

import datetime

import boto3
import requests
from django.conf import settings

RETENTION = 'retention'
EXPIRE_FAST = 'expire_fast'
TEMP_MARKER = 'temp'

service = 's3'
region = 'us-east-1'
t = datetime.datetime.utcnow()
algorithm = 'AWS4-HMAC-SHA256'
credential_scope = '/'.join([t.strftime('%Y%m%d'), region, service, 'aws4_request'])

key = 'temp-22e063cd-4ed3-437c-87ac-2bc6e6f328d0/my_file.txt'

def get_tag_xml(key, value):
    return f"<Tagging><TagSet><Tag><Key>{key}</Key><Value>{value}</Value></Tag></TagSet></Tagging>"

conditions = [
    {"x-amz-algorithm": algorithm},
    {"x-amz-credential": credential_scope},
    {"x-amz-date": t.isoformat()},
    {"tagging": get_tag_xml(RETENTION, EXPIRE_FAST)},
    {"success_action_status": "201"},
    {"bucket": settings.AWS_STORAGE_BUCKET_NAME},
    ["starts-with", "$key", TEMP_MARKER],
]

fields = {
    "x-amz-algorithm": algorithm,
    "x-amz-credential": credential_scope,
    "x-amz-date": t.isoformat(),
    "tagging": get_tag_xml(RETENTION, EXPIRE_FAST),
    "success_action_status": "201",
}

client = boto3.client(
    service_name='s3',
    aws_access_key_id=settings.AWS_ACCESS_KEY_ID,
    aws_secret_access_key=settings.AWS_SECRET_ACCESS_KEY,
    endpoint_url=settings.AWS_S3_ENDPOINT_URL,  # http://127.0.0.1:6000
)

presigned = client.generate_presigned_post(
    settings.AWS_STORAGE_BUCKET_NAME,
    key,
    Fields=fields,
    Conditions=conditions,
    ExpiresIn=60*60,
)

# Demonstrate how another Python program can use the presigned URL to upload a file
object_name = 'my_file.txt'
with open(object_name, 'rb') as f:
    files = {'file': (object_name, f)}
    http_response = requests.post(presigned['url'], data=presigned['fields'], files=files)
print(f'File upload HTTP status code: {http_response.status_code}')
print(http_response.content)

The output of the code is:

File upload HTTP status code: 201
b'<?xml version="1.0" encoding="UTF-8"?>\n<PostResponse><Bucket>my-bucket</Bucket><Key>temp-22e063cd-4ed3-437c-87ac-2bc6e6f328d0/my_file.txt</Key><ETag>&#34;14758f1afd44c09b7992073ccf00b43d&#34;</ETag><Location>http://127.0.0.1:6000/my-bucket/temp-22e063cd-4ed3-437c-87ac-2bc6e6f328d0/my_file.txt</Location></PostResponse>'

I got this log capture:

$  ./mc admin trace --verbose myminio
127.0.0.1:6000 [REQUEST s3.PostPolicyBucket] [2024-05-26T13:23:20.832] [Client IP: 127.0.0.1]
127.0.0.1:6000 POST /my-bucket
127.0.0.1:6000 Proto: HTTP/1.1
127.0.0.1:6000 Host: 127.0.0.1:6000
127.0.0.1:6000 User-Agent: python-requests/2.31.0
127.0.0.1:6000 Accept: */*
127.0.0.1:6000 Accept-Encoding: gzip, deflate
127.0.0.1:6000 Connection: keep-alive
127.0.0.1:6000 Content-Length: 1917
127.0.0.1:6000 Content-Type: multipart/form-data; boundary=e189907d61cb5bf9142c98341cbc85c6
127.0.0.1:6000 <BLOB>
127.0.0.1:6000 [RESPONSE] [2024-05-26T13:23:20.835] [ Duration 3.008ms TTFB 2.989ms ↑ 1.9 KiB  ↓ 319 B ]
127.0.0.1:6000 201 Created
127.0.0.1:6000 X-Amz-Id-2: 726abb4f16358ae8a3c79dbe4408cc087c01b37ec7f880f6f4abad03bc208938
127.0.0.1:6000 X-Amz-Request-Id: 17D32393D6F94138
127.0.0.1:6000 X-Content-Type-Options: nosniff
127.0.0.1:6000 ETag: "14758f1afd44c09b7992073ccf00b43d"
127.0.0.1:6000 Location: http://127.0.0.1:6000/my-bucket/temp-22e063cd-4ed3-437c-87ac-2bc6e6f328d0/my_file.txt
127.0.0.1:6000 Strict-Transport-Security: max-age=31536000; includeSubDomains
127.0.0.1:6000 Vary: Origin,Accept-Encoding
127.0.0.1:6000 X-Xss-Protection: 1; mode=block
127.0.0.1:6000 Accept-Ranges: bytes
127.0.0.1:6000 Content-Length: 319
127.0.0.1:6000 Content-Type: application/xml
127.0.0.1:6000 Server: MinIO
127.0.0.1:6000 <BLOB>
127.0.0.1:6000 

Context

I want to tag the files to use to apply lifecycle rules.

Your Environment

• Version used (minio --version): RELEASE.2024-05-10T01-41-38Z (go1.22.3 darwin/amd64)
• Server setup and configuration: Python 3.10 with boto3
• Operating System and version (uname -a): macOS 13.6

[harshavardhana]

@orientalperil try PutObjectTags API instead or use PutObject API by passing tags as metadata

[harshavardhana]

    "tagging": get_tag_xml(RETENTION, EXPIRE_FAST),

where did you get this example?

[orientalperil]

Yeah I know you can use the Python client to do it, but my goal is to make a presigned POST URL that the browser can use to upload a file and have the tags applied through that at the same time. I don't want to write more code to look for the files later and tag them.

[orientalperil]

    "tagging": get_tag_xml(RETENTION, EXPIRE_FAST),

where did you get this example?

I can't remember but I think I found this example of using the field in a presigned POST URL with XML for tagging somewhere on stackoverflow. It is supported by S3 so Minio should be able to do this too right?

[harshavardhana]

This is unsupported so it won't work just yet.

[jiuker]

fixed by #19816