S3 GetObjectAttributes - meeting operation spec results in failure

[lowecg]

Dependencies

{:deps {com.cognitect.aws/api       {:mvn/version "0.8.652"}
        com.cognitect.aws/endpoints {:mvn/version "1.1.12.415"}
        com.cognitect.aws/s3        {:mvn/version "825.2.1250.0"}}}

The request format mandated by the spec for S3 GetObjectAttributes fails.

 Repro case

(require '[clojure.core.async :as a])
(require '[clojure.spec.alpha :as s])
(require '[clojure.spec.gen.alpha :as gen])
(require '[clojure.java.io :as io])
(require '[clojure.repl :as repl])
(require '[cognitect.aws.client.api :as was])

(def s3 (aws/client {:api :s3}))

(aws/validate-requests s3 true)

(aws/doc s3 :GetObjectAttributes)

;; Spec states that objects are supplied using a collection:

;; -------------------------
;;GetObjectAttributes
;;<snip/>
;;-------------------------
;;Request
;;
;;{:ObjectAttributes
;; [:seq-of
;;  [:one-of
;;   ["ETag" "Checksum" "ObjectParts" "StorageClass" "ObjectSize"]]],
;; :Bucket string,
;; :Key string,
;; :SSECustomerKeyMD5 string,
;; :MaxParts integer,
;; :SSECustomerKey blob,
;; :RequestPayer [:one-of ["requester"]],
;; :ExpectedBucketOwner string,
;; :PartNumberMarker integer,
;; :SSECustomerAlgorithm string,
;; :VersionId string}
;;
;;Required
;;
;;[:Bucket :Key :ObjectAttributes]

(def test-bucket "any for your environment")
(def test-key "any for your environment")

;; 1. To meet the spec, results in API failure:
(aws/invoke s3 {:op :GetObjectAttributes :request {:Bucket test-bucket :Key test-key :ObjectAttributes #{"ObjectSize"}}})

;; {:Error {:HostIdAttrs {},
;;         :ArgumentValueAttrs {},
;;         :Message "Invalid attribute name specified.",
;;         :CodeAttrs {},
;;         :RequestIdAttrs {},
;;         :HostId "xxx,
;;         :MessageAttrs {},
;;         :RequestId "xxx",
;;         :ArgumentNameAttrs {},
;;         :ArgumentName "x-amz-object-attributes",
;;         :ArgumentValue "#{\"ObjectSize\"}",
;;         :Code "InvalidArgument"},
;; :ErrorAttrs {},
;; :cognitect.anomalies/category :cognitect.anomalies/incorrect}

;; Looking at the encoded argument value looks like the collection was pr-str'd directly. A vector results in a similar response

;; 2. change the op-map to remove the collection

(aws/invoke s3 {:op :GetObjectAttributes :request {:Bucket test-bucket :Key test-key :ObjectAttributes "ObjectSize"}})

;; {:clojure.spec.alpha/problems ({:path [:ObjectAttributes],
;;                                :pred clojure.core/coll?,
;;                                :val "ObjectSize",
;;                                :via [:cognitect.aws.s3/GetObjectAttributesRequest
;;                                      :cognitect.aws.s3/ObjectAttributesList],
;;                                :in [:ObjectAttributes]}),
;; :clojure.spec.alpha/spec :cognitect.aws.s3/GetObjectAttributesRequest,
;; :clojure.spec.alpha/value {:Bucket "your bucket",
;;                            :Key "your key",
;;                            :ObjectAttributes "ObjectSize"},
;; :cognitect.anomalies/category :cognitect.anomalies/incorrect}

Workaround

Workaround is to disable validation and send a CSV list of attributes directly

(aws/validate-requests s3 false)

(aws/invoke s3 {:op :GetObjectAttributes :request {:Bucket test-bucket :Key test-key :ObjectAttributes (string/join "," ["ObjectSize"])}})

{:LastModified #inst"2023-03-18T10:07:04.000-00:00", :ObjectSize 616057}

[dchelimsky]

Thanks for the solid report.

[scottbale]

Hello @lowecg, just a quick update that @dchelimsky and I have begun working on this, we've possibly found the issue ... but we're coincidentally both going to be out of office next week. But fingers crossed we'll have some progress to report soon after that. 🤞

[scottbale]

Good news @lowecg - we believe we've solved this in the just-released version 0.8.686.

I'm going to close this issue but if you find anything amiss please re-open or at-mention us. Thanks again for reporting this.