cognitect-labs / aws-api

AWS, data driven
Apache License 2.0
724 stars 100 forks source link

SQS: impossible to create a FIFO queue #226

Closed kirked closed 1 year ago

kirked commented 1 year ago

Thank you for your interest in helping to improve Cognitect's aws-api!

Dependencies

{:deps
 {com.cognitect.aws/api       {:mvn/version "0.8.635"}
  com.cognitect.aws/endpoints {:mvn/version "1.1.12.373"}
  com.cognitect.aws/sqs       {:mvn/version "822.2.1109.0"}
  org.clojure/clojure         {:mvn/version "1.11.1"}}}

Description with failing test case

It seems request-validation behaves improperly for creating FIFO queues in SQS using the :CreateQueue operation, making it impossible to create a FIFO queue.

The validation spec requires "alphanumeric characters, hyphens, or underscores. 1 to 80 in length". This is correct for non-FIFO queues, but doesn't take into account that if the {:FifoQueue "true"} option is set in the :CreateQueue call, that the name MUST end in .fifo (see documentation of the QueueName parameter on this page: https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_CreateQueue.html)

REPL Testcase

(require '[cognitect.aws.client.api :as aws])
(def sqs (aws/client {:api :sqs}))
(aws/invoke sqs {:op :CreateQueue
                 :request {:QueueName "test-queue.fifo"
                           :FifoQueue "true"}})
{:ErrorResponse
 {:Error
  {:Code "InvalidParameterValue",
   :CodeAttrs {},
   :Message
   "Can only include alphanumeric characters, hyphens, or underscores. 1 to 80 in length",
   :MessageAttrs {},
   :Type "Sender",
   :TypeAttrs {}},
  :ErrorAttrs {},
  :RequestId "VWRO3EVWYC5OQOFGD8BSFGHQNFBVJMGNW085REZRCKINDNOYPWSI",
  :RequestIdAttrs {}},
 :ErrorResponseAttrs
 {:xmlns "http://queue.amazonaws.com/doc/2012-11-05/"},
 :cognitect.anomalies/category :cognitect.anomalies/incorrect}

Stack traces

(none)

dchelimsky commented 1 year ago

Your example is formatting the request incorrectly:

> (aws/doc sqs :CreateQueue)
-------------------------
CreateQueue

<p>Creates a new standard or FIFO queue. You can pass one or more attributes in the request. Keep the following in mind:</p> <ul> <li> <p>If you don't specify the <code>FifoQueue</code> attribute, Amazon SQS creates a standard queue.</p> <note> <p>You can't change the queue type after you create it and you can't convert an existing standard queue into a FIFO queue. You must either create a new FIFO queue for your application or delete your existing standard queue and recreate it as a FIFO queue. For more information, see <a href="https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/FIFO-queues.html#FIFO-queues-moving">Moving From a Standard Queue to a FIFO Queue</a> in the <i>Amazon SQS Developer Guide</i>. </p> </note> </li> <li> <p>If you don't provide a value for an attribute, the queue is created with the default value for the attribute.</p> </li> <li> <p>If you delete a queue, you must wait at least 60 seconds before creating a queue with the s
ame name.</p> </li> </ul> <p>To successfully create a new queue, you must provide a queue name that adheres to the <a href="https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/limits-queues.html">limits related to queues</a> and is unique within the scope of your queues.</p> <note> <p>After you create a queue, you must wait at least one second after the queue is created to be able to use the queue.</p> </note> <p>To get the queue URL, use the <code> <a>GetQueueUrl</a> </code> action. <code> <a>GetQueueUrl</a> </code> requires only the <code>QueueName</code> parameter. be aware of existing queue names:</p> <ul> <li> <p>If you provide the name of an existing queue along with the exact names and values of all the queue's attributes, <code>CreateQueue</code> returns the queue URL for the existing queue.</p> </li> <li> <p>If the queue name, attribute names, or attribute values don't match an existing queue, <code>CreateQueue</code> returns an error.</p> </li> </ul> <p>Some actions take lists of parameters. These lists are specified using the <code>param.n</code> notation. Values of <code>n</code> are integers starting from 1. For example, a parameter list with two elements looks like this:</p> <p> <code>&amp;AttributeName.1=first</code> </p> <p> <code>&amp;AttributeName.2=second</code> </p> <note> <p>Cross-account permissions don't apply to this action. For more information, see <a href="https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-customer-managed-policy-examples.html#grant-cross-account-permissions-to-role-and-user-name">Grant cross-account permissions to a role and a user name</a> in the <i>Amazon SQS Developer Guide</i>.</p> </note>

-------------------------
Request

{:QueueName string,
 :Attributes
 [:map-of
  [:one-of
   ["All"
    "Policy"
    "VisibilityTimeout"
    "MaximumMessageSize"
    "MessageRetentionPeriod"
    "ApproximateNumberOfMessages"
    "ApproximateNumberOfMessagesNotVisible"
    "CreatedTimestamp"
    "LastModifiedTimestamp"
    "QueueArn"
    "ApproximateNumberOfMessagesDelayed"
    "DelaySeconds"
    "ReceiveMessageWaitTimeSeconds"
    "RedrivePolicy"
    "FifoQueue"
    "ContentBasedDeduplication"
    "KmsMasterKeyId"
    "KmsDataKeyReusePeriodSeconds"
    "DeduplicationScope"
    "FifoThroughputLimit"
    "RedriveAllowPolicy"
    "SqsManagedSseEnabled"]]
  string],
 :tags [:map-of string string]}

Required

[:QueueName]

-------------------------
Response

{:QueueUrl string}

nil
clj꞉cognitect.scratch꞉> 

The following works:

(require '[cognitect.aws.client.api :as aws])
(def sqs (aws/client {:api :sqs}))
(aws/invoke sqs {:op :CreateQueue
                 :request {:QueueName "test-queue.fifo"
                           :Attributes {"FifoQueue" "true"}}})
kirked commented 1 year ago

Geez, thanks David. I'm a bit bleary-eyed this afternoon. 😵‍💫

dchelimsky commented 1 year ago

You're welcome. It can be confusing since the aws-api docs sometimes align with the AWS docs, and sometimes they don't. They're usually pretty close the aws cli docs, but even that's not 100%, so your best bet is (aws/doc <client> op).