support kwargs for params without having to create a Dict

[oxinabox]

Every function in the API takes a param::Dict{Any,String}

It would be useful if we could also accept normal julia keyword arguments and have that get wrapped up into a param Dict automatically. via params = Dict{String, Any}(string(k)=>v for (k,v) in pairs(kwargs))

[mattBrzezinski]

Yes, this would be nice to do! One annoying thing is the usage of hyphens in optional parameters for instance S3.list_buckets() has a max-keys optional parameter (among others).

We would need to introduce some check to convert _ to -, but otherwise this is very doable.

[mattBrzezinski]

I'm looking into this now, and this won't be easily possible to do. The issue stems from certain services having argument names with hyphens, or underscores, or both.

For example, SSO has an argument called x-amz-sso_bearer_token. If we want this to be a kwarg we could call it x_amz_sso_bearer_token, but we then need to convert this back to x-amz-sso_bearer_token, but its' not as simple as just converting underscores to hyphens as we only want to do this conversion for some.

We would essentially need to create some master list of expected_name => valid_julia_symbol, which seems like a lot of work.

AWS' inconsistencies strike again!

[oxinabox]

What if we translated - into _, and _ into __ (double underscore)?

[mattBrzezinski]

What if we translated - into _, and _ into __ (double underscore)?

That works! I don’t really like going down this gross path but I suppose it’s the only real option.

[oxinabox]

I think var"foo-bar_qux" might work for kwarg names but I haven't checked. I think hyphens are more common than underscores so that seams less nice to use?

[omus]

How about this as an alternative? Use varargs to make an interface that is keyword-like but avoids having to mess with translation:

julia> f(aws_kwargs::Pair{String,<:Any}...) = Dict(aws_kwargs)
f (generic function with 1 method)

julia> f("x-amz-sso_bearer_token" => "1234")
Dict{String, String} with 1 entry:
  "x-amz-sso_bearer_token" => "1234"

I think this is pretty reasonable from a user interface perspective and avoids having to learn the special translation logic when reading AWS documentation. The only real downside here is a little bit of processing required to be done.

[mattBrzezinski]

How about this as an alternative? Use varargs to make an interface that is keyword-like but avoids having to mess with translation:

julia> f(aws_kwargs::Pair{String,<:Any}...) = Dict(aws_kwargs)
f (generic function with 1 method)

julia> f("x-amz-sso_bearer_token" => "1234")
Dict{String, String} with 1 entry:
  "x-amz-sso_bearer_token" => "1234"

I think this is pretty reasonable from a user interface perspective and avoids having to learn the special translation logic when reading AWS documentation. The only real downside here is a little bit of processing required to be done.

It's late, I'm tired, so I might be missing something here. In the example above we would want aws_kwargs to actually be a kwargs..., since all these values should be optional arguments to the request we make. An example,

combine(; kwargs...) = Dict{String, Any}(string(k)=>v for (k,v) in pairs(kwargs))

function S3.list_objects(bucket; aws_config=global_aws_config(), kwargs...)
    return s3("GET", "$(bucket)", combine(kwargs...); aws_config=aws_config)
end

This way the user can just do ? S3.list_objects, see whatever optional arguments there are and pass them along as,

S3.list_objects("my-bucket"; max-keys=1)

But obviously that won't work :/ since there's a hyphen in max-keys. We could do something with the InputShape for each endpoint, but briefly thinking about it, we probably wouldn't get much closer to having something nice.

[omus]

In the example above we would want aws_kwargs to actually be a kwargs..., since all these values should be optional arguments to the request we make.

My alternative is suggesting forgoing using Julia keyword arguments for something keyword-like in order to avoid having to conform to Julia's requirements for variable names. For your posted example:

function S3.list_objects(bucket, aws_kwargs::Pair{String,<:Any}...; aws_config=global_aws_config())
    return s3("GET", "$(bucket)", Dict(aws_kwargs...); aws_config=aws_config)
end

And then call it with:

S3.list_objects("my-bucket", "max-keys" => 1)

The advantage I for this approach is no translation is required to go from Julia keyword to AWS optional argument. Also note this code is fully functional as written.

This way the user can just do ? S3.list_objects, see whatever optional arguments there are and pass them along

You should also be able to document optional arguments as part of the docstring using this proposal.

[oxinabox]

I think var"foo-bar_qux" might work for kwarg names but I haven't checked.

Confirmed this does work

julia> bar(;kwargs...) = kwargs
bar (generic function with 1 method)

julia> bar(var"ab-cd_ef"=2)
pairs(::NamedTuple) with 1 entry:
  Symbol("ab-cd_ef") => 2

 julia> qux(; var"ab-cd_ef"=nothing) = var"ab-cd_ef"
qux (generic function with 1 method)

julia> qux(; var"ab-cd_ef"=42)
42

Not nesc sure that this is better than translating - and _, or using Vararg{Pair{String, Any}} but it is another option.

[mattBrzezinski]

Ah I see thanks for clarifying! I'm a bit hesitant to put in the work to change from a Dict to a Pair as they're both pretty similar? Idk if you have thoughts on this @oxinabox?

The var"ab-cd_ef" solution could work, and I feel like this is in a bit of a better direction? But I feel like it would still be a bit awkward for the end user when they're doing something like:

using AWS
@service S3

resp = S3.list_bucket("my-bucket"; var"max-keys"=1, delimiter="/", prefix="pref")

Although, maybe this is a better step forward? At least if you know an optional argument contains a hyphen you need to preface it with var"optional-arg"?

[omus]

If you really, really want to use keywords then you could make use of a macro to allow the use of - as part of a keyword. Here is some quick code to show that the macro can capture the hyphenated keyword as a single expression:

macro foo(exprs...)
           @show exprs
           :(nothing)
       end
@foo (macro with 1 method)

julia> @foo(; max-keys=1, ab-cd_ef-gh=2)
exprs = (:($(Expr(:parameters, :($(Expr(:kw, :(max - keys), quote
    #= REPL[4]:1 =#
    1
end))), :($(Expr(:kw, :((ab - cd_ef) - gh), quote
    #= REPL[4]:1 =#
    2
end)))))),)

There needs to be some additional design here like deciding if you have S3.@list_bucket or have a generic macro that can be used with any AWS call @aws_kwarg S3.list_bucket(...; max-keys=1)

[mattBrzezinski]

Personally I think for an endpoint required arguments should be positional, and any optional ones should be keywords. Not sure what others think of this, or what their opinions are. I might not be the best use case for determining this, and maybe other users of the package should provide input.

Seems like there are a few options here,

resp = S3.list_objects("my-bucket", Dict("max-keys"=>1))

resp = S3.list_objects("my-bucket", ("max-keys"=>1))

resp = S3.list_objects("my-bucket"; var"max-keys"=1)

resp = @kw S3.list_objects("my-bucket"; max-keys=1)

resp = S3.@list_objects("my-bucket"; max-keys=1)

[oxinabox]

Can we pull stats on how common _ and - are in arguments? Ideally broken down by service (since tbh i don't care about what is common in GroundControl, but i definately do for CloudWatch)

• If _ is rare and - common, I lean towards the translate - to _ and __ to -. (since you would rarely need to remember the rule beyond - <=>_)
• if _ is common and - is rare, I lean towards the var"ab_cd-ef" solution. (Since you would rarely need to use var since most things wouldn't have -)
• if both rare then I lean still lean towards var"ab_cd-ef" solution
• if both common then I lean towards the Vararg{Pair{String}} solution. (it's a bit less nice than being able to use kwargs, but its pretty good)

[mattBrzezinski]

Yep I can pull numbers down, just off-hand looking at it, seems like - is rare, only used in S3 and for various header specific parameters. _ is used throughout various services quite often.

[mattBrzezinski]

All services containing parameters with either a hyphen, underscore, or both in them. Note, SSO is the only service which has both.

[
  {
    "clouddirectory-2017-01-11.normal.json": {
      "-": 64,
      "both": 0,
      "_": 0
    }
  },
  {
    "mobile-2017-07-01.normal.json": {
      "-": 3,
      "both": 0,
      "_": 0
    }
  },
  {
    "chime-2018-05-01.normal.json": {
      "-": 100,
      "both": 0,
      "_": 0
    }
  },
  {
    "runtime.lex-2016-11-28.normal.json": {
      "-": 32,
      "both": 0,
      "_": 0
    }
  },
  {
    "cloudsearchdomain-2013-01-01.normal.json": {
      "-": 1,
      "both": 0,
      "_": 0
    }
  },
  {
    "mq-2017-11-27.normal.json": {
      "-": 17,
      "both": 0,
      "_": 0
    }
  },
  {
    "mobileanalytics-2014-06-05.normal.json": {
      "-": 2,
      "both": 0,
      "_": 0
    }
  },
  {
    "qldb-2019-01-02.normal.json": {
      "-": 0,
      "both": 0,
      "_": 8
    }
  },
  {
    "greengrass-2017-06-07.normal.json": {
      "-": 24,
      "both": 0,
      "_": 0
    }
  },
  {
    "location-2020-11-19.normal.json": {
      "-": 4,
      "both": 0,
      "_": 0
    }
  },
  {
    "quicksight-2018-04-01.normal.json": {
      "-": 59,
      "both": 0,
      "_": 0
    }
  },
  {
    "glacier-2012-06-01.normal.json": {
      "-": 21,
      "both": 0,
      "_": 0
    }
  },
  {
    "lookoutvision-2020-11-20.normal.json": {
      "-": 10,
      "both": 0,
      "_": 0
    }
  },
  {
    "polly-2016-06-10.normal.json": {
      "-": 2,
      "both": 0,
      "_": 0
    }
  },
  {
    "runtime.sagemaker-2017-05-13.normal.json": {
      "-": 9,
      "both": 0,
      "_": 0
    }
  },
  {
    "appconfig-2019-10-09.normal.json": {
      "-": 8,
      "both": 0,
      "_": 15
    }
  },
  {
    "mediapackage-vod-2018-11-07.normal.json": {
      "-": 3,
      "both": 0,
      "_": 0
    }
  },
  {
    "customer-profiles-2020-08-15.normal.json": {
      "-": 16,
      "both": 0,
      "_": 0
    }
  },
  {
    "codeguruprofiler-2019-07-18.normal.json": {
      "-": 3,
      "both": 0,
      "_": 0
    }
  },
  {
    "nimble-2020-08-01.normal.json": {
      "-": 22,
      "both": 0,
      "_": 0
    }
  },
  {
    "lambda-2015-03-31.normal.json": {
      "-": 6,
      "both": 0,
      "_": 0
    }
  },
  {
    "apigateway-2015-07-09.normal.json": {
      "-": 4,
      "both": 0,
      "_": 176
    }
  },
  {
    "lex-models-2017-04-19.normal.json": {
      "-": 0,
      "both": 0,
      "_": 2
    }
  },
  {
    "dataexchange-2017-07-25.normal.json": {
      "-": 3,
      "both": 0,
      "_": 0
    }
  },
  {
    "amplifybackend-2020-08-11.normal.json": {
      "-": 0,
      "both": 0,
      "_": 6
    }
  },
  {
    "connectparticipant-2018-09-07.normal.json": {
      "-": 8,
      "both": 0,
      "_": 0
    }
  },
  {
    "kinesis-video-archived-media-2017-09-30.normal.json": {
      "-": 2,
      "both": 0,
      "_": 0
    }
  },
  {
    "medialive-2017-10-14.normal.json": {
      "-": 6,
      "both": 0,
      "_": 0
    }
  },
  {
    "apigatewayv2-2018-11-29.normal.json": {
      "-": 3,
      "both": 0,
      "_": 0
    }
  },
  {
    "cloudfront-2020-05-31.normal.json": {
      "-": 23,
      "both": 0,
      "_": 0
    }
  },
  {
    "s3control-2018-08-20.normal.json": {
      "-": 60,
      "both": 0,
      "_": 0
    }
  },
  {
    "iot-2015-05-28.normal.json": {
      "-": 8,
      "both": 0,
      "_": 0
    }
  },
  {
    "cognito-sync-2014-06-30.normal.json": {
      "-": 1,
      "both": 0,
      "_": 0
    }
  },
  {
    "iot1click-devices-2018-05-14.normal.json": {
      "-": 3,
      "both": 0,
      "_": 0
    }
  },
  {
    "pinpoint-2016-12-01.normal.json": {
      "-": 178,
      "both": 0,
      "_": 0
    }
  },
  {
    "mediastore-data-2017-09-01.normal.json": {
      "-": 13,
      "both": 0,
      "_": 0
    }
  },
  {
    "kinesis-video-media-2017-09-30.normal.json": {
      "-": 1,
      "both": 0,
      "_": 0
    }
  },
  {
    "runtime.lex.v2-2020-08-07.normal.json": {
      "-": 17,
      "both": 0,
      "_": 0
    }
  },
  {
    "schemas-2019-12-02.normal.json": {
      "-": 3,
      "both": 0,
      "_": 0
    }
  },
  {
    "s3-2006-03-01.normal.json": {
      "-": 467,
      "both": 0,
      "_": 0
    }
  },
  {
    "mediapackage-2017-10-12.normal.json": {
      "-": 3,
      "both": 0,
      "_": 1
    }
  },
  {
    "ebs-2019-11-02.normal.json": {
      "-": 13,
      "both": 0,
      "_": 0
    }
  },
  {
    "sso-2019-06-10.normal.json": {
      "-": 0,
      "both": 4,
      "_": 7
    }
  },
  {
    "codeartifact-2018-09-22.normal.json": {
      "-": 51,
      "both": 0,
      "_": 0
    }
  }
]

[oxinabox]

I think we are in this case:

If is rare and - common, I lean towards the translate - to and _ to -. (since you would rarely need to remember the rule beyond - <=>)

It seems _ is rare at all, and never used in a service that I care about. Unless API Gateway is a super important service, and needs you to constantly configure things with those optional arguments.

[omus]

Although unlikely do check for any params named aws-config that could conflict with the AWS.jl aws_config.

Aside: I'll note I kind of always felt that the aws_config should be the optional first argument for the functions (e.g. define f(::AWSConfig, args...) and f(args...))

[ericphanson]

just to say I like the optional pairs as trailing position arguments (https://github.com/JuliaCloud/AWS.jl/issues/341#issuecomment-832704581). JuMP uses this pattern too for passing options to solvers: https://jump.dev/JuMP.jl/v0.21.10/reference/models/#JuMP.set_optimizer_attributes (which is very similar in that the names are defined by external binaries most the time).

[omus]

I'll mention that boto3 uses CamelCase for its keyword arguments. For example list_objects_v2 takes MaxKeys as an argument. We may also want to adopt this format as the response to list_object_v2 calls includes "MaxKeys". It would be nice for these to be consistent.

[iamed2]

Boto3 gets its argument names from the API files: https://github.com/aws/aws-sdk-js/blob/a42ad578686801607b88358b7a85da48a8952bdc/apis/s3-2006-03-01.normal.json#L6516

I think we probably should too.

[mattBrzezinski]

I'm coming back around to this now, and have a couple more thoughts.

1) Some parameters passed to endpoints must be in specific locations, such as in headers. Example, this does not seem to be the case for an optional parameters so we should be in the clear for that.

2) Using the boto3 CamelCasing naming is a bit more challenging, in your example of MaxKeys the real name that AWS wants when making the request is actually max-keys. I've only spent a few minutes thinking about this, the only way I see us getting this to work is by,

• Maintaining a list of CamelCase -> location_name mappings
• When kwargs are passed in, translate them into the location_name
• Make the request

That seems like a lot of work, although it would be nice to have. Does anyone see another way of doing this better? I'm happy to move forward with Pairs() as this seems like it both cleans things up, and is the most straight forward path.

[iamed2]

I had some waiting time last week so I found out how botocore actually does this, and it is that maintained list. I think it's the best choice and a reasonable change to make at this stage of development, but should be isolated to the higher-level load-on-demand service APIs, otherwise we're stuck with a massive new dataset. I'll note that boto doesn't have any notion of a client at a lower level than our high-level clients (i.e. no s3 function), so they always perform this process when a service is used.

Didn't have time to write up my design findings but here's an edited dump of my exploratory session:

import boto3
s3 = boto3.client("s3")
dir(s3)
# OUT: ['_PY_TO_OP_NAME', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattr__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_cache', '_client_config', '_convert_to_request_dict', '_emit_api_params', '_endpoint', '_exceptions', '_exceptions_factory', '_get_waiter_config', '_load_exceptions', '_loader', '_make_api_call', '_make_request', '_register_handlers', '_request_signer', '_response_parser', '_serializer', '_service_model', 'abort_multipart_upload', 'can_paginate', 'complete_multipart_upload', 'copy', 'copy_object', 'create_bucket', 'create_multipart_upload', 'delete_bucket', 'delete_bucket_analytics_configuration', 'delete_bucket_cors', 'delete_bucket_encryption', 'delete_bucket_intelligent_tiering_configuration', 'delete_bucket_inventory_configuration', 'delete_bucket_lifecycle', 'delete_bucket_metrics_configuration', 'delete_bucket_ownership_controls', 'delete_bucket_policy', 'delete_bucket_replication', 'delete_bucket_tagging', 'delete_bucket_website', 'delete_object', 'delete_object_tagging', 'delete_objects', 'delete_public_access_block', 'download_file', 'download_fileobj', 'exceptions', 'generate_presigned_post', 'generate_presigned_url', 'get_bucket_accelerate_configuration', 'get_bucket_acl', 'get_bucket_analytics_configuration', 'get_bucket_cors', 'get_bucket_encryption', 'get_bucket_intelligent_tiering_configuration', 'get_bucket_inventory_configuration', 'get_bucket_lifecycle', 'get_bucket_lifecycle_configuration', 'get_bucket_location', 'get_bucket_logging', 'get_bucket_metrics_configuration', 'get_bucket_notification', 'get_bucket_notification_configuration', 'get_bucket_ownership_controls', 'get_bucket_policy', 'get_bucket_policy_status', 'get_bucket_replication', 'get_bucket_request_payment', 'get_bucket_tagging', 'get_bucket_versioning', 'get_bucket_website', 'get_object', 'get_object_acl', 'get_object_legal_hold', 'get_object_lock_configuration', 'get_object_retention', 'get_object_tagging', 'get_object_torrent', 'get_paginator', 'get_public_access_block', 'get_waiter', 'head_bucket', 'head_object', 'list_bucket_analytics_configurations', 'list_bucket_intelligent_tiering_configurations', 'list_bucket_inventory_configurations', 'list_bucket_metrics_configurations', 'list_buckets', 'list_multipart_uploads', 'list_object_versions', 'list_objects', 'list_objects_v2', 'list_parts', 'meta', 'put_bucket_accelerate_configuration', 'put_bucket_acl', 'put_bucket_analytics_configuration', 'put_bucket_cors', 'put_bucket_encryption', 'put_bucket_intelligent_tiering_configuration', 'put_bucket_inventory_configuration', 'put_bucket_lifecycle', 'put_bucket_lifecycle_configuration', 'put_bucket_logging', 'put_bucket_metrics_configuration', 'put_bucket_notification', 'put_bucket_notification_configuration', 'put_bucket_ownership_controls', 'put_bucket_policy', 'put_bucket_replication', 'put_bucket_request_payment', 'put_bucket_tagging', 'put_bucket_versioning', 'put_bucket_website', 'put_object', 'put_object_acl', 'put_object_legal_hold', 'put_object_lock_configuration', 'put_object_retention', 'put_object_tagging', 'put_public_access_block', 'restore_object', 'select_object_content', 'upload_file', 'upload_fileobj', 'upload_part', 'upload_part_copy', 'waiter_names']
s3._service_model.operation_names
# OUT: ['AbortMultipartUpload', 'CompleteMultipartUpload', 'CopyObject', 'CreateBucket', 'CreateMultipartUpload', 'DeleteBucket', 'DeleteBucketAnalyticsConfiguration', 'DeleteBucketCors', 'DeleteBucketEncryption', 'DeleteBucketIntelligentTieringConfiguration', 'DeleteBucketInventoryConfiguration', 'DeleteBucketLifecycle', 'DeleteBucketMetricsConfiguration', 'DeleteBucketOwnershipControls', 'DeleteBucketPolicy', 'DeleteBucketReplication', 'DeleteBucketTagging', 'DeleteBucketWebsite', 'DeleteObject', 'DeleteObjectTagging', 'DeleteObjects', 'DeletePublicAccessBlock', 'GetBucketAccelerateConfiguration', 'GetBucketAcl', 'GetBucketAnalyticsConfiguration', 'GetBucketCors', 'GetBucketEncryption', 'GetBucketIntelligentTieringConfiguration', 'GetBucketInventoryConfiguration', 'GetBucketLifecycle', 'GetBucketLifecycleConfiguration', 'GetBucketLocation', 'GetBucketLogging', 'GetBucketMetricsConfiguration', 'GetBucketNotification', 'GetBucketNotificationConfiguration', 'GetBucketOwnershipControls', 'GetBucketPolicy', 'GetBucketPolicyStatus', 'GetBucketReplication', 'GetBucketRequestPayment', 'GetBucketTagging', 'GetBucketVersioning', 'GetBucketWebsite', 'GetObject', 'GetObjectAcl', 'GetObjectLegalHold', 'GetObjectLockConfiguration', 'GetObjectRetention', 'GetObjectTagging', 'GetObjectTorrent', 'GetPublicAccessBlock', 'HeadBucket', 'HeadObject', 'ListBucketAnalyticsConfigurations', 'ListBucketIntelligentTieringConfigurations', 'ListBucketInventoryConfigurations', 'ListBucketMetricsConfigurations', 'ListBuckets', 'ListMultipartUploads', 'ListObjectVersions', 'ListObjects', 'ListObjectsV2', 'ListParts', 'PutBucketAccelerateConfiguration', 'PutBucketAcl', 'PutBucketAnalyticsConfiguration', 'PutBucketCors', 'PutBucketEncryption', 'PutBucketIntelligentTieringConfiguration', 'PutBucketInventoryConfiguration', 'PutBucketLifecycle', 'PutBucketLifecycleConfiguration', 'PutBucketLogging', 'PutBucketMetricsConfiguration', 'PutBucketNotification', 'PutBucketNotificationConfiguration', 'PutBucketOwnershipControls', 'PutBucketPolicy', 'PutBucketReplication', 'PutBucketRequestPayment', 'PutBucketTagging', 'PutBucketVersioning', 'PutBucketWebsite', 'PutObject', 'PutObjectAcl', 'PutObjectLegalHold', 'PutObjectLockConfiguration', 'PutObjectRetention', 'PutObjectTagging', 'PutPublicAccessBlock', 'RestoreObject', 'SelectObjectContent', 'UploadPart', 'UploadPartCopy']
om = s3._service_model.operation_model("ListObjectsV2")
om
# OUT: OperationModel(name=ListObjectsV2)
om.error_shapes
# OUT: [<StructureShape(NoSuchBucket)>]
om.documentation
# OUT: '<p>Returns some or all (up to 1,000) of the objects in a bucket. You can use the request parameters as selection criteria to return a subset of the objects in a bucket. A <code>200 OK</code> response can contain valid or invalid XML. Make sure to design your application to parse the contents of the response and handle it appropriately.</p> <p>To use this operation, you must have READ access to the bucket.</p> <p>To use this operation in an AWS Identity and Access Management (IAM) policy, you must have permissions to perform the <code>s3:ListBucket</code> action. The bucket owner has this permission by default and can grant this permission to others. For more information about permissions, see <a href="https://docs.aws.amazon.com/AmazonS3/latest/dev/using-with-s3-actions.html#using-with-s3-actions-related-to-bucket-subresources">Permissions Related to Bucket Subresource Operations</a> and <a href="https://docs.aws.amazon.com/AmazonS3/latest/dev/s3-access-control.html">Managing Access Permissions to Your Amazon S3 Resources</a>.</p> <important> <p>This section describes the latest revision of the API. We recommend that you use this revised API for application development. For backward compatibility, Amazon S3 continues to support the prior version of this API, <a href="https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjects.html">ListObjects</a>.</p> </important> <p>To get a list of your buckets, see <a href="https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListBuckets.html">ListBuckets</a>.</p> <p>The following operations are related to <code>ListObjectsV2</code>:</p> <ul> <li> <p> <a href="https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html">GetObject</a> </p> </li> <li> <p> <a href="https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObject.html">PutObject</a> </p> </li> <li> <p> <a href="https://docs.aws.amazon.com/AmazonS3/latest/API/API_CreateBucket.html">CreateBucket</a> </p> </li> </ul>'
s3._emit_api_params({"Bucket": "foo", "EncodingType": "bar"}, om, {})
# OUT: {'Bucket': 'foo', 'EncodingType': 'bar'}
api_params = s3._emit_api_params({"Bucket": "foo", "EncodingType": "bar"}, om, {})
request_dict = s3._serializer.serialize_to_request(api_params, om)
request_dict
# OUT: {'url_path': '/foo?list-type=2', 'query_string': {'encoding-type': 'bar'}, 'method': 'GET', 'headers': {}, 'body': b''}
s3._serializer
# OUT: <botocore.validate.ParamValidationDecorator object at 0x102dfc100>
s3._serializer._serializer
# OUT: <botocore.serialize.RestXMLSerializer object at 0x102dfc070>
om.input_shape
# OUT: <StructureShape(ListObjectsV2Request)>
om.input_shape.members
# OUT: OrderedDict([('Bucket', <StringShape(BucketName)>), ('Delimiter', <StringShape(Delimiter)>), ('EncodingType', <StringShape(EncodingType)>), ('MaxKeys', <Shape(MaxKeys)>), ('Prefix', <StringShape(Prefix)>), ('ContinuationToken', <StringShape(Token)>), ('FetchOwner', <Shape(FetchOwner)>), ('StartAfter', <StringShape(StartAfter)>), ('RequestPayer', <StringShape(RequestPayer)>), ('ExpectedBucketOwner', <StringShape(AccountId)>)])
om.input_shape.members['MaxKeys']
# OUT: <Shape(MaxKeys)>
mks = om.input_shape.members['MaxKeys']
mks.name
# OUT: 'MaxKeys'
mks.type_name
# OUT: 'integer'
mks.serialization
# OUT: {'location': 'querystring', 'name': 'max-keys'}
###

[omus]

Personally I think for an endpoint required arguments should be positional, and any optional ones should be keywords. Not sure what others think of this, or what their opinions are – https://github.com/JuliaCloud/AWS.jl/issues/341#issuecomment-832872955

We may want to consider using only keywords arguments for our AWS API parameters as this is how other AWS SDKs such as boto3 handle this.

I believe that our low-level interface should not perform any parameter translation. If we do want to support some kind of keyword translation in the high-level interface we also want to consider how we want to translate AWS API data types. For example CloudFormation's CreateStack API call uses the Parameter type and the Tag type. For example the low-level interface for could look like:

CloudFormation("CreateStack", "Parameters" => [Dict("ParameterKey" => k1, "ParameterValue" => v1), Dict("ParameterKey" => k2, "ParameterValue" => v2)])

where as a high-level interface could be

create_stack(parameters=(k1=v1, k2=v2))

I've opted to use NamedTuples for this high-level interface example due to the more terse syntax.

[iamed2]

I believe that our low-level interface should not perform any parameter translation

What do you mean by this? What do you consider to be the canonical parameter name?

As discussed above, AWS has the name for a parameter (e.g., MaxKeys, RequestPayer), and then the location of the parameter (e.g., max-keys in the querystring, x-amz-request-payer in the header). Could you provide an example like the one you did for CloudFormation (currently this is typed cloudformation, probably best to use that to avoid confusion with the higher-level interface) but for s3?