aws-beam / aws-elixir

AWS clients for Elixir
Apache License 2.0
555 stars 132 forks source link

AWS clients for Elixir

Actions Status Module Version Hex Docs Total Download License Last Updated

🌳 With this library you can have access to almost all AWS services without hassle. ⚡

Features

Usage

Here is an example of listing Amazon Kinesis streams. First you need to start a console with iex -S mix.

After that, type the following:

iex> client = AWS.Client.create("your-access-key-id", "your-secret-access-key", "us-east-1")
iex> {:ok, result, resp} = AWS.Kinesis.list_streams(client, %{})
iex> IO.inspect(result)
%{"HasMoreStreams" => false, "StreamNames" => []}

If you are using AWS.S3, you can upload a file with integrity check doing:

iex> client = AWS.Client.create("your-access-key-id", "your-secret-access-key", "us-east-1")
iex> file = File.read!("./tmp/your-file.txt")
iex> md5 = :crypto.hash(:md5, file) |> Base.encode64()
iex> AWS.S3.put_object(client, "your-bucket-name", "foo/your-file-on-s3.txt",
  %{"Body" => file, "ContentMD5" => md5})

Note that you may need to specify the ContentType attribute when calling AWS.S3.put_object/4. This is because S3 will use that to store the MIME type of the file.

You can also upload to S3 as multipart. If you're facing timeout issues, this strategy is recommended:

client = AWS.Client.create("your-access-key-id", "your-secret-access-key", "us-east-1")
bucket = "your-bucket-name"
filename = "./your-big-file.wav"
# AWS minimum chunk size is 5MB
chunk_size = 5_242_880

# Create the multipart request
{:ok,
 %{
   "InitiateMultipartUploadResult" => %{
     "UploadId" => upload_id
   }
 }, _} = AWS.S3.create_multipart_upload(client, bucket, filename, %{})

file = File.read!(filename)

# Send the file's binary in parts
parts =
  file
  |> String.codepoints()
  |> Stream.chunk_every(chunk_size)
  |> Stream.with_index(1)
  |> Enum.map(fn {chunk, i} ->
    chunk = Enum.join(chunk)

    {:ok, nil, %{headers: headers, status_code: 200}} =
      AWS.S3.upload_part(client, bucket, filename, %{
        "Body" => chunk,
        "PartNumber" => i,
        "UploadId" => upload_id
      })

    {_, etag} = Enum.find(headers, fn {header, _} -> header == "ETag" end)

    %{"ETag" => etag, "PartNumber" => i}
  end)

input = %{"CompleteMultipartUpload" => %{"Part" => parts}, "UploadId" => upload_id}

# Complete the multipart request
AWS.S3.complete_multipart_upload(aws_client, bucket, filename, input)

You can also list objects in a bucket:

# create the client just like the example above
iex> AWS.S3.list_objects_v2(client, "bucket-name-here")

And download a specific object:

# create the client just like the example above
# object key is the "file path" in the S3 bucket
iex> AWS.S3.get_object(client, "bucket-name-here", "object-key-here")

Check all S3 related functions in the docs here.

Remember to check the operation documentation for details: https://docs.aws.amazon.com/

Installation

Add :aws to your list of dependencies in mix.exs. It also requires hackney for the default HTTP client. Optionally, you can implement your own (Check AWS.Client docs).

def deps do
  [
    {:aws, "~> 1.0.0"},
    {:hackney, "~> 1.18"}
  ]
end

Run mix deps.get to install.

Development

Most of the code is generated using the aws-codegen library from the JSON descriptions of AWS services provided by Amazon. They can be found in lib/aws/generated.

Code outside lib/aws/generated is manually written and used as support for the generated code.

Documentation

Online

Local

Note: Arguments, errors and response structure can be found by viewing the model schemas used to generate this module at aws-sdk-go/models/apis/<aws-service>/<version>/.

An example is aws-sdk-go/models/apis/rekognition/2016-06-27/api-2.json. Alternatively you can access the documentation for the service you want at AWS docs page.

Tests

$ mix test

Release

Copyright & License

Copyright (c) 2015 Jamshed Kakar jkakar@kakar.ca

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.