Media Query Application using virtualenv, Chalice, Lambda functions, AWS CLI, Detect image labels using Rekognition, S3, DynamoDB, REST API, Event Actions, SNS Topics, IAM roles. Combine multiple event handlers in Chalice to create an image processing pipeline. It takes as input any image or video and it will identify objects, people, text, scenes, and activities. This results of this analysis can then be queried with a REST API

[coding-to-music]

Media Query Application - combine multiple event handlers in Chalice to create an image processing pipeline. It takes as input any image or video and it will identify objects, people, text, scenes, and activities. This results of this analysis can then be queried with a REST API

https://aws.github.io/chalice/samples/media-query/index.html

https://github.com/aws/chalice/tree/master/docs/source/samples/media-query/code/

Once the application is deployed, use the AWS CLI to

fetch the name of the bucket that is storing the media files:

aws cloudformation describe-stacks --stack-name media-query \
    --query "Stacks[0].Outputs[?OutputKey=='MediaBucketName'].OutputValue" \
    --output text

media-query-mediabucket-9rbalpyrst24

find the URL

chalice url

https://n4dkhj8yqg.execute-api.us-east-1.amazonaws.com/api/

List out all if the media files using the application’s API with HTTPie:

http https://n4dkhj8yqg.execute-api.us-east-1.amazonaws.com/api/

commands for the aws cli

usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:

  aws help
  aws <command> help
  aws <command> <subcommand> help

aws: error: argument subcommand: Invalid choice, valid choices are:

ls                                       | website                                 
cp                                       | mv                                      
rm                                       | sync                                    
mb                                       | rb                                      
presign                                 

aws s3 ls

aws s3 ls media-query-mediabucket-9rbalpyrst24

2021-07-12 23:03:39     111370 architecture.jpg
2021-07-12 23:03:49     111347 othersample.jpg
2021-07-12 23:03:18     111232 sample.jpg
2021-07-12 23:04:05     111288 sample.mp4

[coding-to-music]

video

https://youtube.com/embed/UCZXJpI1dKw

This sample application shows how to combine multiple event handlers in Chalice to create an image processing pipeline. It takes as input any image or video and it will identify objects, people, text, scenes, and activities. This results of this analysis can then be queried with a REST API.

There are several components of this application. The first part is an image processing pipeline. The application is registered to automatically process any media that’s uploaded to an Amazon S3 bucket. The application will then use Amazon Rekognition to automatically detect labels in either the image or the video. The returned labels are then stored in an Amazon DynamoDB table.

For videos, an asynchronous job is started. This is because the analysis for videos takes longer than analyzing images so we don’t want our Lambda function to block until the job is complete. To handle this asynchronous job, we subscribe to an Amazon SNS topic. When the asynchronous job is finished analyzing our uploaded video, an event handler is called that will retrieve the results and store the labels in Amazon DynamoDB.

The final component is the REST API. This allows users to query for labels associated with the media that has been uploaded.

You can find the full source code for this application in our samples directory on GitHub.

git clone git://github.com/aws/chalice
cd chalice/docs/source/samples/media-query/code

We’ll now walk through the architecture of the application, how to deploy and use the application, and go over the application code.

Note

This sample application is also available as a workshop. The main difference between the sample apps here and the Chalice workshops is that the workshop is a detailed step by step process for how to create this application from scratch. You build the app by gradually adding each feature piece by piece. It takes several hours to work through all the workshop material. In this document we review the architecture, the deployment process, then walk through the main sections of the code.

[coding-to-music]

Architecture

Below is the architecture for the application.

The main components of the application are as follows:

• handle_object_created: A Lambda function that is triggered when an object is uploaded to a S3 bucket. If the object is an image, it will call Amazon Rekognition’s DetectLabels API to detect objects in the image. With the detected objects, the Lambda function will then add the object to an Amazon DynamoDB table. If the object is a video, it will call Rekognition’s StartLabelDetection API to initiate an asynchronous job to detect labels in the video. When the job is completed, a completion notification is pushed to an SNS topic.
• handle_object_deleted: A Lambda function that removes the object from the DynamoDB table if the object is deleted from the S3 bucket.
• add_video_labels: A Lambda function that is triggered on video label detection SNS messages. On invocation, it will call Rekognition’s GetLabelDetection API to retrieve all detected objects from the video. It then adds the video with its labels to the DynamoDB Table
• api_handler: A Lambda function that is invoked by HTTP requests to Amazon API Gateway. On invocation, it will query the database based on the received HTTP request and return the results to the user through API Gateway.

[coding-to-music]

Deployment

First, we’ll setup our development environment by cloning the Chalice GitHub repository and copying the sample code in a new directory:

git clone git://github.com/aws/chalice
mkdir /tmp/demo
cp -r chalice/docs/source/samples/media-query/code/ /tmp/demo/media-query
cd /tmp/demo/media-query/

Next configure a virtual environment that uses Python 3. In this example we’re using Python 3.7.

python3 -m venv /tmp/venv37 
. /tmp/venv37/bin/activate

To deploy the application, first install the necessary requirements and install Chalice:

pip install -r requirements.txt
pip install chalice

We’ll also be using the AWS CLI to help deploy our application, you can follow the installation instructions if you don’t have the AWS CLI installed.

For the latest version of the AWS CLI, use the following command block:

curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install

Next, we’ll use the AWS CLI to deploy a CloudFormation stack containing the S3 bucket, DynamoDB table, and SNS topic needed to run this application:

aws cloudformation deploy --template-file resources.json \
    --stack-name media-query --capabilities CAPABILITY_IAM

Record the deployed resources as environment variables in the Chalice application by running the recordresources.py script:

python recordresources.py --stack-name media-query

You can see these values by looking at the .chalice/config.json file. Once those resources are created and recorded, deploy the Chalice application:

chalice deploy

(venv37) root@docker-ubuntu-s-1vcpu-2gb-nyc1-01:/tmp/demo/media-query# python recordresources.py --stack-name media-query
(venv37) root@docker-ubuntu-s-1vcpu-2gb-nyc1-01:/tmp/demo/media-query# 
(venv37) root@docker-ubuntu-s-1vcpu-2gb-nyc1-01:/tmp/demo/media-query# chalice deploy
Creating deployment package.
Updating policy for IAM role: media-query-dev-handle_object_created
Updating lambda function: media-query-dev-handle_object_created
Configuring S3 events in bucket media-query-mediabucket-9rbalpyrst24 to function media-query-dev-handle_object_created
Updating policy for IAM role: media-query-dev-handle_object_removed
Updating lambda function: media-query-dev-handle_object_removed
Configuring S3 events in bucket media-query-mediabucket-9rbalpyrst24 to function media-query-dev-handle_object_removed
Updating policy for IAM role: media-query-dev-add_video_file
Updating lambda function: media-query-dev-add_video_file
Subscribing media-query-dev-add_video_file to SNS topic media-query-VideoTopic-1EB8I6SZH3ZW4
Updating policy for IAM role: media-query-dev-api_handler
Updating lambda function: media-query-dev
Creating Rest API
Resources deployed:
  - Lambda ARN: arn:aws:lambda:us-east-1:708090526287:function:media-query-dev-handle_object_created
  - Lambda ARN: arn:aws:lambda:us-east-1:708090526287:function:media-query-dev-handle_object_removed
  - Lambda ARN: arn:aws:lambda:us-east-1:708090526287:function:media-query-dev-add_video_file
  - Lambda ARN: arn:aws:lambda:us-east-1:708090526287:function:media-query-dev
  - Rest API URL: https://n4dkhj8yqg.execute-api.us-east-1.amazonaws.com/api/
(venv37) root@docker-ubuntu-s-1vcpu-2gb-nyc1-01:/tmp/demo/media-query# chalice url
https://n4dkhj8yqg.execute-api.us-east-1.amazonaws.com/api/

[coding-to-music]

Using the Application

Once the application is deployed, use the AWS CLI to fetch the name of the bucket that is storing the media files:

aws cloudformation describe-stacks --stack-name media-query \
    --query "Stacks[0].Outputs[?OutputKey=='MediaBucketName'].OutputValue" \
    --output text
media-query-mediabucket-9rbalpyrst24

Upload some sample media files to your Amazon S3 bucket so the system populates information about the media files in your DynamoDB table. If you need sample media files, you can use the included samples from the corresponding Chalice workshop assets here.

aws s3 cp ../sample-images/sample.jpg s3://media-query-mediabucket-9rbalpyrst24/sample.jpg
aws s3 cp ../sample-images/sample.mp4 s3://media-query-mediabucket-9rbalpyrst24/sample.mp4
aws s3 cp ../sample-images/architecture.jpg s3://media-query-mediabucket-9rbalpyrst24/architecture.jpg
aws s3 cp ../sample-images/othersample.jpg s3://media-query-mediabucket-9rbalpyrst24/othersample.jpg

Wait about a minute for the media files to be populated in the database and then install HTTPie:

pip install httpie

find the URL

chalice url

https://n4dkhj8yqg.execute-api.us-east-1.amazonaws.com/api/

List out all if the media files using the application’s API with HTTPie:

http https://n4dkhj8yqg.execute-api.us-east-1.amazonaws.com/api/

HTTP/1.1 200 OK
Connection: keep-alive
Content-Length: 279
Content-Type: application/json
Date: Tue, 10 Jul 2018 17:58:40 GMT
Via: 1.1 fa751ee53e2bf18781ae98b293ff9375.cloudfront.net (CloudFront)
X-Amz-Cf-Id: sNnrzvbdvgj1ZraySJvfSUbHthC_fok8l5GJ7glV4QcED_M1c8tlvg==
X-Amzn-Trace-Id: Root=1-5b44f3d0-4546157e8f5e35a008d06d88;Sampled=0
X-Cache: Miss from cloudfront
x-amz-apigw-id: J0sIlHs3vHcFj9g=
x-amzn-RequestId: e0aaf4e1-846a-11e8-b756-99d52d342d60

[
    {
        "labels": [
            "Animal",
            "Canine",
            "Dog",
            "German Shepherd",
            "Mammal",
            "Pet",
            "Collie"
        ],
        "name": "sample.jpg",
        "type": "image"
    },
    {
        "labels": [
            "Human",
            "Clothing",
            "Dog",
            "Nest",
            "Person",
            "Footwear",
            "Bird Nest",
            "People",
            "Animal",
            "Husky"
        ],
        "name": "sample.mp4",
        "type": "video"
    }
]

You can include query string parameters as well to query all objects based on what the file name starts with, the type of the media file, and the detected objects in the media file:

http https://n4dkhj8yqg.execute-api.us-east-1.amazonaws.com/api/ startswith==sample.m

HTTP/1.1 200 OK
Connection: keep-alive
Content-Length: 153
Content-Type: application/json
Date: Tue, 10 Jul 2018 19:20:02 GMT
Via: 1.1 aa42484f82c16d99015c599631def20c.cloudfront.net (CloudFront)
X-Amz-Cf-Id: euqlOlWN5k5V_zKCJy4SL988Vcje6W5jDR88GrWr5uYGH-_ZvN4arg==
X-Amzn-Trace-Id: Root=1-5b4506e0-db041a3492ee56e8f3d9457c;Sampled=0
X-Cache: Miss from cloudfront
x-amz-apigw-id: J04DHE92PHcF--Q=
x-amzn-RequestId: 3d82319d-8476-11e8-86d9-a1e4585e5c26

[
    {
        "labels": [
            "Human",
            "Clothing",
            "Dog",
            "Nest",
            "Person",
            "Footwear",
            "Bird Nest",
            "People",
            "Animal",
            "Husky"
        ],
        "name": "sample.mp4",
        "type": "video"
    }
]

http https://n4dkhj8yqg.execute-api.us-east-1.amazonaws.com/api/ media-type==image

HTTP/1.1 200 OK
Connection: keep-alive
Content-Length: 126
Content-Type: application/json
Date: Tue, 10 Jul 2018 19:20:53 GMT
Via: 1.1 88eb066576c1b47cd896ab0019b9f25f.cloudfront.net (CloudFront)
X-Amz-Cf-Id: rwuOwzLKDM4KgcSBXFihWeNNsYSpZDYVpc8IXdT0xOu8qz8aA2Pj3w==
X-Amzn-Trace-Id: Root=1-5b450715-de71cf04ca2900b839ff1194;Sampled=0
X-Cache: Miss from cloudfront
x-amz-apigw-id: J04LaE6YPHcF3VA=
x-amzn-RequestId: 5d29d59a-8476-11e8-a347-ebb5d5f47789

[
    {
        "labels": [
            "Animal",
            "Canine",
            "Dog",
            "German Shepherd",
            "Mammal",
            "Pet",
            "Collie"
        ],
        "name": "sample.jpg",
        "type": "image"
    }
]

http https://n4dkhj8yqg.execute-api.us-east-1.amazonaws.com/api/ label==Person

HTTP/1.1 200 OK
Connection: keep-alive
Content-Length: 153
Content-Type: application/json
Date: Tue, 10 Jul 2018 19:20:02 GMT
Via: 1.1 aa42484f82c16d99015c599631def20c.cloudfront.net (CloudFront)
X-Amz-Cf-Id: euqlOlWN5k5V_zKCJy4SL988Vcje6W5jDR88GrWr5uYGH-_ZvN4arg==
X-Amzn-Trace-Id: Root=1-5b4506e0-db041a3492ee56e8f3d9457c;Sampled=0
X-Cache: Miss from cloudfront
x-amz-apigw-id: J04DHE92PHcF--Q=
x-amzn-RequestId: 3d82319d-8476-11e8-86d9-a1e4585e5c26

[
    {
        "labels": [
            "Human",
            "Clothing",
            "Dog",
            "Nest",
            "Person",
            "Footwear",
            "Bird Nest",
            "People",
            "Animal",
            "Husky"
        ],
        "name": "sample.mp4",
        "type": "video"
    }
]

You can also query for a specific object:

http https://n4dkhj8yqg.execute-api.us-east-1.amazonaws.com/api/sample.jpg

HTTP/1.1 200 OK
Connection: keep-alive
Content-Length: 126
Content-Type: application/json
Date: Tue, 10 Jul 2018 19:20:53 GMT
Via: 1.1 88eb066576c1b47cd896ab0019b9f25f.cloudfront.net (CloudFront)
X-Amz-Cf-Id: rwuOwzLKDM4KgcSBXFihWeNNsYSpZDYVpc8IXdT0xOu8qz8aA2Pj3w==
X-Amzn-Trace-Id: Root=1-5b450715-de71cf04ca2900b839ff1194;Sampled=0
X-Cache: Miss from cloudfront
x-amz-apigw-id: J04LaE6YPHcF3VA=
x-amzn-RequestId: 5d29d59a-8476-11e8-a347-ebb5d5f47789

[
    {
        "labels": [
            "Animal",
            "Canine",
            "Dog",
            "German Shepherd",
            "Mammal",
            "Pet",
            "Collie"
        ],
        "name": "sample.jpg",
        "type": "image"
    }
]