search

aws-samples / amazon-bedrock-ai-karaoke

Amazon Bedrock AI Karaoke is an interactive demonstration of Amazon Bedrock. Users complete the prompt with the microphone and choose the best response. Powered by Amazon Bedrock, Amazon Transcribe and Amazon ECS Anywhere.
MIT No Attribution
22 stars 4 forks source link
amazon-bedrock amazon-web-services aws bedrock containers ecs-anywhere elastic-container-service genai transcribe

readme

Amazon Bedrock AI Karaoke

Release Notes GitHub star chart License: MIT

AI Karaoke Setup

Amazon Bedrock AI Karaoke is an interactive demo of prompting foundation models with a microphone and asking a human to pick their preferred response.

AI Karaoke Demo

Table of content

Motivation

Firstly, this is meant to be fun! Getting people to interact through a microphone and big push-buttons makes this demo more suitable for summits and demo spaces and resonates with non-technical audiences.

AI Karaoke Setup

Secondly, this introduces customers to several key ideas in generative AI system engineering including:

1) Text to image generation and visual Q/A 2) Importance of creating a data flywheel from human feedback 3) Mitigating challenges of hallucination and safety

User Instructions

1) 💭 First, think of a thing or a place in your minds eye you can describe in detail. For example, "A cute ginger cat wearing a suit and tie" 2) 🎤 Pickup the microphone and un-mute and describe that image! Speak clearly and close to the mic. 3) 🔴 🔵 Review the two images, and push the color button corresponding to the best image. 4) 🎑 🎤 Now ask a question about the selected best image. For example: "What's the color of the cat's tie?" 5) 🔴 🔵 Review the two responses, and push the color button corresponding to the best answer. 6) 🔁 Repeat!

Architecture

Architecture

The key AWS services used in this architecture are:

Prerequisites

  1. Get the code 

    git clone https://github.com/aws-samples/amazon-bedrock-ai-karaoke

  2. Ensure you have AWS credentials configured

  3. To interact with models from Amazon Bedrock, you need to request access to the base models in one of the regions where Amazon Bedrock is available. Make sure to read and accept models end-user license agreements or EULA.

sample

  1. Your AWS account and Region must be bootstrapped to create these resources before you can deploy. If you haven't already bootstrapped, issue the following command: 
    cdk bootstrap aws://ACCOUNT-NUMBER/REGION
  2. Make sure you have Docker installed and running on your machine as this will be used to build the container images used for the ECS tasks.

Local Dev

To get started with this project, it's recommended to first try it out locally on your laptop.

Firstly, we will install and run the front-end website locally:

  1. Move to the website src directory 
    cd ./lib/client/
  2. Install the npm dependencies 
    npm install
  3. Run the website 
    npm start
  4. Open a browser and go to http://localhost:3000

Now, in another terminal - we will started the server application

  1. Move to the server src directory: 
    cd ./lib/server/
  2. Install the python dependencies: (note the RPi.GPIO install will fail on your laptop but this is expected) 
    pip install -r requirements.txt
  3. Run the server: 
    python3 server.py

Deploy to Pi

Once you have tried the application locally, you may choose to deploy it to a Raspberry Pi device with orchestration in the cloud so you can leave the demo running at an event. There are several benefits to this approach of using Amazon ECS Anywhere including:

The following commands show how to package and deploy the application as an AWS CDK application.

First, complete the following pre-requisites:

  1. Install the Raspberry Pi OS (64 Bit) with GUI using Raspberry Pi Imager to your Raspberry Pi and setup SSH keys and internet.
  2. Power up and SSH to the pi

Now it's time to configure the infrastructure to deploy the software to the Raspberry Pi. Mkae sure you've completed the prerequisites above.

  1. Deploy the cdk app

    cdk deploy

  2. When the deployment is paused on the ECS Service creation, login to the AWS Console to get the registration command: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-anywhere-registration.html

  3. SSH onto the raspberry pi and paste the registration command

  4. The deployment should now complete and the service should be running on the pi

  5. Setup the kiosk service. You can find the kiosk.sh and kiosk.service files in the top level scripts directory.

    sudo apt install -y xdotool unclutter fonts-noto-color-emoji
sudo raspi-config > 1 System Options > S5 Boot / Auto Login >B4 Desktop Autologin — Desktop GUI, automatically logged in as ‘pi’ user

    Create the following files and enable the kiosk service

    vim /home/pi/kiosk.sh
sudo vim /lib/systemd/system/kiosk.service
sudo systemctl enable kiosk.service
sudo systemctl start kiosk.service

  6. Either build the full hardware box following the Build Instructions or use the minimal pi setup and use a USB mouse instead of the buttons to select the best response:

    Pi wiring

Troubleshooting

If UI scaling is wrong then first try to correct it from the TV settings. If that fails, try the following:

  1. Connect a USB keyboard to an available slot
  2. Press Ctr + to increase the zoom and Ctr - to decrease the zoom

Vocab Filter

You can add more bad words to mask in the prompt by appending to the lib/server/profanity.txt file. Please note the requirements and limits listed here when adding words to the list.

BOM

Total ~ £250

Tools used

Build instructions

Work in progress

Raspberry pi pin-out

Wire Pin Number Pin Name
Red button COM1 36 GPIO 16
Red button NO3 34 Ground
Blue button COM1 11 GPIO 17
Blue button NO3 9 Ground
Fan 1 5V (Yellow) 4 5V
Fan 1 GND (Black) 6 Ground
Fan 1 PWM (Blue) 8 GPIO 14
Fan 2 5V (Yellow) 2 5V
Fan 2 GND (Black) 39 Ground
Fan 2 PWM (Blue) 37 GPIO 26

Optionally, you can also connect the LED connectors on the buttons to the two spare 3V3/GND pins.

Wire Pin Number Pin Name
Red button LED 17 3V3
Red button GND 25 Ground
Blue button LED 1 3V3
Blue button GND 20 Ground
Pi GPIO

De-registering external instances

This is useful if you need to re-register the raspberry pi to a different ECS cluster

https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-anywhere-deregistration.html

sudo systemctl stop ecs amazon-ssm-agent
sudo apt remove -y amazon-ecs-init amazon-ssm-agent
sudo rm -rf /var/lib/ecs /etc/ecs /var/lib/amazon/ssm /var/log/ecs /var/log/amazon/ssm

Delete ECS checkpoint to avoid creds bug

https://github.com/aws/amazon-ecs-agent/blob/master/agent/app/agent.go#L783-L798

sudo vim /etc/rc.local

rm /var/lib/ecs/data/ecs_agent_data.json

Second WiFi on Pi

sudo vim /etc/wpa_supplicant/wpa_supplicant.conf 
network={
    ssid=""
    psk=""
}

Security

See CONTRIBUTING for more information.

License

This library is licensed under the MIT-0 License. See the LICENSE file.