A telegram bot that can be your audio guide at Art Museums. Send a photo of an artwork, and it will generate an audio description to which you can listen to.
Try it out: https://t.me/harbour_art_guide_bot
We developed this project during the class in Harbour.Space.
Read a HackMD article about the project
This module is in charge of segmenting the image and cropping the relevant part of the image (for e.g. the painting in the image).
You can go here for more details about the image crop module or read this blog post.
This module is in charge of finding the most similar image to the input image in the database and return the relevant information about the image.
You can go here for more details about the reverse image search module or read this blog post.
This module receives the relevant information of the found image and generates a description of the image, in a specific format. You can go here for more details about the descriptor module or read this blog post.
This module receives the description and generates an audio file of the description. You can go here for more details about the text to speech module.
This project uses GitHub Actions for CI/CD. The workflow is defined in .github/workflows/main.yml
. The workflow is triggered on every push or merge request to the main
branch. The workflow consists mainly in a build
job that builds the docker image and pushes it to the GitHub Container Registry. For more information please read CI/CD README
There are 3 ways to run the project:
For the first 2 options you need to set up the environment variables
.env
file by using .env.example
as template).You need 2 concurrent terminals to run the project.
gunicorn -w 4 -k uvicorn.workers.UvicornWorker main:app
python3 main.py
docker build -t art-guide-tg-bot .
docker run art-guide-tg-bot
docker pull arielxx/art-guide-tg-bot:latest
docker run arielxx/art-guide-tg-bot:latest
from descriptor_module import descriptor
from image_crop_module import croper
...
image = Image.open(path)
# input: Pilow image, output: np.array
image_croped = croper.crop_image(image)
...
description = descriptor.describe(features_dict)
print(description)