SUNVA AI: Seamless conversation loop for the deaf

About SUNVA AI

Having conversations leveraging existing communication tools can be a challenge for deafs. While so many live transcription and text-to-speech tools are available in the market, they do not cater to the needs of the deaf and hard of hearing and aren't designed with an accessibility focus.

We are building this focusing on the problem deaf people are facing in India by building prototypes, testing it with the deaf community, and iterating based on the feedback.

STT

Often it could take a lot of time for the deaf person to read transcriptions and respond to them depending on various factors from person to person. SUNVA AI intelligently simplifies and provides various filters on the transcriptions to minimize the time taken to read the transcription and switch gaze between the screen and the person while communicating.

TTS

SUNVA AI provides a text-to-speech + LLM layer features that help the deaf person having speech irregularities to type less and respond more. For example, if the deaf person types "I coffee", SUNVA AI will convert it to "I would like to have a coffee" and speak it out.

Sunva Frontend

Sunva frontend is a single-screen UI where you can read the intelligently simplified transcriptions and write your response, which will be spoken back in the selected speech. Our idea is to build this app, take it to our deaf community volunteers, and based on their feedback, iterate the UI to make it even better.

SUNVA AI workplan

The first version of the POC will have the following features along with speaker diarization and text to speech.

Text (transcript) simplification using LLM
Text highlighting of important and critical keywords (transcript) using LLM
Intelligently apply simplification/highlighting to the transcriptions and provide easy summarizations.

Based on the user feedback from our focus groups, we will refine and add more features to the POC.

SUNVA AI Architecture

Please go through the architecture diagram to understand how SUNVA AI works. Please raise your questions in the issues section if you have any.

How to run POC

Clone the repository
Install the dependencies
```
poetry install
```
Install Redis and start the server in the background
```
redis-server &
```

Create a .env file inside ops dir and add the following variables

DATABASE_URL=YOURPOSTGRES_URL
GROQ_API_KEY=your_api_key
CLAUDE_API_KEY=your_api_key
OPENAI_API_KEY=your_api_key
BASE_MODEL=model_name (eg: Claude 3 Sonnet)
SPEECH_BASE_MODEL=model_name (eg: Whisper Large)
TTS_BASE_MODEL=model_name (eg: coqui-tacotron2)
JWT_SECRET_KEY=your_secret_key

Run the POC
```
uvicorn main:app --reload
```
Open the browser and go to
```
http://localhost:8000/
```

Using Poetry installer

Run this to enter virtual env
```
poetry shell
```
Install the dependencies
```
poetry install
```

How to contribute

There are few ways you can contribute to SUNVA AI.

By providing feedback on the POC
By raising issues in the issues section
By contributing to the codebase based on the issues
Join the SUNVA AI team by filling the p+ai volunteer form and select the SUNVA AI project.

PeoplePlusAI / sunva-ai

readme