search

OpenNyAI / Jugalbandi-Manager

Jugalbandi (JB) Manager is a full AI-powered conversational chatbot platform. It's platform agnostic and can serve multiple channels such as WhatsApp or custom web interfaces. It can handle conversations in both text and voice across any language. It comes with Bhashini Speech models out of the box and can failover to Azure.
https://docs.jugalbandi.opennyai.org/
Apache License 2.0
22 stars 29 forks source link

Issue: Support for cloud speech & translation connectors #173

Open Lekhanrao opened 2 months ago

Lekhanrao commented 2 months ago

Ticket Contents

Develop connectors for AWS and GCP speech & translation connectors.

Goals

To provide support for the speech and translations of data for cloud providers such as GCP and AWS.

Expected Outcome

The stack should have capabilities to do speech and translation using GCP and AWS models by providing appropriate speech/translation keys in the environment.

Acceptance Criteria

  1. The users should be able to do speech_to_text, text_to_speech and translation using GCP and AWS credentials
  2. These two new classes for speech_processor and translator should be put into language/src/extension.py file as composite values along with the existing Dhruva and Azure instances.

Implementation Details

  1. Write connectors for speech & translation services in GCP and AWS by following the Azure speech & translation connector files.
  2. The translation code is present in language/src/translator.py. Create two new classes for GCP and AWS translator by extending the Translator base class.
  3. The speech processor code is present in language/src/speech_processor.py. Create two new classes for GCP and AWS speech processors by extending the SpeechProcessor base class and implement the base methods.

Mockups/Wireframes

### Product Name Jugalbandi ### Organisation Name OpenNyAI ### Domain Legal ### Tech Skills Needed Python ### Mentor(s) @Lekhanrao @KaranrajM ### Complexity Low ### Category Backend
c4gt-community-support[bot] commented 2 months ago

Hi! Important Details - These following details are helpful for contributors to effectively identify and contribute to tickets.

Please update the ticket

Lekhanrao commented 2 months ago

@c4gt-community-support, What are the sub-category types and where do we update that in the Issue template?

Ris-code commented 2 months ago

@Lekhanrao Could you please assign me this issue. I would like to solve this.

Lekhanrao commented 2 months ago

Assigned ticket to @Ris-code

Ris-code commented 2 months ago

Assigned ticket to @Ris-code

Thank you for assigning me the task. I have already wrote the code but how to test it because I am not having the credentials. Will it work with my own keys ? or I need to make a PR for you to check.

KaranrajM commented 2 months ago

Hi @Ris-code you can test with your own keys to test the individual translator and language services. Once you verify that it is working, you can raise a PR and we will test code with the whole application and comment if there are any issues.

Lekhanrao commented 2 months ago

@Ris-code Please let us know if you have any update on this.

Ris-code commented 2 months ago

@Ris-code Please let us know if you have any update on this.

could you please guide how to connect to Whatsapp I am having issues with it.

KaranrajM commented 2 months ago

@Ris-code Please let us know if you have any update on this.

could you please guide how to connect to Whatsapp I am having issues with it.

Hi @Ris-code you need to have pinnacle (one of the business service providers of whatsapp) credentials to connect to whatsapp. But for testing speech and translator services, we can directly test these classes with GCP & AWS keys right? There wont be a need to run the whole chatbot in whatsapp and then test them. We can test them through whatsapp as well but we need pinnacle keys.

Ris-code commented 2 months ago

@Ris-code Please let us know if you have any update on this.

could you please guide how to connect to Whatsapp I am having issues with it.

Hi @Ris-code you need to have pinnacle (one of the business service providers of whatsapp) credentials to connect to whatsapp. But for testing speech and translator services, we can directly test these classes with GCP & AWS keys right? There wont be a need to run the whole chatbot in whatsapp and then test them. We can test them through whatsapp as well but we need pinnacle keys.

Yes I can directly test the APIs. Thank you. Working on it.

yashpal2104 commented 2 months ago

Can you assign this issue to me

Ris-code commented 2 months ago

@Ris-code Please let us know if you have any update on this.

could you please guide how to connect to Whatsapp I am having issues with it.

Hi @Ris-code you need to have pinnacle (one of the business service providers of whatsapp) credentials to connect to whatsapp. But for testing speech and translator services, we can directly test these classes with GCP & AWS keys right? There wont be a need to run the whole chatbot in whatsapp and then test them. We can test them through whatsapp as well but we need pinnacle keys.

Yes I can directly test the APIs. Thank you. Working on it.

I have made a PR changing the translator.py, speech_processor.py, extension.py, .env-dev.template and .env.template files. Please review it . I have already tested all the apis for gcp and aws, it is working fine.

Lekhanrao commented 2 months ago

@yashpal2104, This issue is already being worked upon by Ris-code. Seems like it is almost done and waiting our Team lead's review. So I will get back to you after that review and if we still need help. Thankyou! Btw, there are other issues that you could pick up. I will encourage you to please review our open issues and pick the ones that's of interest to you.

Lekhanrao commented 2 months ago

@KaranrajM, could you please check the updates made by Ris-code and the PR request as well.

yashpal2104 commented 2 months ago

sure no problem I will get to other issues

Lekhanrao commented 1 month ago

@KaranrajM to review the PR and close by Friday 4th Oct.

Lekhanrao commented 2 weeks ago

@KaranrajM is this done?