search

Bhashini-2024 / Bhashini-C4GT

0 stars 0 forks source link

[DMP 2024]: #1

Open Bhashini-2024 opened 4 weeks ago

Bhashini-2024 commented 4 weeks ago

Ticket Contents

Description

Bhashini provides APIs for products to perform Automated speech recognition (ASR). These APIs use models hosted by Bhashini in the cloud and can be simply integrated in web or mobile applications. One such mobile application is the Nipun Lakshya (NL) app. The NL app is an assessment tool for students aged 3 to 8 used by teachers, examiners and mentors in Uttar Pradesh, India. One of the assessment types is Oral Fluency in which a passage is spoken by students and scores are highlighted as the student reads. For this NL app uses Bhashini APIs. However, we cannot assume that the teachers using the NL app would always have stable & high speed internet connectivity. This leads to failure of some assessments, wrong assessment results and also incorrect scores. To address this, offline functionality of the model is crucial, facilitated through Bhashini. We envision creating an offline model which enables offline access to ORF assessments and score highlighting, ensuring uninterrupted use in areas with poor connectivity. We aim to extend Bhashini's ASR model’s capabilities from its current online mode to offline usage.This involves distilling the existing model, storing essential language resources, word databases, and score highlighting algorithms locally on users' devices. By integrating offline capabilities into Bhashini, we ensure seamless functionality of apps like N & expand our adoption further..

Goals & Mid-Point Milestone

1 .Distillation of current ASR models in Hindi.

  1. Have a mobile ready offline model of size < 150 MB
  2. Have accuracy of ~80% in the offline model.

Setup/Installation

  1. You can register for API usage here https://bhashini.gov.in/ulca/user/register
  2. To Explore our models you can refer to this link https://bhashini.gov.in/ulca/model/explore-models
  3. Developer documentation for API can be taken from here: https://dibd-bhashini.gitbook.io/bhashini-apis/appendix

Expected Outcome

  1. A mobile ready distilled model for ASR in Hindi with 80% accuracy.
  2. A guidebook to undertake the same exercise for other languages.

Acceptance Criteria

No response

Implementation Details

  1. You can register for API usage here https://bhashini.gov.in/ulca/user/register
  2. To Explore our models you can refer to this link https://bhashini.gov.in/ulca/model/explore-models
  3. Developer documentation for API can be taken from here: https://dibd-bhashini.gitbook.io/bhashini-apis/appendix

Mockups/Wireframes

No response

Product Name

Niupn Laskhya _Bhashini

Organisation Name

Digital India Bhashini Division

Domain

⁠Learning & Development

Tech Skills Needed

Artificial Intelligence

Mentor(s)

Bhashini Team

Category

Data Science

prahalad2605 commented 3 weeks ago

Hello, I'm Prahalad, a third year student from Chennai. I'm quite experienced in Artificial Intelligence and NLP, along with app development. Through my coursework, I have a solid base in these concepts. I have also interned at IIT-Madras to hone my skills. I find this project very interesting and would love to contribute. Please let me know how to proceed.

Thank you.

Ris-code commented 3 weeks ago

Hello @Bhashini-2024,

My name is Rishav Aich, and I'm currently a third-year student at IIT Jodhpur, pursuing my BTech in Artificial Intelligence and Data Science. I've completed courses in machine learning, AI, deep learning, and software development. I feel confident in my abilities to work with NLP and AI, and I also have knowledge about caching systems.

I'm genuinely excited about the opportunity to work on the issue you've presented. Before diving into the project, I want to ensure that I've understood it correctly. Here's my understanding:

  1. The Nipun Lashaya app is an online reading assessment tool designed for students aged 3 to 8, with a focus on Hindi text.
  2. It currently faces challenges in low-income, internet-limited schools in Uttar Pradesh (UP) due to its reliance on internet connectivity.
  3. To address this issue, the app aims to integrate offline functionality using Bhashini, allowing access to assessments and error highlighting without an internet connection.
  4. The goals include offline storage of words and ensuring that the app works seamlessly without internet connectivity.
  5. To achieve this, it involves developing offline caching mechanisms within Bhashini, which would store essential language resources, word databases, and error highlighting algorithms locally on users' devices.

Could you please confirm if my understanding aligns with the project's objectives?

J0SAL commented 3 weeks ago

Hi @Bhashini-2024, I am interested in working on the project.

Ris-code commented 3 weeks ago

Hey @Bhashini-2024, I have completely read the documentation and understood how to use the APIs and how they function. But APIs are meant to use the models for the asr/translation/tts task. But for this task, we need the saved model (like model.pth if pytorch framework is used) and also the word embeddings. Where can I find these things? Please clarify since the source code is also not shared.

AbhimanyuSamagra commented 3 weeks ago

Do not ask process related questions about how to apply and who to contact in the above ticket. The only questions allowed are about technical aspects of the project itself. If you want help with the process, you can refer instructions listed on Unstop and any further queries can be taken up on our Discord channel titled DMP queries. Here's a Video Tutorial on how to submit a proposal for a project.

msa23003 commented 2 weeks ago

Hi @Bhashini-2024 I feel immense joy and happiness in informing that this project has caught my attention and I will like to work on this project. I assure you that I am sufficiently equipped and can come up with extraordinary results. Kindly give a chance to work on this project. I will send a proposal soon and will wait for your response. #ak2033290@gmail.com(Akanksha Kumari)

07naitik commented 2 weeks ago

Hello! I am a student at BITS Pilani, pursuing a Major in Computer Science, with a Minor in Data Science.

My interests align perfectly with this project, here's why -

  1. I have worked extensively on making small ML models using Tensorflow Lite and Pytorch Mobile, when I as working on my startup. It was an app that could identify clothing (type and features) using the smartphone camera, and generate an outfit around it using AI. The model was much smaller than traditional ML models for the same purpose, while maintaining remarkable accuracy.
  2. I won a hackathon organized by Moveworks on Generative AI, wherein I made a chatbot that can be dynamically trained on any website. It was implemented as a chrome extension. I got a good grasp on Natural Language Processing during the hackathon.
  3. I have worked on a project called Project Vulcan - an AI humanoid conversational robot. It uses Generative AI in the backend.

I have also worked on Automatic Speech Recognition tasks in the past. I am confident that I can successfully complete this project in the stipulated time. I have gone through the links provided over the last couple of days, and am working on my proposal!

Jatayu-u commented 1 week ago

Hello @Bhashini-2024,

I hope this message finds you well. My name is Shaurya Vats, and I am currently in my final year pursuing my undergraduate studies at the Indian Institute of Technology, Kharagpur (IIT Kharagpur). I am reaching out to express my interest in the project of making an offline version of the model.

With a background in language models, I have authored three research papers and developed various applications in the field. After reviewing the details of the project, I am confident that I possess the necessary skills and ideas to make significant contributions to its success. I have also submitted a proposal on Unstop regarding my approach to the project.

I am eager to have the opportunity to contribute to this open-source project at the earliest convenience. Your consideration of my application is greatly appreciated.

Thank you for dedicating your time to review my message.

Best regards, Shaurya Vats

krishnarathore12 commented 1 week ago

Hello @Bhashini-2024 I am Krishna Rathore undergraduate student at IIT Patna. I have a deep passion for AI and also recent advancements in NLP make me wonder about the future of AI.

I have been thinking about the project objectives and have some queries about technical implementation

  1. How will the offline ASR model handle variations in audio quality and background noise in real-world environments, especially considering it won't have access to real-time cloud resources for noise cancellation and audio enhancement?

  2. Is there a risk of the offline ASR model consuming excessive device resources (CPU, memory, battery) during operation, and if so, what measures are in place to mitigate this and ensure smooth performance on a wide range of mobile devices?

  3. Given the dynamic nature of language usage and the evolution of speech patterns over time, how will the offline ASR model adapt and update its language resources and word databases to maintain accuracy and relevance without relying on continuous online updates?

Thanks for dedicating your time Best regards Krishna Rathore