PatterPal - Personal Language Teacher
This project is a submission for the Microsoft AI Classroom Hackathon.
Immerse yourself in language learning: Talk freely on your chosen topics, get live feedback, and engage with a smart conversation partner in multiple languages. Learn your way with PatterPal!
How it works
PatterPal is a web application that allows you to practice your speaking skills in a foreign language. It's like having a language teacher just for yourself. PatterPal will listen to you and give you feedback on your pronunciation and fluency, while also talking with you about your chosen topic.
Technologies
- Azure Speech Services
- Speech to Text
- Pronounciation Assessment
- Synthesis
- Azure App Services
- ASP.NET Core Web App
- Azure Cosmos DB
- Google OAuth 2.0
- OpenAi
- gpt-3.5-turbo
Project Team
We are based in Austria and currently studying Software Engineering at the University of Applied Sciences Upper Austria.
Members:
Testing Instructions
Visit https://patter-pal.azurewebsites.net
Login:
- either use the Special Access Code we have provided or
- or connect your Gmail-Account with the application
- PatterPal only requests the scope necessary for reading the email address from the token. PatterPal does not request any private user data
Select a language of your choice and click the round button that resembles a microphone.
If this is your first visit (or depending on your browser settings), you will need to allow the website to use your microphone.
Click the 🎙️ button and start talking. A few seconds after speaking, your spoken text will gradually show up. The recording will stop after some moments of silence or if you manually click the 🎙️ button again.
A few seconds after the recording has halted, your language teacher will gradually respond. After the response has finished generating, the response will be read to you via Speech-to-Text. Below the language selelection box, you can the metrics regarding your spoken words. Misprounciations will also be highlighed in your spoken text. If you want to stop the Speech-to-Text output, you can click the ✋ button. Also keep in mind that you can change the langauge of the conversation whenever you want.
If you want to your conversation history, you can press the 📃 button on the top right of the screen. It toggles the your conversation history and allows you to start a new conversation You can also rename or delete individual conversations here.
After you had a few conversations with your language teacher, you can visit the stats page by clicking the 📊 button on the top right. Here you can see how your accuracy has changed over time or what words were least accurately pronounced. You can filter your metrics by language and also adjust the analysed time period and time resolution (playing around with these values is a good idea if you've been PatterPal for an extended period of time).
You can get back to the application by clicking the PatterPal icon on the top left. When you are done you can use the 🚪 button on the top right to log out.
Diagrams
Azure Tech Stack
The OpenAi interface is not yet used from Azure (instead platform.openai.com) as it is not currently available for a students' subscription
Data Layer Diagram
WebSocket Communication Workflow
Privacy Policy
Information Collection
- Email-Based Accounts: If you register for an account using your email, we collect and store your email address. This is used for account verification.
- Conversations and Chats: We record and store the details of your conversations and chats with PatterPal. This includes both your input and the responses from PatterPal. This is required so you can access past conversations anytime.
- Pronunciation Analysis: When speaking with PatterPal, we collect and analyze data on your pronunciation accuracy and the mistakes made during spoken text exercises. This information is used to provide personalized feedback and improve your learning experience.
How We Use Your Information
The information we collect is used to:
- Provide, operate, and maintain our services.
- Improve, personalize, and expand our services.
- Understand and analyze how you use our services.
- Develop new products, services, features, and functionality.
- Communicate with you for service-related purposes.
You can delete your Conversation, Chat and Pronunciation Analysis data on our Privacy Page.