Feature Request: Integrate Text-to-Speech AI for Better User Experience

[magyargergo]

Summary

First of all, thank you for the outstanding work on this mod. It's impressive, and I appreciate the effort and dedication that has gone into its development.

Let's jump straight to the point, I propose improving the Text-to-Speech (TTS) functionality of this mod by integrating AI libraries. This could improve voice quality and provide a more immersive experience for players.

Current Situation

The mod currently uses the NaturalVoiceSAPIAdapter for TTS. While it functions adequately, there are other libraries available that offer higher-quality voices and additional features.

Proposed Libraries

1. Microsoft Azure Speech SDK

• Description: Provides advanced TTS capabilities with high-quality, natural-sounding voices. Supports C++ integration and is compatible with Unity projects.
• SDK Information: Microsoft Azure Speech SDK
• Voice Gallery: Microsoft Voice Gallery
• Benefits:
  • Offers neural voices for realistic speech.
  • Supports a wide range of languages and dialects.
  • Provides options for voice customization.
• Considerations:
  • Requires an Azure account; usage may incur costs.
  • Licensing terms should be reviewed to ensure compatibility with the mod's distribution.

2. Amazon Polly

• Description: A cloud service that converts text into lifelike speech using deep learning technologies.
• SDK Information: Amazon Polly SDK for C++
• Available Voices: Amazon Polly Voices
• Benefits:
  • Provides a variety of natural-sounding voices.
  • Supports Speech Synthesis Markup Language (SSML) for advanced control.
  • Allows caching of speech locally to reduce latency.
• Considerations:
  • Requires an AWS account; usage may incur costs.
  • Licensing should be checked to ensure alignment with the mod's distribution model.

3. Google Cloud Text-to-Speech

• Description: Delivers natural-sounding speech using WaveNet and neural network technologies.

• Documentation: Google Cloud Text-to-Speech

• Available Voices: Google Cloud Voices

• Benefits:

  • High-fidelity voices with a wide selection.
  • Supports SSML for detailed speech customization.

• Considerations:

  • Requires a Google Cloud account; usage may incur costs.
  • Licensing and terms of service need to be reviewed for mod use.

    3. OpenAI Text-to-Speech

• Description: OpenAI offers advanced AI models capable of generating natural-sounding speech. Their Text-to-Speech service provides high-quality voice synthesis that can enhance the auditory experience within applications.

• Documentation: OpenAI Text-to-Speech Quickstart

• SDK Information: OpenAI C++ library

• Benefits:

  • Utilizes state-of-the-art AI for highly realistic voice output.
  • Supports multiple languages and accents.
  • Can convey different emotions and speaking styles to match game atmospheres.

• Considerations:

  • Requires an OpenAI account; usage may incur costs based on API usage.
  • Licensing terms need to be reviewed to ensure compatibility with the mod's distribution.

Benefits of Integration

• Improved Voice Quality: More natural and expressive voices enhance player immersion.
• Customization: Ability to match voices with the game's atmosphere and characters.
• Accessibility: Better support for players who prefer or require audio content.

Additional Considerations

• Integration Effort: Assess compatibility with the current C++ codebase and Unity engine.
• Performance Impact: Evaluate any potential performance implications.
• Licensing: Ensure that the chosen library's licensing terms are compatible with the mod's distribution.

Conclusion

Integrating one of these advanced TTS libraries could significantly enhance the player experience. I'm interested in contributing to this effort and would be happy to assist with research or development if this aligns with your project's goals.

Thank you for considering this feature request!

[gexgd0419]

NaturalVoiceSAPIAdapter supports the Azure Speech service.

You can use the online Azure voices if you have an Azure account, create a speech resource, then enter the key and region in the installer dialog.

You can create an Azure account for free, and if you choose the free tier while creating the speech resource, there will be no cost. The quota is 0.5 million characters per month, which may be enough for a single user, but not if shared with every user.

So I think that adding support for those speech SDKs would be great, but users should use their own accounts and keys, and pay for their own usage.