Text extraction pipeline to cater for all songs.

[cmm25]

Summary: Currently, the project relies on YouTube’s captioning system for lyrics extraction. However, only a limited number of YouTube videos have captions enabled, restricting the number of songs from which we can extract lyrics. This issue proposes the development of a text extraction pipeline that can work independently of YouTube captions, enabling us to extract lyrics from a broader range of songs.

Problem:

• Limited song coverage: Only a small subset of YouTube songs have captions enabled.
• Many songs unsupported: A large number of the songs we are targeting lack captions, making it impossible to extract lyrics using the current method.
• Inconsistent results: Even when captions are available, they may be inaccurate or incomplete, further reducing the reliability of the current approach.

Proposed Solution:

1. Build a text extraction pipeline that:

   • Works with audio files directly, rather than relying on captions.
   • Uses techniques such as automatic speech recognition (ASR) to convert song audio into text (lyrics).
   • Optionally incorporates NLP post-processing to clean and format the extracted lyrics.

2. Consider alternative APIs or services that might offer more comprehensive lyric extraction.

Challenges:

• Audio quality and background noise may affect the accuracy of ASR.
• Regional accents, singing styles, and language differences will need to be addressed in the pipeline.
• Potential processing overhead due to real-time or large-scale audio transcription.

Next Steps:

• Research available ASR tools (e.g., Google Speech-to-Text, Whisper by OpenAI).
• Prototype the integration of an ASR system with the current pipeline.
• Test the system on a range of songs, both with and without captions.

Milestone:

• To be determined

Labels:

• Enhancement
• Feature Request

---

[chochinlu]

Sure. Just do it! 👍

Creating a correct translated text can be a challenge. The current approach in this project is simply providing a textbox for the user to manually correct the translation. 😆