search

5-stones / react-native-readium

📚 📖 React Native ebook reader for iOS, Android, & Web via Readium
MIT License
89 stars 25 forks source link

Add ability to get currently displayed page text from ePub #27

Closed elraito closed 1 year ago

elraito commented 1 year ago

Please add a method to to get text from currently displayed page.

Whats would this enable:

It would be helpful to implement accessibility options i.e. text-to-speech implementations. Or ability to get a copy of text if the ePub might be educational for example. Ability to create implementation for citations. etc....

Personally i would love if it was in one of two formats: a) A method accessible through ref you can manually call. b) A callback that is fired on page change.

I personally favor option a but this is only an opinion.

mickael-menu commented 1 year ago

What are the use cases for the text of the currently displayed page instead of the whole text of the publication or current chapter?

elraito commented 1 year ago

For example grabbing displayed text for synthezizing voice for people with visual impairment through another library without synthezizing whole chapter. Implementing custom translation etc.

mickael-menu commented 1 year ago

TTS is supported in Readium, and can start from the current page. It's not available yet in the React native package though.

For the translation, you can get the text of a user selection and translate it. Translating the current page but not the full chapter would be complicated, and I don't really see the use case.

elraito commented 1 year ago

What languages does readium support for tts when it gets implemented in react native? Can it be customized? Will it parse language specific characters? All these questions can have an answer if the developer has access to implement custom tts. Not all books have chapters short enough for reasonable tts synthezizing in one batch and user selection is not always reliable or user friendly. Sadly o dont speak java or swift to open a pull request myself

jspizziri commented 1 year ago

@elraito IIRC readium's TTS support relies on the underlying OS's TTS functionality. Therefore anything available there would be available in Readium.

@mickael-menu correct me if I'm wrong.

jspizziri commented 1 year ago

@elraito I'm going to close this issue in favor of the preexisting TTS issue. https://github.com/5-stones/react-native-readium/issues/24

mickael-menu commented 1 year ago

The TTS in Readium is made of several independent components:

  1. A Content Iterator API which is used to extract the text (and more) from a publication. This is useful for TTS but also other use cases like search or to reformat the content in a custom view (accessibility reader).
  2. A Decorator API which is used to draw decorations on the pages. This is used to highlight the speaking sentences in the context of TTS.
  3. The TTS engine itself, that speaks utterances (sentences). You can provide your own TTS engine for full customization, but Readium provides a default implementation using the native TTS APIs on each platform, which support many languages. Users can add more languages from the system settings.
  4. And finally an orchestrator (TTS Navigator) which combines the other components to play the TTS across the whole publication and synchronize it with the EPUB navigator.

Not all books have chapters short enough for reasonable tts synthezizing in one batch and user selection is not always reliable or user friendly.

Sentences are never synthesized by the TTS engine in one batch. Utterances are computed lazily for a whole chapter, but synthesized/spoken one by one when needed.

I'm asking all these questions to see if something is missing in the core Readium API. In any case you should be able to inject your own JavaScript for unusual use cases.