search

tsengwoody / Access8Math

Allows access math content written by MathML ; Assist writing math content by LaTeX
GNU General Public License v2.0
32 stars 15 forks source link

Idea re turning images into MathML and reading with Access8Math #89

Open marcuslowx opened 9 months ago

marcuslowx commented 9 months ago

Hi. Access8Math works great in cases where equations are already in Math ML. I am wondering though whether a pipeline is possible where images of equations (which is how we often encounter math online and in textbooks) can be turned into MathML using something like Meta's Nougat? Maybe this is something to explore in another project, but ultimately a great solution would be a single plug-in that can both turn images of math into MathML and then read it. Anyway, keen to know what you think. Also keen to contribute, though i haven't done much of this type of programming. Thanks for a great add-in.

tsengwoody commented 9 months ago

Thank you for reaching out and sharing your thoughts on improving Access8Math with the potential integration of image-to-MathML conversion capabilities. Your idea of turning images of equations into MathML is indeed a valuable and useful concept, especially for enhancing accessibility in digital learning materials. Regarding the feasibility of using Meta's Nougat for this purpose, it is a viable option. However, it would likely be necessary to build an executable file (exe) because setup/install such a complex python package like Nougat directly into the NVDA Python environment may not be straightforward Embarking on this feature would indeed be a substantial undertaking. It might be more feasible to implement this feature as part of a separate addon, where dedicated efforts can focus on overcoming the technical challenges and ensuring a seamless user experience.

cary-rowen commented 9 months ago

It might be a good idea to create a separate add-on using Mathpix engine, I have this idea.