mzdk100 / rigela

这是一个使用纯Rust编写的读屏(Screen Reader)项目,用于视力有障碍的人群操作电脑,软件会将屏幕上的各种信息转换成语音输出。
Apache License 2.0
155 stars 7 forks source link

Doesn't read slider values ​​and crashes in text editors. #1

Closed m1maker closed 2 months ago

m1maker commented 5 months ago

Hello. I want to report some bugs:

1: Sliders

m1maker commented 5 months ago

This is backtrace:

 Running `target\x86_64-pc-windows-msvc\debug\rigela-main.exe`                                                      

thread '' panicked at main\src\event_core\editor.rs:121:66:
attempt to multiply with overflow
stack backtrace:
0: std::panicking::begin_panic_handler
at > /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library\std\src\pani> cking.rs:645
1: core::panicking::panic_fmt
at > /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library\core\src\pan> icking.rs:72
2: core::panicking::panic
at > /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library\core\src\pan> icking.rs:144
3: > rigela_main::event_core::editor::impl$2::subscribe_msaa_events::async_fn$0:> :closure$0
at .\main\src\event_core\editor.rs:121
4: win_wrap::msaa::event::hook_proc
at .\win-wrap\src\msaa\event.rs:146
5: RegisterWindowMessageA
6: KiUserCallbackDispatcher
7: NtUserGetMessage
8: GetMessageW
9: > windows::Win32::UI::WindowsAndMessaging::GetMessageW n32::Foundation::HWND>
at C:\Users\Ecler.cargo\registry\src\index.crates.io-> 6f17d22bba15001f\windows-> 0.56.0\src\Windows\Win32\UI\WindowsAndMessaging\mod.rs:1352
10: win_wrap::message::get_message
at .\win-wrap\src\message.rs:61
11: > win_wrap::message::message_loop<win_wrap::msaa::event::impl$2::new::cl> osure$0::closure_env$0 ents::async_fn$0::closure_env$1> >
at .\win-wrap\src\message.rs:204
12: > win_wrap::msaa::event::impl$2::new::closure$0 us::subscribe_focus_events::async_fn$0::closure_env$1>
at .\win-wrap\src\msaa\event.rs:179
13: core::hint::black_box
at > /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce\library\core\src\hin> t.rs:286
note: Some details are omitted, run with RUST_BACKTRACE=full for a > verbose backtrace.
error: process didn't exit successfully: target\x86_64-pc-windows-> msvc\debug\rigela-main.exe (exit code: 0xc0000409, > STATUS_STACK_BUFFER_OVERRUN)

mzdk100 commented 5 months ago

Thank you for your question. The slider value cannot be read. I will mark the record first. For your second question - editor crash, which is also our ongoing work. We are trying to adapt to the editor information reading of notepad++ applications and will soon solve your problem.

m1maker commented 5 months ago

Yes. I would also like to say that SAPI5 voices that are installed as a custom speech synthesizer, such as Espeak, cannot be used. Also, in document windows such as WebBrowser, its contents are also not readable. Other than that, I want to say that this is really good work.

mzdk100 commented 5 months ago

In the Chinese environment, ESpeak's performance is very poor, so I did not include it in it. However, if you have such a need, I will consider adding it. As for the inability to use SAPI5 voice installed on the computer, I may need to investigate this issue.

mzdk100 commented 5 months ago

Web browser support is also ongoing, but this may take some time. I hope you can participate in the contribution (if you are interested in this project).

m1maker commented 5 months ago

Yes. I might be able to help, but I don't know Rust. I know C and C++. But from the code, it looks like you're trying to use Bass to output SAPI speech and remove a few milliseconds of chatter at the beginning to speed it up. However, support for other speech synthesizers is still needed. I can also suggest that you connect some kind of scripting language to it so that you can write extensions for RigelA, like NVDA. In any case, I will try to take part in this project, as I really want it to develop.

mzdk100 commented 2 months ago

@m1maker I just implemented the reading of slider control values. Do you want to test it?

m1maker commented 2 months ago

Hello. Thanks for your feedback! Everything works great. But it would also be nice to add the option to pronounce percentages, and not just change the pitch of the sound.

Now, I would also recommend that you speak Microsoft UI Automation notifications; some programs display the A11y notification in this way. I have a library that can speak to different speech engines or screen readers, and cross-platform. There is ENGINE_UIA, which does not work with Rigela because it does not support UIANotification. Still, I would advise you to add a COM interface for interaction and speech control, like IJawsAPI or NVDAControllerClient. Thank you! I hope you don't stop there.

mzdk100 commented 2 months ago

At present, this project is still very rudimentary, but I believe that all of these will be realized in the future.