Text editing suddenly becomes broken until NVDA restart

ABuffEr commented 10 months ago

Steps to reproduce:

Unfortunately, I cannot provide any steps, it happens suddenly and without an apparent reason.

Actual behavior:

While I'm writing something in a browser, like a comment on Facebook, or a mail in Thunderbird, NVDA says "blank" where there was text until a second ago, sometimes still shows on Braille display, sometimes not, reading char by char with arrows, but not by line or with control+home or control+end.

No change if I switch on other windows, or disable and enable again focus mode, refresh the page etc.

Only an NVDA restart works.

It's just happened while I'm writing a comment on Facebook, and in that tab there is still the described situation (I not restarted yet). But the strange is that I can write here as usual.

Expected behavior:

NVDA should continue to work properly.

NVDA logs, crash dumps and other attachments:

It's just happened, I switched log level to debug and isolated these lines:

DEBUGWARNING - NVDAObjects.IAccessible.ia2TextMozilla.MozillaCompoundTextInfo._getText (21:53:51.281) - MainThread (7464): Tried to walk up past the root. Objects probably dead. [...] IO - inputCore.InputManager.executeGesture (21:53:51.522) - winInputHook (6408): Input: kb(laptop):rightArrow DEBUG - editableText.EditableText._hasCaretMoved (21:53:51.534) - MainThread (7464): Caret move detected using bookmarks. Elapsed 0.00503087 sec, retries 2 IO - speech.speech.speak (21:53:51.550) - MainThread (7464): Speaking [CharacterModeCommand(True), 'r', EndUtteranceCommand()] IO - speech.speech.speak (21:53:51.551) - MainThread (7464): Speaking [BreakCommand(time=1000), 'Roma', EndUtteranceCommand()]

System configuration

NVDA installed/portable/running from source:

Installed.

NVDA version:

NVDA alpha-30573,2f0a4116, but it's happening from months, from time to time.

Windows version:

Windows 10 22H2.

Name and version of other software in use when reproducing the issue:

Firefox 121, Thunderbird 122... and some previous versions of last months, as I said.

Other information about your system:

Other questions

Does the issue still occur after restarting your computer?

Yes.

Have you tried any other versions of NVDA? If so, please report their behaviors.

I used 2023.3rc2 for a while, and I'm quite sure it happened with that version too.

But I read also some basic users (so with stable versions, probably) that reported this issue.

If NVDA add-ons are disabled, is your problem still occurring?

Unfortunately, I cannot test with add-ons disabled, being the issue so irregular. But no add-ons appears in the log when the issue raises.

Does the issue still occur after you run the COM Registration Fixing Tool in NVDA's tools menu?

Yes, ran it various times in these months.

gauravahir28 commented 10 months ago

I can confirm this issue with NVDA 2023.3 and Slack! Even when there is text available in the text field, NVDA continues to read 'blank' until we restart NVDA. I couldn't identify specific steps, as it happens very randomly!

govsta1981 commented 10 months ago

I have seen this same thing happen in Thunderbird 115 and Beeper.

Adriani90 commented 10 months ago

@gauravahir28, @govsta1981 are you seeing this with addons disabled?

Is this also happening when no braille display is connected?

gauravahir28 commented 10 months ago

No, I use the IBM TTS Add-on. I couldn't try after disabling add-ons as it requires an NVDA restart, and after restarting NVDA, the problem does not persist. I also don't use a braille display. I'll attempt to use NVDA after disabling add-ons for a while and see if I can still replicate this issue.

geoffshang commented 10 months ago

I've also seen this in Slack, but it's pretty rare. It did happen to me yesterday though.

I was able to navigate by character and word, but reading by line, either using up and down arrows or NVDA+up arrow just said blank.

I just noticed that I was running 2023.2, so have just upgraded.

I am not using a Braille display and I have no add-ons installed.

Running Windows 11.

XLTechie commented 10 months ago

CC @jcsteh @LeonarddeR

jcsteh commented 10 months ago

I've seen this occasionally too. It's the same underlying issue as https://github.com/nvaccess/nvda/issues/15862#issuecomment-1833556035, but I don't know why it happens. In this case, the IAccessibleHyperlink interface breaks.

I originally thought this was Firefox specific, but if it's happening in the Slack app, that's not the case because the Slack app is based on Chromium.

jcsteh commented 10 months ago

When someone next sees this in the Slack app, can you please do the following:

Focus the text box which isn't working.
Press NVDA+control+z to open the NVDA Python console.
Paste this command and press enter: import IAccessibleHandler as iah; focus.firstChild.IAccessibleObject.QueryInterface(iah.IA2.IAccessibleHyperlink).startIndex
Provide the output here.

I want to verify that Slack is failing the same way as Firefox does. Thanks.

govsta1981 commented 10 months ago

I've not tried disabling add-ons. While I have a braille display, it's not always connected. I still see this issue whether it is or is not connected.

gauravahir28 commented 10 months ago

@jcsteh - I could replicate this issue, but getting "0" as output after performing steps given above! I also attempted the same steps on a functioning text field (MS Teams), and the output there is also "0"!

michaelDCurran commented 10 months ago

I just saw it in Slack. and when queryInterface to IAccessibleHyperlink I get:

Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "monkeyPatches\comtypesMonkeyPatches.pyc", line 32, in __call__
_ctypes.COMError: (-2147023179, 'The interface is unknown.', (None, None, None, 0, None))

ABuffEr commented 10 months ago

Hi, just for reference, same as reported by @michaelDCurran but in Firefox, tried some days ago. I don't use Slack or Teams.

amirsol81 commented 10 months ago

Quite incidentally, it happened to me yesterday in Thunderbird 122 Beta 4. NVDA kept reporting blank, but pressing Control+A followed by Control+C actually copied the text I had already typed. I was using one of the more recent NVDA alphas released last week.

jscholes commented 10 months ago

@jcsteh In the Slack desktop app:

>>> import IAccessibleHandler as iah; focus.firstChild.IAccessibleObject.QueryInterface(iah.IA2.IAccessibleHyperlink).startIndex
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "monkeyPatches\comtypesMonkeyPatches.pyc", line 32, in __call__
_ctypes.COMError: (-2147023179, 'The interface is unknown.', (None, None, None, 0, None))

jcsteh commented 10 months ago

It would appear then that we're dealing with the same problem. So, this definitely isn't Mozilla specific.

alexstine commented 10 months ago

@jcsteh Just reproduced it in Slack on the latest stable of NVDA.

>>> import IAccessibleHandler as iah; focus.firstChild.IAccessibleObject.QueryInterface(iah.IA2.IAccessibleHyperlink).startIndex
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "monkeyPatches\comtypesMonkeyPatches.pyc", line 32, in __call__
_ctypes.COMError: (-2147023179, 'The interface is unknown.', (None, None, None, 0, None))

Adriani90 commented 9 months ago

I guess it is not really related, but I was asking myself if a deeper look into #7719 could help further? But since this seems to occur only in focus mode I might be wrong. just thinking out loud :-).

Adriani90 commented 6 months ago

Can anyone still reproduce this in NVDA last alpha?

ABuffEr commented 6 months ago

Can anyone still reproduce this in NVDA last alpha? Yes, sadly. Just happened:

NVDA alpha-31971,c5954630

Firefox 126.0

seanbudd commented 5 months ago

Does anyone if other ATs experience this behaviour e.g. JAWS? It seems like interacting with IAccessible interfaces is breaking after some period of time, it would be useful to determine if this may be a bug in browsers or windows.

jcsteh commented 5 months ago

This is very unlikely to impact JAWS because JAWS runs all of these queries in-process. This problem only seems to impact cross-process calls; i.e. calls using the COM marshaler. Note that it doesn't impact NVDA's buffer rendering, only cross-process NVDA code (text editing, table navigation, etc.).

jcsteh commented 5 months ago

What's really curious to me is that I've never seen this break the IAccessible2 interface, only the IAccessibleHyperlink and IAccessibleTable2 interfaces. I haven't been able to figure out what's special about those two.

ABuffEr commented 4 months ago

Hi, in alpha it's happening more and more frequently, very annoying. I have the impression that it often happens after I opened a dialog to upload a file, or when I insert an emoji by Windows emoji composition. Then, in 2024.2 on another laptop, I noticed other, small, similar, random issues with Outlook and Teams (no output deleting text with backspace, no suggestion output writing a partial address in To: field... disappeared after an NVDA restart. Are we sure that these issues haven't a shared cause? Something about interface handlers instantiation, window switching, garbage handling...?

jcsteh commented 4 months ago

in alpha it's happening more and more frequently, very annoying.

Is this still using Firefox?

Are we sure that these issues haven't a shared cause?

Until we have sufficient evidence to suggest that they are, let's assume they aren't. This issue is already complicated and difficult to debug enough as it is. :)

ABuffEr commented 4 months ago

Is this still using Firefox? Yes, 128.0 at the moment. And Thunderbird beta of same version.

mltony commented 3 months ago

FWIW, a friend of mine who is a Jaws user complained of the same problem with jaws and Chrome about a month ago - this might suggest that the problem is perhaps not on NVDA side.

jcsteh commented 3 months ago

I'm wondering whether this is caused by releasing COM references on the wrong thread, in turn caused by a circular Python reference somewhere we haven't fixed. COM is very clear that you should not release references from the wrong apartment, so it's conceivable that doing so ends up breaking the COM marshaler in unforeseen and wonderful ways.

That would raise the question of why we've never seen this until the last year or so and why it's gotten worse particularly in the last few months. There are a few possibilities:

This problem always existed in NVDA, but something changed in Windows that makes the COM marshaler less tolerant of it.
We introduced a new circular reference somewhere. So, we previously released some references on the right (main) thread, but now we don't.
The circular reference always existed, but some timing changed somewhere (either in Python or NVDA) which causes it to be released at a different time on the wrong thread and this new timing breaks the marshaler.

Looking at a log where this issue just occurred, I do see a garbageHandler warning about 25 minutes before the failure. It's the only warning from garbageHandler in that log and that log is for a session about an hour long. I can't remember whether I tried to compose a message in Slack in those 25 minutes, but it's rather suspicious. I'm turning on garbageHandler debug logging now to try to get more info next time this happens.

michaelDCurran commented 3 months ago

If true, some ideas to avoid this:

Disable automatic garbage collection runs. and instead, in the main thread, each time core goes to sleep, do a possible gc.collect for the highest generation currently over its threshold according to gc_get_count and gc_get_threshold. Essentially moving all garbage collection into the main thread. This makes sure that at least all our STA COM objects will be released on the main thread. Currently other COM objects like UIA are MTA and therefore I don't think matters which thread we do it from. Or
In a comtypes __del__ monkeypatch, If currently not running in the main thread, stop calling Release there, and instead queue a function to our core event queue to Release on the main thread.

Unfortunately as far as I know, there is no nice way to query Windows to find out which apartment a given COM pointer is running in. On a per interface level, we can query the registry to find out what its threading model is, and at least be more careful if it is supposed to be STA. But I believe some of the other threading models act like STA if created within an STA, so probably does not help much.

jcsteh commented 3 months ago

Strictly speaking, I don't think you're supposed to release MTA specific objects on an STA, since it's still a different apartment. However, UIA doesn't seem to care. I can't recall whether UIA actually has a threading model of both, but regardless, we know this works because we use UIA objects all the time on the main thread.

I think either of these ideas could work and I've considered both of them myself. There is maybe some concern about giving the main thread even more work than it already has, but since the gc can run at any time, that can already happen anyway. If we were going to release COM references on the main thread, I'm not quite sure how we'd do that because the Release method is still on self, but self is supposed to die. We'd need to somehow store the raw pointer and method without storing the Python reference, which sounds... fiddly.

jcsteh commented 2 months ago

I just saw this again and there was no garbageHandler logging before it happened, so I don't think this is due to garbage collection on the wrong thread. :( There was some garbageHandler logging a minute or two after and that did include IAccessibleText, IAccessibleAction and ia2Web.EditorChunk objects, but that happened on the main thread, not some background thread, so the COM objects were released from the right thread. Back to square 0.

nvaccess / nvda