Closed DrSooom closed 4 years ago
Hmmmm,
Is it something to do with your NVDA configuration profile, or really some weird regression?
I cannot reproduce this on the latest alpha, powered by py3
That were/are always fresh portable Betas of NVDA. NVDA 2018.1 is installed on my system. Here is my config file. I guess it's a regression. Furthermore you can't compare these Betas with the Py3-Alphas because of PR #9544 and #9545.
Ah, I see
But i am still wondering
Do you have reasons to think that this is a bug in NVDA and not in liblouis?
@leonardder: You ask me things? That could be also caused by the Liblouis update from 3.8.0 to 3.10.0 over 3.9.0. This issue didn't occur in NVDA 2019.1.1 with Liblouis 3.8.0.
Therefore: CC: @bertfrees, @egli, @dkager and @nishimotz (if you are using Liblouis).
@DrSooom Could you please test this with the last Python 2 Alpha of NVDA available here. I am asking because in issue #9486 the errors are similar, and it is caused by liblouis builds being optimized. If my hypothesis would be correct perhaps disabling optimization for liblouis builds for the time being would be reasonable. @leonardder What do you think?
NVDA Alpha-18064 logfile on loading "zh-tw.ctb" after NVDA restart in Debug Mode:
Initializing global plugin handler
DEBUGWARNING - braille.BrailleHandler.setDisplayByName (20:39:28.825):
Error in initial display after display load
Traceback (most recent call last):
File "braille.pyc", line 1686, in setDisplayByName
File "braille.pyc", line 1976, in initialDisplay
File "braille.pyc", line 1840, in handleGainFocus
File "braille.pyc", line 1845, in _doNewObject
File "braille.pyc", line 1516, in getFocusRegions
File "braille.pyc", line 615, in update
File "braille.pyc", line 428, in update
File "louisHelper.pyc", line 65, in translate
File "louis\__init__.pyc", line 183, in translate
WindowsError: exception: access violation reading 0x08DF7608
IO - braille.BrailleBuffer.update (20:39:29.046):
Braille regions text: [u'NVDA ist bereit']
IO - braille.BrailleHandler.update (20:39:29.046):
Braille window dots: 13457 12367 1457 17 - 24 234 2345 - 12 15 1235 15 24 2345
DEBUG - core.main (20:39:29.048):
Initializing core pump
And here is the complete logfile (start at line 149) after NVDA Alpha-18064 (Portable on a SSD) totally crashed after a few more restarts in Debug Mode. Only the Windows Explorer was opened during these tests. The HUC Braille Tables weren't tested.
Hi, I've just tested and I wasn't able reproduce this bug. Tested with 2019rc1 and alpha-18222,be63d19d (portable versions). I Restarted NVDA about 10 times.
@Andre9642: Regarding NVDA 2019.2 RC1 I can confirm this. I also didn't get any errors. I wonder why. Therefore: Closing for now.
This issue is back in NVDA 2019.2 RC2 (portable). Here is the logfile with "zh-tw.ctb" and freezing on startup (which wasn't logged, because I was able to kill the process "nvda.exe" via the Task Manager). The HUC Braille Tables are also affected.
A nice new log entry with NVDA 2019.2 Beta-18345 on startup. Complete logfile
RuntimeError: Can't translate: tables [u'louis\tables\zh-tw.ctb', 'braille-patterns.cti'], inbuf N V D A i s t b e r e i t , typeform None, cursorPos c_long(0), mode 4
Note: There is a strange character between the letters in the logfile, It's not a space (U+0020) – maybe CR, LF or CRLF together. CC: @leonardder, @zstanecic, @bertfrees and @egli
@DrSooom are you able to reproduce this in NVDA 2019.2 final release?
Sadly absolutely no differences with NVDA 2019.2 Stable (Portable). Tested multiple times on a SSD and on a USB Flash Drive. Issue #9973 as well as this one here are still valid.
Question: Why did this all work as expected in NVDA 2019.2 RC1? What changes have been made since 2019.2 RC1 to 2019.2 Stable regarding liblouis? The file "zh-tw.ctb" wasn't changed (I checked the SHA-256 sums).
And here follows the significant part of the logfile (HUC8 Braille Tables were included into "de-de-comp8.ctb", but it's the same with "zh-tw.ctb" alone):
Initializing global plugin handler
IO - braille.BrailleBuffer.update (12:51:58.680):
Braille regions text: [u'Desktop fst']
ERROR - core.main (12:51:59.104):
Traceback (most recent call last):
File "core.pyo", line 461, in main
File "braille.pyo", line 1799, in message
File "braille.pyo", line 428, in update
File "louisHelper.pyo", line 65, in translate
File "louis\__init__.pyo", line 183, in translate
WindowsError: exception: access violation writing 0x00000000
DEBUG - core.main (12:51:59.104):
Initializing core pump
Question: Why did this all work as expected in NVDA 2019.2 RC1? What changes have been made since 2019.2 RC1 to 2019.2 Stable regarding liblouis
Nothing has changed in the braille area as far as I'm aware, except for the fact that all dll's (including liblouis) have been recompiled.
Here is the logfile based on NVDA 2019.2 Portable running from a USB 3.0 Flash Drive (connected via USB 2.0). This time several Unicode characters in the Basic Latin block (U+0000 to U+007F) weren't loaded correctly. See line 181 to 194.
Same result on startup with NVDA 2019.2.1 Beta-18416 (Portable, release date: 2019-08-15) as I described here. Personally I would give this issue here a P2, as it is critical for Chinese users – and maybe others too.
This definitely looks like a liblouis issue to me, therefore I'd like to suggest opening an issue against liblouis as well.
Just to make sure, does this problem also occur with NVDA 2019.1?
@feerrenrut I think we should consider reverting the update to liblouis 3.10, i.e. go back to liblouis 3.9 and see what that does here.
Note that this issue cannot be reproduced in latest alpha snaps powered by py3 and liblouis 3.11.
I am currently using it and tried to reproduce the issue, but no luck here.
Alpha is at liblouis 3.10 still.
Op 19-8-2019 om 10:00 schreef zstanecic:
Note that this issue cannot be reproduced in latest alpha snaps powered by py3 and liblouis 3.11.
I am currently using it and tried to reproduce the issue, but no luck here.
From: Leonard de Ruijter notifications@github.com Sent: Monday, August 19, 2019 9:00 AM To: nvaccess/nvda nvda@noreply.github.com Cc: zstanecic zvonimirek222@yandex.com; Mention mention@noreply.github.com Subject: Re: [nvaccess/nvda] Errors on loading large (Chinese) braille tables with NVDA 2019.2 Beta-18176, 2019.2 Beta-18345 and 2019.2 RC2 (#9982)
@feerrenrut https://github.com/feerrenrut I think we should consider reverting the update to liblouis 3.10, i.e. go back to liblouis 3.9 and see what that does here.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nvaccess/nvda/issues/9982?email_source=notifications&email_token=ACVCDEZDKFYEI4W2MQSUPWTQFJAF5A5CNFSM4IHHDP22YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4R4WGY#issuecomment-522439451 , or mute the thread https://github.com/notifications/unsubscribe-auth/ACVCDE2D3TDNIPBJ3YERB53QFJAF5ANCNFSM4IHHDP2Q . https://github.com/notifications/beacon/ACVCDE56D353KMPOFTAFQ7LQFJAF5A5CNFSM4IHHDP22YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4R4WGY.gif
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nvaccess/nvda/issues/9982?email_source=notifications&email_token=AAXIOAFPTYTWZLZ2RL2JWMLQFJHIPA5CNFSM4IHHDP22YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4SBF4Y#issuecomment-522457843, or mute the thread https://github.com/notifications/unsubscribe-auth/AAXIOABAYQ6VV7COPYZ7MGLQFJHIPANCNFSM4IHHDP2Q.
Really?
There were no backwards compat changes in the log of liblouis 3.11, so i am really confused now
There isnj't a liblouis 3.11, it is due for release in september.
Just to make sure, does this problem also occur with NVDA 2019.1?
As mentioned in the issue description of issue #9973, the HUC Braille Tables already works as expected with 2019.1.1 Stable (Portable). And after restarting NVDA with "zh-tw.ctb" multiple times, I can confirm that this issue definitively don't occur in 2019.1.1 Stable (Portable on a SSD) as well. Please note that "zh-tw.ctb" was extended between Liblouis 3.8.0 and 3.10.0. Therefore I also tested NVDA 2019.1.1 with the 3.10.0 version of "zh-tw.ctb" and "spaces.uti" (is required, new in Liblouis 3.10.0). The result was the same – no errors. So this issue isn't caused by the braille tables files themselves.
I'm still wondering why that issue didn't occur in NVDA 2019.2 RC1. That's really strange because it's software and not a lottery. Are there any building log or configurations building files for 2019.2 RC1 and 2019.2 RC2, which we could compare?
And please read @lukaszgo1's comment again.
That's really strange because it's software and not a lottery.
This is a common outcome with uninitialized variables, buffer over-runs, and several other similar classes of errors. It may not be a new bug, it may take a specific combination of things that affect the memory layout of the the application in order to result in this outcome.
It sounds like we will need to debug liblouis to track down the cause.
I created a try build based on NVDA 2019.2 with Liiblouis 3.9 instead of 3.10:
Is this issue really enough to revert entire upgrade to Liblouis 3.10, and release 2019.2.1? It might make more sense to create try build using release flag from the current master and test with it.
NVDA (2019.2.1) Try-18471 raises no errors for "zh-tw.ctb" (3.9.0 and 3.10.0 were tested), for the HUC8 Braille Tables included into "de-de-comp8.ctb" and for the HUC8 Braille Tables using alone. But it still doesn't like the HUC6 Braille Tables (stand-alone and included into "de-g1.ctb"). After further investigations, this behavior also exists in NVDA 2019.1.1 (Portable). Issue #9973 only describes the situation with the HUC8 Braille Tables. So here are four log files regarding the HUC6 Braille Tables. NVDA 2018.1 with Liblouis 3.3.0 (or 3.4.0) has no problems with the HUC6 Braille Tables (included into others and stand-alone). I think I have to take a deeper look at the HUC6 Braille Tables in the upcoming weeks/months – after that extreme heat has gone.
If you're now going to release NVDA 2019.2.1, please also include PR #10104 and PR #10153 as these two bugs are serious for Win7 users. It would be really grateful having them solved in the very last Python 2 version of NVDA.
To clarify, I'm not saying we are going to release a 2019.2.1, I only wanted to investigate whether a downgrade to liblouis 3.9 would solve the issue. It has the potential that it will solve it.
Liblouis 3.11 is going to be released soon, so in my view the more important question should be is the bug still occurring with Python 3 builds of NVDA using soon to be released Liblouis 3.11 compiled as a release would be?
The issue does not occur on Python 3 builds of NVDA, so I doubt that Liblouis 3.11 would introduce a bug with this. But there are enough reasons to upgrade to liblouis 3.11 asap after its release.
Hi, as for reverting to 3.9 in 2019.2.1, we need to be careful – it isn’t Chinese braille users who are concerned about it – Korean braille users are also keeping an eye on this, afraid that it can introduce regressions (again). Thanks.
From: Leonard de Ruijter notifications@github.com Sent: Wednesday, August 28, 2019 11:28 PM To: nvaccess/nvda nvda@noreply.github.com Cc: Subscribed subscribed@noreply.github.com Subject: Re: [nvaccess/nvda] Errors on loading large (Chinese) braille tables with NVDA 2019.2 Beta-18176, 2019.2 Beta-18345 and 2019.2 RC2 (#9982)
The issue does not occur on Python 3 builds of NVDA, so I doubt that Liblouis 3.11 would introduce a bug with this. But there are enough reasons to upgrade to liblouis 3.11 asap after its release.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/nvaccess/nvda/issues/9982?email_source=notifications&email_token=AB4AXEFWP62G5VOLBYKLRVDQG5T5XA5CNFSM4IHHDP22YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5NMUUQ#issuecomment-526043730 , or mute the thread https://github.com/notifications/unsubscribe-auth/AB4AXEAMC4H726RBDXSSM33QG5T5XANCNFSM4IHHDP2Q .
So the occurred errors are caused by the file size – and not by the 65,536(+) definitions. Liblouis 3.8.0 (NVDA 2019.1.1) handles larger files just a little bit better than Liblouis 3.10.0 (NVDA 2019.2). But in the end both aren't bug-free. Maybe we should wait until Liblouis 3.11.0, which could be released next week, and try to include it into a possible NVDA 2019.2.1. CC: @egli and @bertfrees
So, that's all I could do on my side. Hope it helps a little bit.
PS: I'm still so glad that these errors aren't caused by the HUC Braille Tables. 😉
Splitting "zh-tw.ctb" into two parts at line 25,000 and including the second part into the first file via the include command doesn't solve this problem in NVDA 2019.2. Well, excluding the second part of that braille tables solved it, but that's not the solution we are looking for.
It is quite interesting that this issue absolutely doesn't occur with NVDA 2019.2 RC1 when using "zh-tw.ctb" or the HUC8/HUC6 Braille Tables (including into other braille tables and stand-alone). As long as the IME isn't used, downgrading NVDA from 2019.2 to 2019.2 RC1 or just updating NVDA to 2019.2 RC1 is currently the only workaround for solving this issue.
Possibly linked to https://github.com/liblouis/liblouis/issues/848
Update.
It is being considered to go back to liblouis 3.10 in NVDA 2019.2.1. There are some major disadvantages though, see #10232
Note that there are several conflicting interests here, particularly old issues being reintroduced when downgrading.
@leonardder: You meant Liblouis 3.9.0, or? You wrote "Liblouis 3.10.0", which is the current Liblouis version in NVDA 2019.2.
GitHub re-ordered the "Close issue" and the "Comment" button. Bad GitHub. Very, very bad GitHub. 😉
Just for reference: It looks like The beta snapshot mentioned first in this issue was produced before 2019.2rc1, not after. Therefore the fact that rc1 does not have the bug but rc2 does, is simply a fluke due to memory layout or something. Also, it is worth pointing out that that beta snapshot would not have been optimized, yet it still contains the bugs. Only tagged releases (E.g. beta1, or rc2 etc) are optimized.
I forced a new release try build of 2019.2rc1. I wouldn't be surprized if running this produces the bug? https://ci.appveyor.com/api/buildjobs/fh494hpy0ycd7vm3/artifacts/output%2Fnvda_try-release-2019.2rc1-18694%2Cbc11064f.exe If so, this proves that the problem is completely random, at least to the build process. Either way, tomorrow I will get a debugger on this, and see if I can understand the problem in liblouis, and perhaps there is a chance we can come up with a patch for 3.10.
See https://github.com/liblouis/liblouis/pull/856
cc @Andre9642
See this comment. Same error with 2019.2.1 Try-RC1-18694 and "zh-tw.ctb" at the very first restart in Debug Mode after saving the configuration. Therefore I'm not going to test the HUC Braille Tables.
DEBUGWARNING - braille.BrailleHandler.setDisplayByName (12:41:34.282):
Error in initial display after display load
Traceback (most recent call last):
File "braille.pyo", line 1686, in setDisplayByName
File "braille.pyo", line 1976, in initialDisplay
File "braille.pyo", line 1840, in handleGainFocus
File "braille.pyo", line 1845, in _doNewObject
File "braille.pyo", line 1516, in getFocusRegions
File "braille.pyo", line 615, in update
File "braille.pyo", line 428, in update
File "louisHelper.pyo", line 65, in translate
File "louis\__init__.pyo", line 183, in translate
WindowsError: exception: access violation reading 0x969775A4
ERROR - core.main (12:41:34.496):
Traceback (most recent call last):
File "core.pyo", line 461, in main
File "braille.pyo", line 1799, in message
File "braille.pyo", line 428, in update
File "louisHelper.pyo", line 65, in translate
File "louis\__init__.pyo", line 184, in translate
RuntimeError: Can't translate: tables [u'louis\\tables\\zh-tw.ctb', 'braille-patterns.cti'], inbuf N
This log was copied directly from the Log Viewer. Therefore everything after the "N" is missing.
@leonardder https://github.com/leonardder: You meant Liblouis 3.9.0, or? You wrote "Liblouis 3.10.0", which is the current Liblouis version in NVDA 2019.2.
Yes, sorry.
Can anyone other than @DrSooom , actually reproduce this? I am trying to understand the full impact of this within the NVDA community. I am unable to reproduce this myself, and reading all the comments so far here, it looks as though no one else is saying they can either? Obviously with any crash or memory corruption we would like to address it. However, so far we are not coming up with any solutions, and I would like to release 2019.2.1beta1 as soon as possible, already with fixes for several other crashes, including in Gmail, Windows 7 start menu, and Windows 7 explorer metadata fields. At this stage I do not think that we will be able to come up with a safe solution to this bug for 2019.2.1. Reverting to Liblouis 3.9 will reintroduce other bugs, plus it seems as though 3.9 possibly had some of this issue already.
Hi
I cannot reproduce this with chinese tables.
From: Michael Curran notifications@github.com Sent: Tuesday, September 24, 2019 8:45 AM To: nvaccess/nvda nvda@noreply.github.com Cc: zstanecic zvonimirek222@yandex.com; Mention mention@noreply.github.com Subject: Re: [nvaccess/nvda] Errors on loading large (Chinese) braille tables with NVDA 2019.2 Beta-18176, 2019.2 Beta-18345 and 2019.2 RC2 (#9982)
Can anyone other than @DrSooom https://github.com/DrSooom , actually reproduce this? I am trying to understand the full impact of this within the NVDA community. I am unable to reproduce this myself, and reading all the comments so far here, it looks as though no one else is saying they can either? Obviously with any crash or memory corruption we would like to address it. However, so far we are not coming up with any solutions, and I would like to release 2019.2.1beta1 as soon as possible, already with fixes for several other crashes, including in Gmail, Windows 7 start menu, and Windows 7 explorer metadata fields. At this stage I do not think that we will be able to come up with a safe solution to this bug for 2019.2.1. Reverting to Liblouis 3.9 will reintroduce other bugs, plus it seems as though 3.9 possibly had some of this issue already.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nvaccess/nvda/issues/9982?email_source=notifications&email_token=ACVCDE2SZRLSAS4CMFALAELQLGZOFA5CNFSM4IHHDP22YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7NIRSI#issuecomment-534415561 , or mute the thread https://github.com/notifications/unsubscribe-auth/ACVCDE22GXBWE2VTSOE44PTQLGZOFANCNFSM4IHHDP2Q . https://github.com/notifications/beacon/ACVCDE7YVIPGYJ4IWIX2WRLQLGZOFA5CNFSM4IHHDP22YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7NIRSI.gif
@michaelDCurran and @zstanecic: Please test on Win7x64 with Chinese Taiwan/Mandarin (zh-tw.ctb) because it's the largest braille table in Liblouis 3.10.0. I was able to reproduce this error also on my NUC (win7x64, NVDA 2015.3 is still installed) with NVDA 2019.2.1 Try-RC1-18694 Portable. The HUC Braille Tables weren't tested yet with this NVDA version, as this error already occurs with the largest Liblouis braille table.
WindowsError: exception: access violation reading 0xEEC675AC
This discussion is getting really Long. We Need clear Impact results on users, especially chinese users. @larry801, @dingpengyu could you please Elaborate exactly your test results and how significant the user Impact is? Thanks!
This discussion is getting really Long. We Need clear Impact results on users, especially chinese users. @larry801, @dingpengyu could you please Elaborate exactly your test results and how significant the user Impact is? Thanks!
hi Adriani In Chinese, it is divided into Simplified Chinese (zh_CN) and Traditional Chinese (zh_TW) issue 9982 Test braille is zh_TW So look for the maintainer of NVDA zh_TW to test this thank
@dingpengyu: Quote from my comment:
Therefore I also tested NVDA 2019.1.1 with the 3.10.0 version of "zh-tw.ctb" and "spaces.uti" (is required, new in Liblouis 3.10.0). The result was the same – no errors. So this issue isn't caused by the braille tables files themselves.
I have heard that some users use a custom table with nearly 100000 entries. how can I test without a braille display?
@DrSooom I have tried to reproduce it with portable version of NVDA 2019.2 placed on my data partition, under Windows 7 without success. To make sure that it isn't something specific to your NVDA config I've even copied your config file - still no error. Things that are different:
Are there any other differencess that you can think of?
First of all I'm so glad that these errors aren't caused by my HUC Braille Tables alone, because other large braille tables like "zh-tw.ctb" lead to the same results I described in issue #9973 yesterday. CC: @leonardder
Steps to reproduce:
Actual behavior:
This time I was able to reproduce both errors I mentioned in issue #9973 on a SSD, but I couldn't figure out if the Chinese braille table "zh-tw.ctb" (of course without the HUC Braille Tables) was completely loaded or not. Maybe you have to restart NVDA multiple times to get these errors.
Here is the first logfile with no error sound on startup. And here is the second one with an error sound and no braille output directly after startup. Always start at line 126. And please take a look at the timestamps in the second logfile, because there aren't any log entries from the beginning of the error sound until the Protocol Viewer was opened. That's so strange, therefore I repeated restarting NVDA in Debug Mode multiple times, because I thought I did something wrong. Normally NVDA should also have logged pressing ArrowRight and ArrowLeft in the Windows Explorer (Detail View) as well as the steps to open the Protocol Viewer via the NVDA menu. But that was never the case.
Expected behavior:
No errors on loading large braille tables.
System configuration
NVDA installed/portable/running from source:
Portable
NVDA version:
2019.2 Beta-18176
Windows version:
Win7x64
Other information about your system:
Optelec ALVA BC680 is connected via USB2. The USB1 port wasn't tested.
Other questions
Does the issue still occur after restarting your PC?
Not tested.
Have you tried any other versions of NVDA? If so, please report their behaviors.
Tests with 2019.1.1 (Portable) are pending. But I guess it's the same behavior like in issue #9973 – it works as expected. And smaller braille tables such as the Finish 8-dot Computer Braille ("fi-fi-8dot.ctb") seems to work with NVDA 2019.2 Beta-18176 as expected too. But deeper tests are pending here too.