languagetool-org / languagetool

Style and Grammar Checker for 25+ Languages
https://languagetool.org
GNU Lesser General Public License v2.1
12.27k stars 1.38k forks source link

ArrayIndexOutOfBoundsException #4046

Closed ColeBantam closed 2 years ago

ColeBantam commented 3 years ago

LibO_Version exeption

ColeBantam commented 3 years ago

Last bug: Your decision if to release another 5.2.x bugfix :) But, is there a 5.2.x Snapshot that contains a fix that I can test?

Heap Space Issue: I guess the heap space issue is a LibO bug? As FredKruse mentioned, LibO allocates the heap space dynamically. LibO itself seems to check the heap space size on start, so if I set -Xmx to more then 750M I get a message that JRE is "corrupt". A bit misleading, but ok. But it seems that LibO itself allocates more then 750M. That must have been the culprit, why LangTool didn't start at one of my last tests. So, is there something that can be done by LangTool, or do I have to report that in LibO Tracker? Btw, this all only applies to the 32Bit version of course. In 64Bit LibO I successfully tested heap spaces up to 16G.

danielnaber commented 3 years ago

Current snapshots are always available at https://internal1.languagetool.org/snapshots/

ColeBantam commented 3 years ago

Current snapshots are always available at https://internal1.languagetool.org/snapshots/

I know, but these are master branch snapshots, not 5.2.x branch snapshots.

danielnaber commented 3 years ago

I know, but these are master branch snapshots, not 5.2.x branch snapshots.

That's right, we'd need to build branch snapshots manually.

ColeBantam commented 3 years ago

That's right, we'd need to build branch snapshots manually.

So it depends on your decision if there will be another 5.2.x release or not. If you want to further stabilize 5.2 branche, a snapshot containing the last fix would be great. Otherwise I would change to testing current snapshots again.

ColeBantam commented 3 years ago

Hello again :) Its been a while, but finally I have a new exception:

next-exeption13

Is this a LT-Bug or can I use it as evidence of a LibO Bug in assigning Java Heapspace? Because it happened while I tried to find a way to reproduce the Problem in LibO that it allocates more heapspace dynamically than it can handle. (in 32Bit LibO).

EDIT: I just tried to close all the dialogues and it happened again, that the spellchecking-window is stuck. That happens quite often. Anything I can do to deliver useful information when that happens? The document from that I opened LT still is responsive and I can close it, but not the LT-Spellcheck-Window. That only disappears when I shutdown the LibO-Quickstarter. Oh wait, this time the window became responsive after closing the document. The last time (maybe half an hour before) it only disappeared after closing the Quickstarter.

FredKruse commented 3 years ago

The Problem with the spell and grammar check dialog should be solved by a change I committed today. Please test it with 5.3-snapshot tomorrow. The Out-Of-Memory-Error seems to be caused by using the "Use more than one CPU-Core" option. If you choose this option, LT runs in multithread mode, which uses also more memory, I think.

ColeBantam commented 3 years ago

Thanks for the commit regarding non-responding-spellcheck-window :) Ill test the snapshot tomorrow.

Regarding the exception: Is it possible to say, if the exception was caused by LT itself, or maybe by LibO because of allocating to much heapspace and thus killing java?

If it was a LT-Bug, it should not be skipped just because it was with an "abnormal" setting. I know that it is a "warning" next to the multi-thread-option, but sooner or later that option should become default, shouldn't it? Or will it remain an "experimental feature" forever?

ColeBantam commented 3 years ago

Please test it with 5.3-snapshot tomorrow

Well, didnt take too long for the first exception...

next-exeption14

Because heap-space is mentioned in the exception. I cant really do anything about that. Its the 32Bit version of LibO and heapspace is not limited by me. So LT has about 750MB, more is not possible in the 32bit version. Multi-threading is already disabled!

next-exeption15

After the first exception I started LibO again and began writing this comment and some other stuff. When I looked back a few minutes later, the second exception already was there, without me doing anything in LibO!

I have to revert back to 5.2.2. now, because I need to do some work which is impossible with this snapshot.

FredKruse commented 3 years ago

I have to test this. It will take a while I have to build a 32 bit test environment. Probably some features have to reduced for the 32 Bit version.

ColeBantam commented 3 years ago

Thanks for having a look into that. But before reducing functionality for 32bit hosts, have a look at bug #4413.

ColeBantam commented 3 years ago

This just happened in LibO 7.1.0.2. Is this a LibO bug? Because C++ is mentioned and LT is (I think) pure Java...

next-exeption16

ColeBantam commented 3 years ago

BTW: Is there a LT version that has a know memory leak, which I could use to demonstrate the LibO heap space allocation bug? Or any other idea how I could make LibO to allocate more and more heap space till it reaches the maximum it can handle? I need something to make that reproducible to file a bugreport in LibO Tracker...

FredKruse commented 3 years ago

I am a little at a loss. I tested LT on a 32-bit Version of Linux Mint (19.2). The default configuration is LO 6.0.7.3. Openjdk 11.0.10). The latest snapshot is used. Beside the default options I added some optional style rules to generate as many matches as possible and switch to multi thread mode. I loaded four documents (real world texts), each one 300 pages approximately, circa 1200 pages at all. The reserved heap space grows up to more than 1 GB, the used was around 900 MB. LT took a long time, but it runs without crash. So I wonder what is wrong on your machine? Could there be a problem with your RAM? The last bug is very strange it is triggered by a very old part of code, and it happens in LO itself. It seems also related by a fail of allocation of memory.

ColeBantam commented 3 years ago

Hi Fred. Thanks for putting so much work into this :)

I also did (and currently do) a lot if tests with LT, and I have some news. But I will post those to the Performance-Bug, as its mostly about heap space.

But I also have a new exception that quite sure really is a LT bug:

LibO_7 0 4 2_32Bit_Java10_CurrentSnapshot-Crash

On Windows this happens 100% reproducible on two different machines. Just open LibO, open a new empty document and close the document (but NOT LibO).

More in #4413

ColeBantam commented 3 years ago

Does the above commit also fix the nullpointer exception?

danielnaber commented 3 years ago

User feedback:

wenn ich Text lösche dann kommt seit der installation von Version 5.4 (2021-06-25 09:43:51 +0000) eine java.lang.ArraayIndexOutOfBoundsException

javaLangError

ColeBantam commented 3 years ago

I have just tested 5.4 Plugin in LibO 7.1.5, and I didn't notice any exceptions so far, even not on deleting text.

My Specs: Windows 10 20H2, 64Bit, Azul Systems Java 12.0.2

But I noticed two other things that might be a problem:

1.) I couldn't trigger the "using too much heap space → switch to paragraph only mode"- Message. Don't know if the message has been removes, but the functionality is still there, or if the message should show up?

2.) The "LanguageTool" Menu in EXTRAS is greyed out and not accessible though. That means I can't have an eye on heap space usage aso..

FredKruse commented 3 years ago

The OutOfBundsException arises if the whole text is deleted. This problem is solved now. Regarding the last comment: I can't reproduce the second point. If a Writer-window is open, the LanguageTool-menu should be accessible.

ColeBantam commented 3 years ago

I can't reproduce the second point. If a Writer-window is open, the LanguageTool-menu should be accessible.

I solved the menu-problem by deleting this file: "C:\Users\UserName\AppData\Roaming\LibreOffice\4\user\config\soffice.cfg\modules\swriter\menubar\menubar.xml"

ColeBantam commented 3 years ago

LangTool-Exeption-2021 09 30 Exeption after deleting a page-break (manueller Seiten Umbruch).

FredKruse commented 3 years ago

Could you please test, if the same bug exists in the latest 5.5-snapshot: https://internal1.languagetool.org/snapshots/LanguageTool-20210930-snapshot.oxt The problem should be already solved, I think.

ColeBantam commented 2 years ago

I can't reproduce it in LT5.4, so I can't say if the Snapshot fixes the problem.

FredKruse commented 2 years ago

The issue is solved. I close it.