ArrayIndexOutOfBoundsException

[ColeBantam]

[danielnaber]

Does it still happen with the latest snapshot? https://internal1.languagetool.org/snapshots/LanguageTool-20201218-snapshot.oxt

[FredKruse]

@danielnaber: As far as I see, this bug shouldn't happen with the latest snapshots. I did a lot of changes (also at this part of code). The latest version should much more stable.

[ColeBantam]

My first impression was, that the current snapshot is more stable, but then ....

Ahhh, the exact same exeption once again...

[FredKruse]

@ColeBantam : The issue should be solved. Please test it tomorrow.

[ColeBantam]

its getting more and more stable... BUT .....

[FredKruse]

The issue should be solved. Please test tomorrow.

[ColeBantam]

Thanks, I'll test tomorrow. Merry Christmas 🎅 to the Languagetool-Team :)

[ColeBantam]

My X-Mas Present :D

[FredKruse]

Hopefully my X-Mas present is a little bit nicer ;) Please test it tomorrow.

[ColeBantam]

Hopefully my X-Mas present is a little bit nicer ;) Please test it tomorrow.

Thanks, I'll test your Present tomorrow :D

Meanwhile I stumbled upon another exception ...

This time it didn't happen on a fancy test-situation, but simply on writing a simple letter with 399 English words.

[FredKruse]

Ok, this seems to be the same bug. Please test tomorrow, if it is solved.

[ColeBantam]

The same? ok. Looked so different to me... Ill try tomorrows build.

[ColeBantam]

Current Snapshot seems stable for the moment. Didn't do a lot of real world text processing in Writer yesterday though. But tests with my big test-document were fine.

[ColeBantam]

Hello again :)

Here is my new years day exception :D

[FredKruse]

Could you please describe, at what user action this exception arise?

[ColeBantam]

Could you please describe, at what user action this exception arise?

I think I was deleting the content from several rows from the first of 4 tables in a writer document. Or it happend while cutting & pasting content from one buch of rows to another. Not sure at what step exactly it happened.

[FredKruse]

I just committed a small workaround to fix the issue. This will solve the problem above. But there seams to be a general problem with the handling of tables. I will test the problem in the next weeks.

[ColeBantam]

Thanks :) So I'll download the new snapshot tomorrow...

[ColeBantam]

Okkkkkkkeeyyyy, 5.3 Snapshot crashes on every writer start...

[ColeBantam]

LanguageTool-20210102-snapshot seems to work for the moment...

[FredKruse]

@danielnaber: Do you have any idea? I reported this problem already at issue #4088. I am not able to start the extension. It is not possible to do any tests. It seems to be a problem with the AbstractUnitConversionRule.

[danielnaber]

4088 has been fixed now

[ColeBantam]

LanguageTool-20210104-snapshot seems ok so far.

[ColeBantam]

Even though it might not be very useful to you, because it doesn't contain any debug information, I wanted to share the last crash with you:

It's a nice coincidence, that the crash happened while I was writing a news-article about the current languagetool-plugin :D

BTW: If it gets annoying that I reuse that bug on and on, just tell me, then Ill create a new bug for the next crash.

[ColeBantam]

And the next exception, this time more verbose luckily ...

[danielnaber]

The snapshot gets daily updated, so if you use that, I suggest updating regularly.

[ColeBantam]

I only use the snapshots because I was being told to do so by you 20 days ago :D And because the 5.2 release doesn't contain the latest fixes for all the exceptions in this bugreport. So I doubt that its useful to you, if I use the stable build and post exceptions that happen in that one...? I just upgraded to the last snapshot ...

[danielnaber]

So I doubt that its useful to you, if I use the stable build and post exceptions that happen in that one...?

Well, we should try to fix bugs in 5.2, as it's supposed to be a stable release. @FredKruse What do you think, should we publish a 5.2.1 release?

[ColeBantam]

and once again with the current snapshot ...

[ColeBantam]

Well, we should try to fix bugs in 5.2, as it's supposed to be a stable release. @FredKruse What do you think, should we publish a 5.2.1 release?

A 5.2.x branch with focus on stability would be really very welcome ... The LibreOffice 7.x and LanguageTool combination is really a pain so far ... :/ Never had that much issues with previous LibO / LanguageTool combos.

[FredKruse]

In this discussion there are three different problems, I think:

1. OutOfMemoryError: This is not an LT error. It is caused by missing heap space. Properly you try to work with a very huge document or with many large documents in the same session. LT uses caches to handle rules on text level. They grow with the size of the loaded documents. In LO you can expand the size of heap space by adding the parameter -Xmx1G (for 1 GB, -Xmx2G for 2 GB, etc.) as java parameter in LO in Tools/Options/Advanced.
2. Release 5.2.1: Two bugs were detected, related to release 5.2. One was fixed with commit https://github.com/languagetool-org/languagetool/commit/26080c3688ccf2dd7281e1dd928bbac9164a9fd2. If we create a branch 5.2.1, we should start with this release, but I have to add another small fix, before we release it. @danielnaber: could you create such a branch?
3. NullPointerException on 5.3-snapshot: The code has changed meanwhile. I can not reproduce it. I did some changes at the core of the engine in 5.3 (hopefully the last time) to optimize cache handling. @ColeBantam: Please test it with the newest snapshot and report the bugs here. It would be much easier, if we could fix the bugs of 5.3 before it will be released.

[danielnaber]

could you create such a branch?

I think you have permission to do that yourself. Just start from tag v5.2, not from current master.

[FredKruse]

@danielnaber: I created the branch https://github.com/languagetool-org/languagetool/compare/v5.2.1-branch and did the changes, which solves the reported bugs regarding the v5.2. I am not familiar with the changes, which have to be done for versioning. Could you please add the changes?

[danielnaber]

@FredKruse The release is now out (also see https://forum.languagetool.org/t/ann-languagetool-5-2/6465/2?u=dnaber).

[ColeBantam]

I wish there would have been a possibility to test 5.2.1 prior to release :/

@FredKruse Thanks for the hint to the heap-size allocation in LibO. Do you know what default size LibO uses and if its different in x86 and x64 builds?

[ColeBantam]

While experimenting with different heap space settings:

[ColeBantam]

Next exception while doing tests with different heap space settings ... Seems related but not the same as before...

Update 1: This Exception is reproducible. Whenever I open my BIG test-file (403 thousand words) and click on the "Text erneut prüfen (Umschalt + F7)" option immediately, it triggers the above exception. All with heap space set to 256M.

Update 2: I initially wanted to give you some feedback on the effect of different heap space settings. But the above exception shows up to often to really do some serious tests. With 512M heap space alone, the exception came up in 2 out of 5 tries. The only thing I can say for sure, is that 1G heap space doesn't work at all (not with 100k words and not with 400k words) except on very small files (137 words), but what for should I need 1GB heap space for such tiny documents? So languagetool or LibO rally doesn't like big heap spaces (at least in the 32Bit version).

[FredKruse]

Did all exceptions happens, when you're running the grammar check dialog?

[ColeBantam]

Yes, I think all the exceptions yesterday were triggered by the "Text erneut prüfen (Umschalt + F7)"-function.

[FredKruse]

The reported bugs are fixed. Hopefully, I fixed the general problem with the check dialog. @danielnaber: Could we put an oxt file at some place, that it can be tested, before we release it? @ColeBantam: Would you be able to test the new version, before we release it?

[ColeBantam]

Of course I would test a snapshot with new fixes.

[danielnaber]

Please give it a try: https://languagetool.org/download/LanguageTool-5.2.2-beta.oxt - note that it will be called "5.2" in the extension manager, it's "5.2.2" in LanguageTool's "About" window.

[ColeBantam]

5.2.2 looks very promising :) I can't trigger any exception so far with tests. I'd like to use it in real world for a couple of days though before releasing it.

Also the tipp with the "about" window gave me a good insight :) Now I can say, that 32Bit LibO uses 256MB heap space by default. Using 512MB works too. 1024M (or 1G) does NOT work in LibO, at least languagetool doesn't work with that setting. Maybe bigger heap spaces work in 64bit version, but I guess that shouldn't be necessary. My huge-doc (400k words) is a testfile, I don't think thats a typical real world size for most LibO users :D

[ColeBantam]

Short Update on the 5.2.2 Beta Plugin: I had two fatal LibO crashes today while working on my newsletter document. Since there was no visible exception I can't tell if it was triggerd by the plugin or LibO itself. After the first crash LibO offered the chance to send an Report to the developers which I did. The second crash did not offer anything. I then removed the -Xmx512M-Parameter and the next 50 minutes in LibO were flawless. No more visible exceptions from the plugin so far.

[danielnaber]

@ColeBantam and @FredKruse Do you think we can release 5.2.2 now?

[ColeBantam]

Its a big improvement compared to 5.2.1, that speaks for release. The two fatal crashes might speak against it, if they were triggered from LangTool, which I don't know. As I don't think we will be able to really say what was responsible for the crashes, I'd say better releasing it, as it fixes serious bugs.

[FredKruse]

The crashes are a mystery to me. I have worked a lot with the latest version of LT without any crash of LO. However, I am using LO 7.0.4 64-Bit Linux version. The heap space is something between 512 MB and 1 GB depending on the size of the document (It allocates automatically without any use of parameters). Maybe LO is not able to allocate enough heap space, and that is the reason for the crashes. It seems there are no more LT exceptions now. So, I think, we should release 5.2.2 now.

[danielnaber]

5.2.2 is now online at https://extensions.libreoffice.org/en/extensions/show/languagetool and https://extensions.openoffice.org/en/project/languagetool. If there are no complaints, I will activate the update from LibreOffice's extension manager (which still offers 5.1).

[ColeBantam]

Thanks for all your efforts and the 5.2.2 Release :)

But to make it not too boring for you, here is a new exception :D

This one came up while my tests with LibO 7.1.0.2, so the Image from my first post here is not correct on this exception. Shall I make a new bugreport for issues in LibO 7.1.x?

Update: Forgot to mention, that Java also is updated now. Oracle Java 8.0 Update 281.

Update 2: Ah, and I had one of those situation where LangTool is stalled (hangs but no exception). I think this was because LibO allocated more then 512MB to the heap and as my tests with 1G heap space showed, Langtool (or the LangTool/Libo Combo) doesn't like such big heap spaces in 32Bit LibO.

[FredKruse]

@ColeBantam: Thank you for testing, you've found another bug. @danielnaber: I am not sure, if we should release a new version. The issue was a real bug, but it should happen very rarely. I think, a "check text again" was initiated and LT was faster than LO (LO has not created a xComponent at the moment when LT asked for it).