[BETA] Saving raises an error, resulting in file loss

[TrapinchO]

EDIT:

At this point, the primary bug is deletion of a word that exists as an etymological parent for other words, then saving. PolyGlot will cull the word at save time, creating a diff between the in memory file and the file written to disk. This also disrupts the ability to autosave temp versions, creating really annoying popups periodically.

ORIGINAL TEXT:

Polyglot version: Windows BETA from 18th May 2022 (latest as of writing this)

While saving (autosave, also manual save) the following exceptions are thrown (I got the first one several times, the other one only once). After the file is closed and reopened, "file is corrupted..." prompt is shown. I was able to get a "uncorrupted" file using the temporary file, before losing it to the same error. Reinstalling Polyglot does not help

Steps to reproduce: I was unable to reproduce it.

EDIT: my assumptions are 1) the program got corrupted after downloading and building the source code (which it should not) 2) when it auto-saves while having etymology window opened

EDIT 2: so far it seems like only etymology is affected

[TrapinchO]

Found an error log:

java.version : 17
java.vendor : Oracle Corporation
java.vendor.url : https://java.oracle.com/
java.vm.specification.version : 17
java.vm.specification.name : Java Virtual Machine Specification
java.vm.version : 17+35-2724
java.vm.vendor : Oracle Corporation
java.vm.name : OpenJDK 64-Bit Server VM
java.specification.version : 17
java.specification.vendor : Oracle Corporation
java.specification.name : Java Platform API Specification
java.class.version : 61.0
java.ext.dirs : null
os.name : Windows 10
os.arch : amd64
os.version : 10.0

-=-=-=-=-=-=-
ityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:86)
    at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:97)
    at java.desktop/java.awt.EventQueue$5.run(EventQueue.java:746)
    at java.desktop/java.awt.EventQueue$5.run(EventQueue.java:744)
    at java.base/java.security.AccessController.doPrivileged(AccessController.java:399)
    at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:86)
    at java.desktop/java.awt.EventQueue.dispatchEvent(EventQueue.java:743)
    at java.desktop/java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:203)
    at java.desktop/java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:124)
    at java.desktop/java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:113)
    at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:109)
    at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
    at java.desktop/java.awt.EventDispatchThread.run(EventDispatchThread.java:90)

[DraqueT]

Thank you for the details there! Sorry for the very long replies, I have been crazily busy these last few months. X|

[DraqueT]

Ok, I have tracked down 1/2 of what's going on here, but at this point need to be able to reproduce this. HAve you found any reliable way to do so? Autosaves while the etymology window is open are not activating this bug for me.

[TrapinchO]

Managed to find the likely cause: multiple word entries with the same conlang word make mess in the etymology

Steps for reproduction: (polyglot version unchanged, since it seems like the builds are not updated) 1) create a new file 2) create words test, test2 with local equivalents a and c respectively 3) save the file as test.pgd 4) create word test with local equivalent b 5) save 6) go to word test2's etymology and make the test its parent 7) save 8) go to test2's etymology and remove the parent 9) save 9a) error

a side note: if you select the second test on step 6, both of the entries get added

[DraqueT]

Hot damn, you are good at this! Thank you for finding this one. Just found the root cause and I;ll be able to tidy it up. Part of the new file integrity checker which prevents files from becoming corrupt.

[TrapinchO]

Issue still persist on the broken file. (build from 7th August, latest as of now)

[TrapinchO]

Attempted to unpack the broken file and remove everything within EtymologyCollection. Issue seemed fixed at first but later got #1203 and this again.

[DraqueT]

I am making a new build now! Give it 30 mins and it should be good to retest this.

[DraqueT]

On Windows, that is. If you are testing on Linux, I need a day or so to fix its build. If on Mac... I no longer have access to a mac for dev work and need to remedy that. X|

[TrapinchO]

I am making a new build now! Give it 30 mins and it should be good to retest this.

Build does not seem to be updated as of writing this. The latest windows beta is from 7th of August 2022.

[DraqueT]

Apologies for that, I am reworking my build systems at the moment, and the mechanism that copied the compiled file failed. Should be fixed now.

[TrapinchO]

Downloaded the latest windows beta (18th August 2022, 11:00), still happens for both old and completely new files.

Also it seems to be fixed after removing the etymology (the recovery prompt still appears after reopening the file though)

[TrapinchO]

@DraqueT please reopen

I got this error on windows beta from 30th September 2022 (latest as of writing this). I was editing an old file, which was previously affected by this.

[DraqueT]

Thanks for the heads up! Could you link me to the file? If this is popping up again, I need to make more robust tests surrounding it.

[DraqueT]

@TrapinchO Also as a heads up, I did go through the steps you found which raised the error in regard to etymology farther back in this thread. That one in particular I'm not getting any more.

[TrapinchO]

Reproduction steps for the file: (windows beta from 30th September 2022) 1) add word test 2) add etymology drha 3) save (it should be ok) 4) add etymology enol (second one, delete the first one after choosing) 5) save (crash)

Adding the word itself, not-filling required fields is ok, non-synonym etymology seems to be fine too. Fyi synonyms in the file a drha and enol.

EDIT: also the steps from further up the thread are working for me now too.

what a way to publish a conlang

test_broken.zip

[DraqueT]

Heyo, sorry for leaving this one for so long. New job makes life crazy. This is one of the last ones that I wanted to tackle as absolutely required before the next release though.

[lrruiz]

I have accidentally reproduced this error on Beta Build 2022-10-16 by adding words to my Lexical Families. I've done so to words without an etymology link.

I made a new test language, made a new lexical family, added a five random words using the Word Generator, and saved. I closed the program, opened the file, and added a single word to the lexical family list, and it gave the error again (on an almost blank language).

Opening the file, making a change to the lexical family, and deleting the change still causes the error.

Deleting the temporary files in the /USER/Polyglot folder doesn't fix the issue. A new xxTEMPPGTFILExx file that is 7KB is created every time the save crashes.

Incidentally, I was not able to recreate the issue using TrapinchO's test file at first. Only after adding a new lexicon entry to their list, and then adding the word to the lexical family list, did it cause an error. Maybe I just had to wait for the first autosave? Regardless, it happened using that file too.

I have not tried reverting to the stable build to try it there.

[DraqueT]

@lrruiz Thank you for the details here! I have been implementing a powerful comparison engine to better enforce save fidelity, and this is one of the kinks that are still being worked out from it.

EDIT: I can reproduce, thank you for the heads up here!

[DraqueT]

I am tentatively closing this again... I have done some extensive testing, and believe that I have plugged at least all known holes in terms of file saving. Ultimately I'm planning on replacing the XML parser entirely, but that will take a decent amount of work that I'm not ready to do yet. @lrruiz you can download the new beta with fixes here:

https://draquet.github.io/PolyGlot/

[lrruiz]

Thank you as always for spending time on this! Unfortunately, using Beta build 2022-12-20 17:42, I still can't add words to my lexical family lists without causing the "written file does not match file in memory" save error.

Now, I haven't fully tested this out to check if there is data loss, but there is a workaround to save the modified (offending) file. When the program fails to save the file in the original location (my personal conlang files are saved to a different drive), it makes a temporary file in in C:Users/USER/Polyglot. If that file is then renamed with a .pgd file extension, it opens just as if it had been the original file—it even maintains the changes made to the lexical family lists. I can carry on and continue adding words to the lexicon and overwriting the original file; it will save like normal unless I mess with the lexical family list again.

Seeing that I'm satisfied using Polyglot for its other features, I'm not bummed at all, and appreciate the improvements done already.

Dered!

[DraqueT]

@lrruiz Could you please send a copy of the language file to me at draquemail@gmail.com? I think I might need to look at this directly. Also, I appreciate you getting back to me so quickly! It's great to have folks willing to help me track down these issues. ^^

[DraqueT]

@lrruiz Ok! New beta up which I have tested with the language file you sent along. Let me know if this solves your issues! :3

[DraqueT]

I have not heard more about this and not been able to reproduce in a few months after the fix. Closing for now.