Save spontaneously corrupted

[Headjack]

For whatever reason, a bunch of files are now full of NULL; artifacts, uistate and #WW~ included.

save at https://anonfiles.com/file/f9596fd0428e89ec9e4f4a79d99dc6ef

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

[Headjack]

I had this happen again in the same install, do you guys want me to up the folder?

[KA101]

There was a version a day or two ago where Narc somehow borked the JSON handler--that could certainly cause save problems, but he fixed the issue promptly. You using latest?

[Headjack]

No I haven't been since a few days ago, I'll update it and see if it happens again. Thanks.

[narc0tiq]

There was a version a day or two ago where Narc somehow borked the JSON handler--that could certainly cause save problems

I was thinking that, but then you couldn't load the save in the first place, I think. Still would like to know if this install was from the same period, though.

[Headjack]

@narc0tiq it was in 714a48f, but the NULLification didn't happen until after hours and hours of play, and I think one savefile migration to a new version.

[narc0tiq]

Ah, yeah, that's from after I broke and re-fixed it (just a few hours later, but definitely after).

Hmm. If it is the same issue as #8947, it might be related to #8683.

That files are getting written full of zeroes is kind of scary. If it hadn't happened to two people now, I'd be wondering more about hard-drive corruption or RAM failure (to be fair, it might still be that), or perhaps some other application interfering with the files on disk.

This is likely to be really hard to figure out, given there's no real starting point for debugging.

[Zireael07]

Given that an older save and an older build work fine, I'm more inclined to suspect programming error than HDD/RAM/other apps.

EDIT: 2061 works fine while 2064 had hanged on saving once, probably causing the later corruption. So glad I'd kept the build and the save in 2061. I even went through the same sequence of events and to other tinymaps and nothing happened.

[illi-kun]

I see no way to reproduce/test this bug. We didn't receive any similar reports for 1+ year, so I'm going to close this.

Feel free to re-open (with more specific details) if it still happens.

[Vzor-]

@illi-kun @Zireael07 I have been a similar issue for dozens of builds over the course of a year or so. It is how I lose every character who lives long enough (usually half a year to a year and a half). I don't mod, I have no hdd issues and my system is a clean windows 10 box. It's just windows with some IDEs and gimp. This time it failed, it gives me failed to read errors then loads a world, the map looks right, but my car is gone and walking around brings more errors. The save files mentioned usually go something like this "(normal map file so far) ..."t_grass","t_tree_blackjack","t_tree_young","t_gra... (rest of file is thousands of null characters.)" The last time I closed the game everything was fine, I opened my laptop and went to continue and the save was corrupted. It was suggested to me that I mention this here but I can't think of a way to make it reproducible, so I can't imagine you could do anything more than shooting in the dark.

[narc0tiq]

Since this issue was opened, I've seen (and heard of) this happening in an entirely unrelated context: app.config files edited using the .Net framework's ConfigurationManager sometimes end up partially or completely overwritten by zeroes (NUL characters) with no rhyme or reason. It is not reliably reproducible, though I have an unconfirmed suspicion that it happens only on some machines (suggesting hardware failure or partly corrupted OS on those machines). (Counter-point: it seems to only happen to some files; is it about how the files are accessed? That might make it about a specific Windows API.)

Regardless, the only way to fully resolve this is to observe it happening (watching system calls, the contents of the file on disk, and any other pertinent data), so as you said, I can offer no more than shots in the dark. Maybe leave the machine running for a few minutes after closing Cataclysm, if you're in a habit of shutting it down immediately thereafter. Maybe launch a memory-hogging process after Cataclysm (Firefox/Chrome are good candidates) in hopes of forcing leftover caches to be written to disk before anything zeroes them (this may alternately make the problem more common). Maybe run the game on a different machine (a virtual machine, possibly). Maybe update or reinstall your OS or try a different OS.

Either way, it's really weird.

[BlythT]

Sorry for the necro, but I am also facing this issue. However in reading your replies I believe I understand what the issue is. I run ubuntu linux on an old laptop and often shut down my computer straight after closing C:DDA. So as you said I suspect caching to be the culprit.

Cannot speak for other users problems however.

[kevingranade]

There are a few measures we can take to try to avoid short reads and short writes.

Add some flush calls when exiting a game (A game session, flushing when exiting the program is likely too late).

Use a copy-and-move scheme for file writes. In situations that result in a corrupt file now, it would instead result in an old file, and it would also be detectable.

On the mitigation front, we could write a checksum or hash along with the with files. With map files at least we can regenerate a similar file instead of killing the game.

There are likely others, but those are the highest chance of making it work.