region_file.cpp error: FATAL: Condition "written_size != (end_pos - block_offset)" is true.

[IamTheCarl]

My windows users are experiencing the following error. I am able to replicate it under Wine on my Linux system as well.

ERROR: FATAL: Condition "written_size != (end_pos - block_offset)" is true.
   at: save_block (modules/voxel/streams/region/region_file.cpp:371) - FATAL: Condition "written_size != (end_pos - block_offset)" is true.

Branch info:

vscode ➜ /opt/godot/modules/voxel (godot3.4) $ git status
On branch godot3.4
Your branch is up to date with 'origin/godot3.4'.

nothing to commit, working tree clean
vscode ➜ /opt/godot/modules/voxel (godot3.4) $ git log
commit 49f7f90786a4f286b037cdcdcead6a589024a1fd (grafted, HEAD -> godot3.4, origin/godot3.4)
Author: Marc <marc.gilleron@gmail.com>
Date:   Tue Jan 11 22:12:05 2022 +0100

    Merge pull request #347 from Gamerfiend/godot3.4

    Mono CICD for Godot 3.4

Windows cross compiler info:

vscode ➜ /opt/godot/modules/voxel (godot3.4) $ x86_64-w64-mingw32-g++ --version
x86_64-w64-mingw32-g++ (GCC) 9.3-posix 20200320
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Is there any additional information I can provide that is helpful?

[IamTheCarl]

I have log files from other users getting similar errors, but triggered by different lines from this file: https://github.com/Zylann/godot_voxel/blob/godot3.4/streams/region/region_file.cpp#L371 https://github.com/Zylann/godot_voxel/blob/godot3.4/streams/region/region_file.cpp#L416

[Zylann]

How large is the region file? Any errors prior to the fatal one?

The only possibilities that come to mind are:

• Godot 3 does not support files bigger than 2Gb. Maybe the file became larger?
• Region files dont support being that big either due to their limited number of sectors. Did you store lots of metadata in there?
• sizeof(int) in the code is not portable and should probably be changed, however on Windows it is 32-bit so that's unlikely to be the issue.
• The engine only uses one thread for I/O so there should not be a concurrency bug. The resource is also protected by mutex. Are you trying to modify the file or the resource from outside the I/O thread? (like, in the main thread?)

Do you have a minimal project reproducing the issue?

[IamTheCarl]

• There are only two region files, both 18kb in size.
• I do not store any metadata.
• I don't have any custom code modifying the region files. I do store some of my own files outside of the regions directory. None of them should interfere with meta.vxrm.

I can see if I can produce a minimal reproduction of the bug, but since I'm suspicious the compiler is part of the issue, I'm not sure how to get that to you if it is.

Can you make use of a Docker container?

Would it be better if I just gave you access to view my whole project's repository? It too is built with a Docker container, so if you have Docker, you won't need to install anything else to build it.

[Zylann]

I have no experience with Docker at all. I think if you can reproduce on Windows using MSVC, then I should be able to reproduce as well.

[IamTheCarl]

I develop primarily on Linux so that is, unfortunately, a rather difficult setup for me to make. I'll see if I can diagnose the issue myself.

Give me a few minutes to make a plan. I think we can both benefit if I do this right.

[IamTheCarl]

Here is your minimal reproduction of the bug. WARNING: There's a lot of flashing since it's just rapidly modifying and saving terrain as fast as it possibly can.

The number of saving cycles it goes through before a crash happens seems kinda random. :(

https://github.com/IamTheCarl/voxel_tools_minimal_crash

[Zylann]

I see it's not been made explicit yet in this issue, so here is the relevant code from the godot3.x branch:

https://github.com/Zylann/godot_voxel/blob/726bd807506cef7f7e8ee34ba774ec793ec5f034/streams/region/region_file.cpp#L360-L371

The error occurs if the check at the last line fails. As you might notice, there aren't many possible causes for that. It really looks like get_position() returns an unexpected value, or store_buffer did not write the expected amount of bytes... Some more notes:

• As mentionned earlier, sizeof(int) could be a problem, and was replaced with sizeof(uint32_t) in the master branch. But even then, after some other testing, it sounds like it's not really the issue.
• Also mentionned earlier, file size is not to be suspected because this bug has been seen on much smaller region files, and the format itself doesnt even support enough sectors to grow that big.
• I wrote some tests in master writing/reading from region files, and so far none of them were able to reproduce this, at least on Windows with MSVC.
• It feels like MinGW or GCC could be a problem for some reason, but that's only because nobody reproduced it with MSVC yet.
• In one testing session with some extra logging, it appeared the written size was off by one byte. However I don't see any reason for it to be off by exactly one byte in my code...

[Zylann]

Moving the insight found in https://github.com/Zylann/godot_voxel/issues/365 so I can close the duplicate:

In the Godot 3.x Mono Builds of the Github Actions CI as it is now, FileAccess::get_position() does not work correctly. It returns a position as if the file was open in text mode. There is no text mode in Godot. All files are supposed to be binary. Therefore there is no code fix available at the moment. An example in how it manifests is, if you store 10 bytes containing the value 10 (which corresponds to the character \n), then FileAccess::get_position() returns 20, instead of 10, even though only 10 bytes actually get written. This can cause nasty bugs with binary save files. On Windows, files are open in binary mode here: https://github.com/godotengine/godot/blob/33500a1529ba1fb0e471aea0e4af3e704619a828/drivers/windows/file_access_windows.cpp#L61 And uses _ftelli64 to obtain the position: https://github.com/godotengine/godot/blob/33500a1529ba1fb0e471aea0e4af3e704619a828/drivers/windows/file_access_windows.cpp#L210 However _fseeki64 and fseek seem to be used quite inconsistently. Not sure if that's supposed to have an impact. It is currently unknown why this happens in that specific build, this needs to be investigated.

You can verify if your Godot version has this bug by running this minimal project: https://github.com/Zylann/godot_voxel/files/8403297/StoreBuffer.zip

Current workarounds:

• Try building your own Mono version
• Don't use the Mono version
• Use VoxelStreamSQLite instead of VoxelStreamRegionFiles