Closed aaclayton closed 1 year ago
An update to anyone who comes searching and finds this issue. The root cause of the problem was line ending conversion behavior of git on Windows that, when the data was pulled was attempting to change line endings in the the LevelDB binary files which caused them to become corrupted.
There are two solutions that we found worked. We could solve the problem by doing either of the following:
core.autocrlf
set to input
to avoid changing line endings when pulling code. (https://git-scm.com/book/en/v2/Customizing-Git-Git-Configuration).gitattributes
to the project with a data/** binary
rule targetting all the files generated by LevelDB (https://docs.github.com/en/get-started/getting-started-with-git/configuring-git-to-handle-line-endings)I think the exact issue is that since the MANIFEST-*
file is binary, but does not use a file extension that is recognizable as such, git on Windows was trying to do line ending conversion in that file.
We have been working with classic-level as the data backend for an application that supports Windows/Mac/Linux. The data stored by the application needs to be portable. We had not encountered problems with this, but I have today encountered a scary problem where a database most recently updated in a Linux environment can no longer be opened in Windows:
Context
v16.19.0
v18.13.0
1.2.0
Error
When attempting to call
await db.open()
:Contents of the database folder are as follows:
The
CURRENT
file does referenceMANIFEST-000021
.This error is very alarming, because we had been operating under the expectation that the databases created and modified by LevelDB (and therefore classic-level) are portable. Is this expectation wrong? Are there any troubleshooting steps or further details that I can provide to help diagnose the problem?
The absolute path (
location
) which I'm providing to the ClassicLevel constructor isC:\Users\aaclayton\Documents\Foundry\FoundryData\Data\worlds\ember-dev\data\actors
. Something internal to ClassicLevel (or LevelDB) is appending the manifest file to this path but using a/
separator rather than a windows path separator. I don't know whether this is part of the issue or innocuous.This error is not reproducible with every DB I create in a Linux environment and transfer to windows. I have had success moving databases around before (in either direction). I have not encountered this error before but it is highly troubling. Thank you very much for your guidance.