search

kriszyp / lmdb-js

Simple, efficient, ultra-fast, scalable data store wrapper for LMDB
Other
481 stars 39 forks source link

Error: MDB_CORRUPTED: Located page was wrong type #185

Closed skyzyx closed 1 year ago

skyzyx commented 1 year ago

Cross-posting for visibility: https://github.com/parcel-bundler/parcel/issues/8288

There seems to be an issue with how Parcel leverages this library on x86_64 Macs vs arm64 Macs.

kriszyp commented 1 year ago

Posted response on the parcel ticket, but thank you for cross-posting so I was aware of this.

kriszyp commented 1 year ago

@skyzyx Here is what I have done so far to attempt to reproduce your issue, replicating your node version on arm64, in hopes that the issue isn't really about MacOS version, but maybe could be reproduced on linux arm64 as well: https://github.com/kriszyp/lmdb-js/blob/master/.github/workflows/test-arm64.yml Unfortunately this runs without any issue. And unfortunately, it is very difficult to find any CI out there with real M1 processors (in the past I have just relied on friends, other devs to verify lmdb-js on M1, and as far as I know, lmdb-js works on the majority of Macs with M1s). I think my next step will be to see if I can provision an AWS machine with an M1, since I think they recently made those available. Let me know if you have any other ideas.

kylebernhardy commented 1 year ago

@kriszyp I can get an AWS M1 machine spun up for you. Let me know what you need.

kriszyp commented 1 year ago

@skyzyx We fired up a couple AWS instances to see if we could reproduce this. The AWS Mac M1 instance seemed to run tests without issue and I did my best to recreate your project and it seemed to build fine on the M1 as well. I also tried this on an AWS Arm64 instance (Graviton 2). I did initially see the segfault, however this actually seems to just be a result of me not checking in the automated binary dependency versions, and therefore getting an outdated binary, but this shouldn't affect the actual lmdb-js releases (and I will fix this issue of installing from source on the next release). Doing a real npm install --build-from-source seemed to properly build and the tests ran without issue on that machine. So I haven't been able to really reproduce any issue.

(And thank you @kylebernhardy for setting up the AWS instances!)

skyzyx commented 1 year ago

So, it turns out that I'm stupid.

I spent days trying to figure out why my project kept giving me this error, but every test case I tried to put together kept succeeding.

Once I deleted the .parcel-cache directory, the problem went away. I think — somehow — I'd managed to build the .parcel-cache directory with Intel settings on my Apple Silicon Mac.

I do want to say, however, that your customer-focused attitude was a tremendous breath of fresh air, and that I greatly, greatly appreciate your willingness to help out a stranger on the internet.

From the bottom of my heart, thank you kind stranger.