Spreads / Spreads.LMDB

Low-level zero-overhead and the fastest LMDB .NET wrapper with some additional native methods useful for Spreads
http://docs.dataspreads.io/spreads/libs/lmdb/api/README.html
Mozilla Public License 2.0
80 stars 9 forks source link

Win x86 support ? #40

Closed RockNHawk closed 4 years ago

RockNHawk commented 4 years ago

Hi,

Spreads.LMDB work well on x64, but in Windows x86 , will throw a libspreads_lmdb.dll not found error.

I'v fond libspreads_native.dll in x86 bin folder, but libspreads_lmdb.dll only in x64 folder,

does preads.LMDB support Windows x86?

Thank you!

buybackoff commented 4 years ago

LMDB does support x86, but the file formats are not binary compatible with x64, so for me it was a deal breaker. Also it's much slower. So I didn't compile for x86.

But you could compile it yourself or get the binary somewhere else and place next to this library dll, it should just work.

buybackoff commented 4 years ago

https://www.nuget.org/packages/LightningDB/ has it. You will need to rename it though and Spreads' helper methods won't work (e.g. Find LessOrEqual and similar)

RockNHawk commented 4 years ago

Thanks for your guidance, I have some compile environment error, could you help me, I installed MinGW, and installed these package:

image

but when i run mingw32-make.exe in Spreads.LMDB\lib\libspreadsdb\src\libspreadsdb folder, this error happened:

error: unknown type name 'NtCreateSectionFunc'

the detail error:

C:\git\lmdb\Spreads.LMDB\lib\libspreadsdb\src\libspreadsdb>C:\MinGW\bin\mingw32-make.exe
gcc  -DMDB_MAXKEYSIZE=0 -pthread -O2 -g  -W -Wall -Wno-unused-parameter -Wbad-function-cast -Wuninitialized   -I. -I../../lmdb/libraries/liblmdb -fPIC  -shared -Wl,--output-def=libspreads_lmdb.def -Wl,--out-implib=libspreads_lmdb.lib -Wl,-soname -Wl,--dll spreadsdb.c ../../lmdb/libraries/liblmdb/mdb.c ../../lmdb/libraries/liblmdb/midl.c -o libspreads_lmdb.dll  -pthread -s
../../lmdb/libraries/liblmdb/mdb.c:55:26: error: unknown type name 'NtCreateSectionFunc'
55 | typedef NTSTATUS (WINAPI NtCreateSectionFunc)
    |                          ^~~~~~~~~~~~~~~~~~~
../../lmdb/libraries/liblmdb/mdb.c:61:8: error: unknown type name 'NtCreateSectionFunc'
61 | static NtCreateSectionFunc *NtCreateSection;
    |        ^~~~~~~~~~~~~~~~~~~
../../lmdb/libraries/liblmdb/mdb.c:68:26: error: unknown type name 'NtMapViewOfSectionFunc'
68 | typedef NTSTATUS (WINAPI NtMapViewOfSectionFunc)
    |                          ^~~~~~~~~~~~~~~~~~~~~~
../../lmdb/libraries/liblmdb/mdb.c:75:8: error: unknown type name 'NtMapViewOfSectionFunc'
75 | static NtMapViewOfSectionFunc *NtMapViewOfSection;
    |        ^~~~~~~~~~~~~~~~~~~~~~
../../lmdb/libraries/liblmdb/mdb.c:77:26: error: unknown type name 'NtCloseFunc'
77 | typedef NTSTATUS (WINAPI NtCloseFunc)(HANDLE h);
    |                          ^~~~~~~~~~~
../../lmdb/libraries/liblmdb/mdb.c:79:8: error: unknown type name 'NtCloseFunc'
79 | static NtCloseFunc *NtClose;
    |        ^~~~~~~~~~~
../../lmdb/libraries/liblmdb/mdb.c:4333:14: error: unknown type name 'NTSTATUS'; did you mean 'RPC_STATUS'?
4333 | mdb_nt2win32(NTSTATUS st)

Thank you!

buybackoff commented 4 years ago

Something with #ifdefs. Those symbols are defined by LMDB for not to include them from windows headers. The place where they are defined is unreachable.

RockNHawk commented 4 years ago

I giveuped ...., thank you!

buybackoff commented 4 years ago

Why could't you just use the existing binary?

RockNHawk commented 4 years ago

I used the existing binary from LightingDB, but have exception, I will take a debug for more info, here is the simple exception info:

Error Open DB Environment C:\Users\Administrator\Data\Volume.C:MDB_INVALID: File is not an LMDB file
Spreads.LMDB.LMDBException: MDB_INVALID: File is not an LMDB file
at Spreads.LMDB.Interop.NativeMethods.ThrowLMDBEx(Int32 res) in G:\Dev\FIDS\lib\DataSpreads\lib\Spreads.LMDB\src\Spreads.LMDB\Interop\NativeMethods.cs:line 127
at Spreads.LMDB.LMDBEnvironment.Open() in G:\Dev\FIDS\lib\DataSpreads\lib\Spreads.LMDB\src\Spreads.LMDB\LMDBEnvironment.cs:line 193

this exception's exception.Message is Exception of type 'Spreads.LMDB.LMDBException' was thrown. and LMDBException.Code is 8.

Spreads.LMDB.LMDBException: Exception of type 'Spreads.LMDB.LMDBException' was thrown.
at Spreads.LMDB.Interop.NativeMethods.ThrowLMDBEx(Int32 res) in G:\Dev\FIDS\lib\DataSpreads\lib\Spreads.LMDB\src\Spreads.LMDB\Interop\NativeMethods.cs:line 127
at Spreads.LMDB.LMDBEnvironment.Open() in G:\Dev\FIDS\lib\DataSpreads\lib\Spreads.LMDB\src\Spreads.LMDB\LMDBEnvironment.cs:line 193
at Labx.LMDB.LmDatabase`2.InitAsync(ITrace parentTrace) in C:\git\app\src\Labx.LMDB\LmDatabase.cs:line 163
buybackoff commented 4 years ago

Are you trying to open a file created by x64?

RockNHawk commented 4 years ago

@buybackoff No, db file is new created. The first case is re-open a db file created by x86 version, and the db file have data.

RockNHawk commented 4 years ago

May be the https://www.nuget.org/packages/LightningDB/ LMDB version mismatch? Could you compile a spreads lmdb dll for me? Because I used some spleads extended feature like Find.

buybackoff commented 4 years ago

May be the https://www.nuget.org/packages/LightningDB/ LMDB version mismatch?

It's the same thing, different names.

Could you compile a spreads lmdb dll for me?

No, not until I need it myself. Look at Makefile (upstream and Spreads), it's very simple for x64. For x86 you may need to define some params (commnented in Makefile), download MinGW x86 (or try to use https://github.com/microsoft/vcpkg, but I do not know if it supports x86), and if you are lucky you will get the binary in 20 mins total. Or you will have to dig deeper and investigate, then test that it really works.

Because I used some spleads extended feature like Find.

You may look how the Find methods are implemented here: https://github.com/Spreads/Spreads.LMDB/blob/master/lib/libspreadsdb/src/libspreadsdb/spreadsdb.c Nothing stops you from doing the same in C#.

buybackoff commented 4 years ago

Win x86 is not supported by this project. (same as nested transactions)

A pool request with x86-adjusted Makefile and instructions to build is welcome.

RockNHawk commented 4 years ago

Okay, thank you very much !!