search

bmatsuo / lmdb-go

Bindings for the LMDB C library
BSD 3-Clause "New" or "Revised" License
157 stars 59 forks source link

Invalid constant valMaxSize in lmdb/val.go when run in 32 bit OS #81

Closed michael2m closed 8 years ago

michael2m commented 8 years ago

Getting "array bound is too large" from Go in lmdb/val.go

Originates from valMaxSize constant. On 32-bit processors this is 1<<31-1 (2GB) rather than 1<<32-1 (which would be the unsigned integer length). Also on 64-bit processors the max value could be much larger for LMDB. Although as stated in comments this is for portability. However this strongly constrains database size.

bmatsuo commented 8 years ago

Derp. My apologies. The runtime malloc.go code is somewhat confusing and I guess I misread it. Can you tell me what operating system and arch you are compiling for?

I believe the C library is designed with a hard limit of 1<<32-1 for a single MDB_val (the constant MAXDATASIZE). So I don't think larger sizes need to be considered.

Sorry for the delayed response. I have been on a rather long holiday.

bmatsuo commented 8 years ago

@michael2m I looked into the problem and I understand why the compilation fails during type checking. I pushed PR #83 to fix the situation. Can you confirm that the compilation error you saw has gone away? I will merge the change when that is confirmed.

Unfortunately I have a hard time verifying compilation on platforms I don't possess because cgo cannot be cross-compiled simply.

michael2m commented 8 years ago

Partially my mistake, it is about the maximum data item size. Which is in LMDB set to 32 bits (0xffffffffUL in source, I believe lmdb.c). So please ignore my remarks about 64-bit arch. I think your PR fix is acceptable for practical purposes. Retried on target device (ARMv7, 32bit) and compilation succeeds. Thanks for the fix.