py-lmdb fails to install on Cygwin

[automorphis]

Affected Operating Systems

• Windows
• Cygwin

Affected py-lmdb Version

1.2.1

py-lmdb Installation Method

pip install lmdb

Using bundled or distribution-provided LMDB library?

Bundled

Describe Your Problem

py-lmdb fails to install on Cygwin. I did some research and it seems this is because Cygwin does not support the full POSIX pthread specification (please see mailarchive.com and cygwin.com). I am able to compile successfully using the flag -DMDB_USE_ROBUST=0, as suggested in this comment on the LMDB repo. However, I am not able to import lmdb after installation.

Errors/exceptions Encountered

When trying to pip install lmdb without -DMDB_USE_ROBUST=0, I get the following:

  build/lib/mdb.c:4898:53: error: ‘PTHREAD_MUTEX_ROBUST’ undeclared (first use in this function); did you mean ‘PTHREAD_MUTEX_DEFAULT’?
   4898 |   if (!rc) rc = pthread_mutexattr_setrobust(&mattr, PTHREAD_MUTEX_ROBUST);
        |                                                     ^~~~~~~~~~~~~~~~~~~~
        |                                                     PTHREAD_MUTEX_DEFAULT
  build/lib/mdb.c:4898:53: note: each undeclared identifier is reported only once for each function it appears in
  error: command 'gcc' failed with exit status 1

When I pip install lmdb with -DMDB_USE_ROBUST=0, when trying to import lmdb, I get the following:

  import lmdb
  Traceback (most recent call last):
    File "/home/sage/py-lmdb/lmdb/__init__.py", line 42, in <module>
      from lmdb.cpython import *
  ModuleNotFoundError: No module named 'lmdb.cpython'

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
    File "/home/sage/py-lmdb/lmdb/__init__.py", line 48, in <module>
      from lmdb.cffi import *
    File "/home/sage/py-lmdb/lmdb/cffi.py", line 353, in <module>
      import cffi
  ModuleNotFoundError: No module named 'cffi'

[automorphis]

After reading the py-lmdb docs, I realize I forgot to pip install cffi. After I did that, however, I get the following error when I try to open a new Environment:

  >>> import lmdb
  >>> from pathlib import Path
  >>> test = str(Path.home() / "test")
  >>> db = lmdb.open(test)
  Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
    File "/home/sage/py-lmdb/lmdb/cffi.py", line 776, in __init__
      raise _error(path, rc)
  lmdb.cffi.InvalidParameterError: b'/home/sage/test': b'Invalid argument'
  >>> Path("/home/sage/test").is_dir()
  True

[automorphis]

I did some more troubleshooting/research and I've localized the problem to a line in mdb.c (github.com). I am getting the error EINVAL from pthread_mutexattr_setpshared.

[jnwatson]

That's a limitation of cygwin. See https://www.mail-archive.com/cygwin@cygwin.com/msg167155.html

[automorphis]

Thank you! For what it's worth, I added the line

extra_compile_args += ['-DMDB_USE_POSIX_SEM']

to setup.py of py-lmdb and it seemed to resolve the issue. I only tested Transaction.get and .put, though. I'm fairly ignorant of mutex and semaphores.

Edited to add: The next day, I was not able to get get to work. put returned True, but get would throw MDB_CORRUPTED. I localized the error to this line in cmd.c. I don't know why it worked before and why it doesn't work now. I am just going to use WSL2 and leave Cygwin behind.