MDB_BAD_RSLOT after fork due to spare txn

Affected Operating Systems

Linux

Affected py-lmdb Version

1.4.0

py-lmdb Installation Method

sudo pip install lmdb==1.4.0

Also built from source to enable debugging.

Using bundled or distribution-provided LMDB library?

Bundled

Distribution name and LMDB library version

(0, 9, 29)

Describe Your Problem

Hi,

Using LMDB in a program that forks but which does not use LMBD inside the fork is giving me the following error

Traceback (most recent call last):
  File "/Servers/bcde/lmdb_test.py", line 25, in <module>
    txn = env.begin(write=False)
lmdb.BadRslotError: mdb_txn_renew: MDB_BAD_RSLOT: Invalid reuse of reader locktable slot

I've written the following testcase to reproduce the issue

import lmdb
import os
import time

print("      python: Opening environment")
env = lmdb.Environment('db.lmdb')
print("      python: Opened environment %r" % env)
print()

print("      python: Starting transaction")
txn = env.begin(write=False)
print("      python: Started transaction %r" % txn)
print()

print("      python: Deleting transaction")
del txn
print("      python: Deleted transaction")
print()

if (os.fork() != 0):
    os.wait()
    print("      python: Exited forked process")
    print()
    print("      python: Starting transaction (no. 2)")
    txn = env.begin(write=False)
    print("      python: Started transaction (no. 2) %r" % txn)
else:
    print("      python: Inside forked process")

On 1.4.0 this is giving me the following output

python: Opening environment
lmdb.cpython: env_new:1301: mdb_env_open(0x561033edf610, 'db.lmdb', 2097152, 644);
lmdb.cpython: db_from_name:981: DbObject '(null)' opened at 0x7f3b9fe5ce90
lmdb.cpython: env_new:1311: EnvObject 'db.lmdb' opened at 0x7f3b9fe4cc30
      python: Opened environment <Environment object at 0x7f3b9fe4cc30>

      python: Starting transaction
lmdb.cpython: make_trans:829: make_trans(env=0x7f3b9fe4cc30, parent=(nil), write=0, buffers=0)
      python: Started transaction <Transaction object at 0x7f3bae671ab0>

      python: Deleting transaction
lmdb.cpython: trans_dealloc:3242: 0x7f3bae671ab0: caching trans
lmdb.cpython: trans_dealloc:3248: 0x7f3bae671ab0: deleting trans
lmdb.cpython: trans_clear:3208: 0x7f3bae671ab0: clearing trans
lmdb.cpython: trans_clear:3218: 0x7f3bae671ab0: db is/was 0x7f3b9fe5ce90
      python: Deleted transaction

      python: Inside forked process
lmdb.cpython: env_clear:1118: 0x7f3b9fe4cc30: env_clear
lmdb.cpython: invalidate:340: invalidating parent=0x7f3b9fe4cc30 child 0x7f3b9fe5ce90
lmdb.cpython: env_clear:1126: 0x7f3b9fe4cc30: killing spare txn 0x561033f635b0
lmdb.cpython: txn_abort:3199: 0x561033f635b0: aborting
lmdb.cpython: env_clear:1132: Closing env
      python: Exited forked process

      python: Starting transaction (no. 2)
lmdb.cpython: make_trans:829: make_trans(env=0x7f3b9fe4cc30, parent=(nil), write=0, buffers=0)
lmdb.cpython: make_trans:859: using cached txn
Traceback (most recent call last):
  File "/Servers/bcde/lmdb_test.py", line 25, in <module>
    txn = env.begin(write=False)
lmdb.BadRslotError: mdb_txn_renew: MDB_BAD_RSLOT: Invalid reuse of reader locktable slot
lmdb.cpython: env_clear:1118: 0x7f3b9fe4cc30: env_clear
lmdb.cpython: invalidate:340: invalidating parent=0x7f3b9fe4cc30 child 0x7f3b9fe5ce90
lmdb.cpython: env_clear:1132: Closing env

If I comment out caching the transaction in trans_dealloc I get

      python: Opening environment
lmdb.cpython: env_new:1301: mdb_env_open(0x558d9790a610, 'db.lmdb', 2097152, 644);
lmdb.cpython: db_from_name:981: DbObject '(null)' opened at 0x7fc25e690da0
lmdb.cpython: env_new:1311: EnvObject 'db.lmdb' opened at 0x7fc25e680c30
      python: Opened environment <Environment object at 0x7fc25e680c30>

      python: Starting transaction
lmdb.cpython: make_trans:829: make_trans(env=0x7fc25e680c30, parent=(nil), write=0, buffers=0)
      python: Started transaction <Transaction object at 0x7fc26cea5ab0>

      python: Deleting transaction
lmdb.cpython: trans_dealloc:3248: 0x7fc26cea5ab0: deleting trans
lmdb.cpython: trans_clear:3208: 0x7fc26cea5ab0: clearing trans
lmdb.cpython: txn_abort:3199: 0x558d9798e5b0: aborting
lmdb.cpython: trans_clear:3218: 0x7fc26cea5ab0: db is/was 0x7fc25e690da0
      python: Deleted transaction

      python: Inside forked process
lmdb.cpython: env_clear:1118: 0x7fc25e680c30: env_clear
lmdb.cpython: invalidate:340: invalidating parent=0x7fc25e680c30 child 0x7fc25e690da0
lmdb.cpython: env_clear:1132: Closing env
      python: Exited forked process

      python: Starting transaction (no. 2)
lmdb.cpython: make_trans:829: make_trans(env=0x7fc25e680c30, parent=(nil), write=0, buffers=0)
      python: Started transaction (no. 2) <Transaction object at 0x7fc26cea5ab0>
lmdb.cpython: trans_dealloc:3248: 0x7fc26cea5ab0: deleting trans
lmdb.cpython: trans_clear:3208: 0x7fc26cea5ab0: clearing trans
lmdb.cpython: txn_abort:3199: 0x558d97928300: aborting
lmdb.cpython: trans_clear:3218: 0x7fc26cea5ab0: db is/was 0x7fc25e690da0
lmdb.cpython: env_clear:1118: 0x7fc25e680c30: env_clear
lmdb.cpython: invalidate:340: invalidating parent=0x7fc25e680c30 child 0x7fc25e690da0
lmdb.cpython: env_clear:1132: Closing env

As I'm not actively using LMDB inside the forked process I believe that this should be something that LMDB supports but maybe I need to change my code to explicitly close the environment before forking.

jnwatson / py-lmdb