Explore Vineyard

[jakirkham]

Another shared memory store is Vineyard, which came up in this discussion ( https://github.com/dask/distributed/pull/6503#issuecomment-1223418006 ). It may be worth adding this to the comparison here.

cc @sighingnow

[martindurant]

Yep, definitely in the plan. The first time I installed vineyard, something got messed up, but seems to be working fine now. I should be able to make a naive implementation pretty easily. As with lmdb, doubtless someone better versed can tell be what config and access pattern is actually recommended :)

[sighingnow]

@jakirkham Thank you for adding vineyard into the plan. I'm composing the vineyard PR and will add it to this repo soon.

[sighingnow]

As with lmdb, doubtless someone better versed can tell be what config and access pattern is actually recommended :)

I also failed to installed the python-lmdb package in the requirements.txt.

  Using cached python-lmdb-1.0.0b1.tar.gz (8.1 kB)
    ERROR: Command errored out with exit status 1:
     command: /usr/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-p2pnc__v/python-lmdb/setup.py'"'"'; __file__='"'"'/tmp/pip-install-p2pnc__v/python-lmdb/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-install-p2pnc__v/python-lmdb/pip-egg-info
         cwd: /tmp/pip-install-p2pnc__v/python-lmdb/
    Complete output (5 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-p2pnc__v/python-lmdb/setup.py", line 5, in <module>
        with open("README.md") as f:
    FileNotFoundError: [Errno 2] No such file or directory: 'README.md'
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

[jakirkham]

Closing since this has been added