search

rob-smallshire / keeper

A filesystem based value store for Python 3.3 and above
MIT License
2 stars 1 forks source link

filestorage gets blocked by 'temp' file #1

Open abingham opened 4 years ago

abingham commented 4 years ago

It seems that sometimes keeper leaves a 'temp' file around on accident. When it does, subsequent attempts to construct FileStorage instances results in a FileExistsError:

  File "/Users/abingham/.virtualenvs/acquisition-system/lib/python3.8/site-packages/keeper/keeper.py", line 205, in __init__
    self._storage = filestorage.FileStorage(dirpath)
  File "/Users/abingham/.virtualenvs/acquisition-system/lib/python3.8/site-packages/keeper/storage/filestorage.py", line 30, in __init__
    os.mkdir(self._temp_root_path)
FileExistsError: [Errno 17] File exists: '/Users/abingham/sandbox/acquire/keeper/temp'
rob-smallshire commented 4 years ago

I've pushed keeper 0.96.3 to PyPI. If that doesn't fix it, please report back here.

abingham commented 4 years ago

I'm seeing the same issue, with a slightly different traceback:

  File "/Users/abingham/.virtualenvs/acquisition-system/lib/python3.8/site-packages/keeper/storage/filestorage.py", line 36, in __init__
    self._ensure_directory_exists(self._temp_root_path)
  File "/Users/abingham/.virtualenvs/acquisition-system/lib/python3.8/site-packages/keeper/storage/filestorage.py", line 55, in _ensure_directory_exists
    os.mkdir(dirpath)
FileExistsError: [Errno 17] File exists: '/Users/abingham/sandbox/acquire/keeper/temp'

FWIW, I've only seen this when processing stereo images, so maybe there's some race-condition when writing two things to the storage.

rob-smallshire commented 4 years ago

Try 0.96.4.

abingham commented 4 years ago

It seems more stable, but I'm getting a new traceback:

  File "/Users/abingham/.virtualenvs/acquisition-system/lib/python3.8/site-packages/keeper/keeper.py", line 131, in __exit__
    self.close()
  File "/Users/abingham/.virtualenvs/acquisition-system/lib/python3.8/site-packages/keeper/keeper.py", line 169, in close
    with self._reopen(mode='rb', encoding=None) as self._file:
  File "/Users/abingham/.virtualenvs/acquisition-system/lib/python3.8/site-packages/keeper/keeper.py", line 160, in _reopen
    return open(name, mode=mode, encoding=encoding)
FileNotFoundError: [Errno 2] No such file or directory: '/Users/abingham/sandbox/acquire/keeper/temp/1f1aa273-3662-4f8f-8ab4-44ffac821ebf'
abingham commented 4 years ago

I haven't been able to reproduce this most recent error. If/when it pops up again I'll try to capture the full traceback. Go ahead and close this if you want...it's working well enough now.