grantjenks / python-diskcache

Python disk-backed cache (Django-compatible). Faster than Redis and Memcached. Pure-Python.
http://www.grantjenks.com/docs/diskcache/
Other
2.4k stars 134 forks source link

Use local OS' path separator #304

Open csm10495 opened 9 months ago

csm10495 commented 9 months ago

I use diskcache and save the cache directory to a share drive. Later on, I use diskcache again to pull / update the data.

The problem I recently hit was when I added a Linux host to my previously-all-Windows pool:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/diskcache/core.py", line 1173, in get
    value = self._disk.fetch(mode, filename, db_value, read)
  File "/usr/local/lib/python3.10/site-packages/diskcache/core.py", line 281, in fetch
    with open(op.join(self._directory, filename), 'rb') as reader:
OSError: [Errno 22] Invalid argument: '/mnt/z-drive/Temp/shared/csmutil.cache/32\\25\\8a75134ac891498afaeb4efa3ec7.val'

It seems like diskcache saves a path inside the db based off the OS' path separator., which leads to this error ^. Can the path be normalized to the local-OS' path seperater?

csm10495 commented 9 months ago

(One other side note: it was a bit of a pain to find this error, since the default value was getting used since it caught the OSError). Maybe the caught error could be more granular in the future :)

csm10495 commented 9 months ago

Lots of possible ways to allow this:

csm10495 commented 9 months ago

For anyone out there who hits this, you can resolve via:

<diskcache_obj>._sql('UPDATE Cache SET filename = REPLACE(filename, "\\", "/") WHERE filename is not NULL').fetchall()

I run this every time i get a cache object just in case.