replikativ / konserve

A clojuresque key-value/document store protocol with core.async.
Eclipse Public License 1.0
299 stars 25 forks source link

java.nio.file.NoSuchFileException due to .nfs* files? #67

Closed pmspire closed 1 year ago

pmspire commented 2 years ago

I am using fs-store on an NFS volume, and am getting (partial) stacktraces similar to

java.nio.file.NoSuchFileException: /nfs/.nfsbe549e9e950952130000002b
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:177)
at java.nio.channels.FileChannel.open(FileChannel.java:287)
at java.nio.channels.FileChannel.open(FileChannel.java:335)
at konserve.filestore$migrate_file_v1.invokeStatic(filestore.clj:409)
at konserve.filestore$migrate_file_v1.invoke(filestore.clj:398)
at konserve.filestore$migrate_in_list_keys.invokeStatic(filestore.clj:552)
at konserve.filestore$migrate_in_list_keys.invoke(filestore.clj:550)
at konserve.impl.default$list_keys.invokeStatic(default.cljc:229)
at konserve.impl.default$list_keys.invoke(default.cljc:225)
at konserve.impl.default.DefaultStore._keys(default.cljc:426)
at konserve.core$keys.invokeStatic(core.cljc:281)
at konserve.core$keys.invoke(core.cljc:275)
<my application code>

In filestore.clj, it looks like list-files does no filtering, unlike for example count-konserve-keys, which filters for .ksv files. I wonder if list-files may be seeing temporary .nfs* files (in case you're not familiar: http://nfs.sourceforge.net/#faq_d2) that, by the time they are later referenced, no longer exist; and whether perhaps filtering out files konserve should not care about (if that's possible to define) earlier might avoid this.

I am working with 0.6.0-SNAPSHOT due to issue #60 opened by my colleague. (Specifically, 0.6.0-20211004.084316-6 -- I tried the latest 0.6.0-SNAPSHOT and got clojure.lang.Compiler$CompilerException: java.lang.RuntimeException: Unable to resolve symbol: key-vec in this context, compiling:(konserve/filestore.clj:640:25), just FYI.)

Thanks in advance for any thoughts you might have.

whilo commented 2 years ago

Hey @pmspire, thanks for reporting! Yes that makes sense. Can you open a PR that fixes the issue for you? I will make sure it is merged quickly.

pmspire commented 2 years ago

Will do ASAP, thats @whilo .

pmspire commented 2 years ago

I have a provisional fix, but am so far unable to test it, because

  1. I am still getting the CompilerException mentioned above, which I do not understand because key-vec is let-bound on the previous line. I tried replacing key-vec with nil in the problematic expression, but then get
  2. clojure.lang.Compiler$CompilerException: java.lang.IllegalAccessError: new-fs-store does not exist, which seems to be true -- new-fs-store is defined in filestore.clj on master, but not on development.

I would be happy to open a PR with my untested code, or to try any suggestions you might have.

whilo commented 2 years ago

Sure, feel free to open a PR, I just don't have nfs around atm.

whilo commented 1 year ago

Should be closed with the PR.