Maybe: use implementation-specific synchronized hash-table features where possible

[appleby]

The qvm-app-ng package contains a global hash table for storing persistent QVMs in qvm-app-ng::*persistent-qvms*. Concurrent access to the table is protected with a global lock. Many common lisp implementations either support thread-safe concurrent readers/writers by default, or else by specifying a :synchronized keyword option to make-hash-table. Probably these implementation-specific extensions use finer-grained locking and would result in faster concurrent access than using a single global lock.

Potentially this code for thread-safe access to a global hash-table could be pushed down into a sub-package that might also be useful for the job management portions of the API.

This will probably remain low-priority until and unless the global hash-table lock turns out to be performance bottleneck in practice, which seems unlikely given the nature of how qvm-app is used.

Links to implementation-specific docs

SBCL & ECL support a :synchronized keyword:

• http://www.sbcl.org/manual/#Hash-Table-Extensions
• https://common-lisp.net/project/ecl/static/ecldoc/Standards.html#Hash-tables

CCL claims (nearly) lock-free thread-safe access is the default:

• https://trac.clozure.com/ccl/wiki/Internals/LockFreeHashTables

LispWorks supports atomic read/write with other extension when you need atomicity across multiple accesses:

• http://www.lispworks.com/documentation/lw70/LW/html/lw-138.htm#12543
• http://www.lispworks.com/documentation/lw70/LW/html/lw-575.htm#77199
• http://www.lispworks.com/documentation/lw70/LW/html/lw-136.htm

[phoe]

Have you ever implemented this synchronization for multiple CL implementations? I ask, because I could use a separate TRIVIAL-SYNCHRONIZED-HASH-TABLE system that provides a uniform API for accessing a hash table from multiple threads.

[appleby]

No, nothing like that was ever implemented, only a thin wrapper over SBCL's synchronized hash-tables that falls back to bordeaux-threads and a global lock on all other implementations. The wrapper only provides functions actually needed by the parent qvm-app-ng package, and is missing things like MAPHASH, HASH-TABLE-SIZE, etc., and is therefore not widely useful outside of qvm-app-ng (and is even its usefulness within qvm-app-ng is questionable!).

stylewarning has elsewhere suggested the possibility of writing our own portable hash-table implementation including features like synchronization and custom hash functions, but so far that has not been done.