search

opencog / atomspace

The OpenCog (hyper-)graph database and graph rewriting system
https://wiki.opencog.org/w/AtomSpace
Other
798 stars 225 forks source link

Implement incoming-set caching. #1373

Open linas opened 6 years ago

linas commented 6 years ago

Implement incoming-set caching. When an atomspace is too large to fit into RAM, we must leave some of it on disk. However, it can become important to make sure that the incoming set of an atom has been loaded into RAM i.e. into the atomspace. Currently, repeatedly loading the incoming set is very slow and wasteful, and so its important to know whether an incoming set has already been loaded or not. Technically, this caching can be done in the atomspace, I guess...

akolonin commented 6 years ago

Linas, if there is an estimation to which extent conventional OS swapping is not helpful and to which extent it could be done better than that? And yes, if doing this at all, it should be done in Atomspace so Atomspace would perform like any DB, like Allegro Cache. Note, that persistent DBs with such capacity all are 10-100 times slower than pure in-memory DB because of overhead on guarding access to items that could be cached-in or cached-out. I mean, if it is done, the performance may drop down below the watermark of practical usability.

linas commented 6 years ago

Yes. What can I say. I worked as an operating system kernel programmer for 5-10 years. I ported the Linux kernel to the IBM mainframe. I spent several years doing performance measurement and optimization on a web+database system. I'm very aware of the issues.

In 2018-2022 if you want to get good performance, use SSD drives attached via NVMe to PCIEv3. Its expensive - last I looked, it was $800 per terabyte. Getting a system planar with more than 2 or 3 PCIEv3 x4 connectors is also expensive and hard to find.

A much cheaper practical alternative is to use SATA2 or SATA3-attached SSD disks, but these are maybe 4x slower or more. (I don't quite recall, but SATA is about like one PCIE lane). Use spinning disks for backup only, not for live data.

linas commented 6 years ago

Anyway, the atomspace is already painfully slow, It would be nice to totally redesign it from scratch for scalability and etc. but this is a massively huge undertaking. Its also an undertaking that will almost surely end in failure unless you have top-class system architects work on it. We have to live with what it is, for now.

Anyway, the atomspace is thread-safe; it has some locks; the atoms are immutable, and the atoms and values are references with smart pointers, which means that you can use your favorite lock-free programming technique. https://en.wikipedia.org/wiki/Software_transactional_memory Of course, that is much easier said than done.

The biggest performance problem is that data access in the atomspace is insanely random, so you are almost guaranteed to have cache misses, even third-level cache misses almost all the time. Which means that all algorithms run at the speed of RAM instead of th speed (and bandwidth) of cache. Note that modern Intel and AMD CPU's have a bandwidth to RAM that is 10x less or 20x less than bandwidth and latency to cache. The current atomspace is a poor fit for modern CPU's.

linas commented 1 year ago

Current idea is that this can be accomplished with a new kind of ProxyNode e.g. a CachingProxyNode or variants thereof. The ProxyNodes allow general transformations and pipelines to be applied to AtomSpace data as it moves around.