aigents / aigents-java

Aigents Java Core Platform
MIT License
30 stars 12 forks source link

Aigents Cloud Storage #31

Open akolonin opened 4 years ago

akolonin commented 4 years ago

For the high-performance and high-capacity Aigents Servers supporting thousands and millions of users, we would need to change the existing storage design of the Aigents. Currently, it involves: A) in-memory custom graph DB https://github.com/aigents/aigents-java/blob/master/src/main/java/net/webstructor/core/Storager.java (stored in al.txt snapshots) B) "temporal graphs" for indexing source-specific historical graph data https://github.com/aigents/aigents-java/blob/master/src/main/java/net/webstructor/data/GraphCacher.java
C) "long-term memory" storage of the object instances https://github.com/aigents/aigents-java/blob/master/src/main/java/net/webstructor/core/Archiver.java D) cache of the web data https://github.com/aigents/aigents-java/blob/master/src/main/java/net/webstructor/self/Cacher.java While the above work fine for single-user Aigents instances and up to few-hundred-user instances, it may not scale well if we get thousands and millions of concurrent Line Clients (such as per #25 , #26 #27 and #28 ) so the following would have to get done:
1) Redesign and refactor the above so we use interfaces instead of classes and the implementations of those interfaces are served by factories obtained at the Body level https://github.com/aigents/aigents-java/blob/master/src/main/java/net/webstructor/agent/Body.java (see getPublisher singleton factory for instance)
2) Choose the Graph/SQL/Object DB engine for alternative implementation of these interfaces (like Neo4J/PostgreSQL/MongoDB) 3) Have the job done :-)