This document is not complete and can change over time.

File system abstraction

Purpose

Persistent data storage is a common problem that can be solved in many ways. However, each available solution has its own drawbacks, and any of them doesn't meet all our requirements that we can imagine.

Requirements

Independent of the transport layer.
Identifies data using predictable id.
Extensible.
Control redundancy of files.
Scalable across physical devices

Layers

The file system abstraction can be represented as a unidirectional(?) layers or graph of cooperating services.

Store

The lowest level of data persistence abstraction. Key-value store for hash-based IDs associated with blob.

Requirements

Id is predictable from the data which points to. So the same data will always generate the same unique Id.
Id requirement is implying data immutability. So is not possible to modify existing data without modifying the id.
The storage interface allows only for basic operations, the domain-specific functions will be delivered on higher layers.

API
```
typealias Id = String
```

interface Store { suspend operator fun plus(data: T): Id operator fun get(id: Id): T operator fun minus(id: Id): Boolean operator fun contains(id: Id): Boolean fun set(): Set fun additions(): Flow }

### Data

#### ID
Is a summary result of sha256 and the size of the given data.
`id(D) = sha256(D) + size(D); where D is any data`

#### Blob
On this level of abstraction, it can be any kind of data that can be stored on a hard drive.

## Graph

The abstraction, responsible for storing and indexing relations between blobs and stories.

### Requirements
* Graph is an independent service that knows Store API
* Graph can recognize the special type of blobs called Stories.
* Graph is indexing relations for each recognized story added to the store.
* Graph can serve information about relations between blobs.

### API

typealias Type = String

interface Graph { fun source(id: Id): Set fun target(id: Id): Set fun type(name: Type): Set }


### Data

#### Story
It's a unified kind of blob that can describe another blob.
The story should contain, version id, type name, set of relations plus optional body, the key-value map.

data class Story( val ver: Int = 0, val type: Type = "", val rel: Set = emptySet(), val body: Map<String, Any> = emptyMap() )



## Push service

Gateway to the file system, responsible for handling upcoming messages outside of the file system.
... TODO

cryptopunkscc / crypton