Adding persistence

[chrisdukakis]

Description

This issue deals with addition of persistence for storing critical data and increasing efficiency of address generation and transaction issuance.

Motivation

At the moment, persisting data for addresses generation, input selection and transaction monitoring is managed by end user. It would be best to abstract the complexity away and remove legacy code that attempts to solve problem by querying the network (see here & here).

Requirements

• [ ] Define an interface for storage adaption (https://github.com/iotaledger/iota.js/issues/297)
• [ ] Allow one client to connect to database or lock it if multiple clients are supported.
• [ ] Persist last key index which should be atomically incremented for each new address.
• [ ] Persist transactions before broadcasting to network. Persisted transactions can be cleared once they are confirmed. Transactions might not be recoverable after broadcasting, hence we need this feature. Transactions at rest could be represented by the bundle essence and optionally signatures or indexes.
• [ ] Deprecate use of network calls to generate addresses and select inputs, and perhaps index offset.

Open Questions

(Following applies to IOTA deployed in IoT and is not relevant to IOTA in it's current "Internet" state.)

Efficient transaction input selection should work on IoT devices with constrained resources and high throughput. In other words, as address list grows, device might be incapable to persist it and/or process it. Adding to that, one can't receive if doesn't spend, because network progresses by forgetting about the past;

"Iotas can disappear. If you haven’t been touching your wallet for a long time then there may be very few nodes remembering your balance left."

-Come-from-Beyond (https://medium.com/@comefrombeyond/economic-clustering-and-iota-d3a77388900)

Should we implement sweeps to reconstruct one portion of state at a time? Sweeps can trade increased time for decreased space; instead of storing all addresses with metadata, we fetch transactions/balances over variable search space, sweep any tokens to new addresses and await inclusion.

However one could target an address with incoming transaction and send another of lower value. We can't assume that address owner sees original transaction, and if second transaction is detected first, "wrong" sweep could be triggered. Rest of tokens may be replayed and/or stolen.

A different sweeping strategy could use spending before depositing. Sweep signals the desired deposit amount. One sweep output in a new bundle can be used for funding more transfers immediately. Outputs of distinct sweeps can be mixed in one bundle only if all sweeps are included. This strategy only requires to store pending transactions. All work is done over an async (Promise) Buffer which is solely populated by key index stream per deposit request. Setting a buffer length regulates the flow of deposits and withdrawals. When we clear pending transactions we get room for new deposits. Addresses, hashes of reattachments, inclusion state and balances can be mapped/reduced over transactions buffer.

Note: Brainstorming on open questions is encouraged, but development will focus on list of requirements for now.

[obany]

Closing as this relates to an unmaintained branch of the library, which will be incompatible with the new Chrysalis release of the network.