search

iotaledger / iri

IOTA Reference Implementation
Other
1.15k stars 370 forks source link

Investigate how we can get rid of the PreviousEpochSpentAdresses files from IRI and eliminate the `WereAddressesSpentFrom` API call #1003

Open jakubcech opened 6 years ago

jakubcech commented 6 years ago

Description

We want to get rid of the PreviousEpochSpentAdresses files from IRI so we can eliminate the WereAddressesSpentFrom API call altogether.

We cannot remove this API call as it's a supported scenario by Trinity for example. We need to implement a different solution, e.g., a bloom or a cuckoo filter instead.

This will be a first iteration of resolving this issue, where we compress the file.

Motivation

The current solution with persisting an ever-growing file is not maintainable as the file can easily (and has already) grown to > 100MB and that hits a wall in GitHub and git-lfs complicates things for users. The current implementation was only put in place as a temporary fix for the stateless wallet. Likely not be needed anymore with Trinity.

We need to talk with the Trinity team on this proposal.

We'll need to analyze how this behaves. This can be a separate issue.

Requirements

gjeee commented 6 years ago

just wondering: suppose you reinstall trinity somewhere else (so losing its local state) and you connect to a full node which has been pruned/snapshotted recently, how can trinity determine where to start in the address space (so that it starts from a unspent address)?

jakubcech commented 6 years ago

Hi @gjeee, we'll actually look at implementing a filter first so that we don't disrupt something that Trinity relies on. :)

Thoralf-M commented 5 years ago

@gjeee Trinity could check the addresses for funds and just mark every address with a lower index as used