search

tomp2p / TomP2P

A P2P-based high performance key-value pair storage library
http://tomp2p.net
Apache License 2.0
441 stars 123 forks source link

About P2P network restart #114

Closed qing3gan closed 9 years ago

qing3gan commented 9 years ago

I used Android to build a P2P network and upload some data, but when the network restart these data will disappear? If yes, how to recover these data?

nicoruti commented 9 years ago

The default Storage implementation is volatile (StorageMemory); If you stop a peer, it looses all data. Therefore, if you stop all peers, all data will be gone. You can use a persistent implementation of the storage like StorageDisk or write an own one.

The tricky part with persistent storage is the validity of the data: Data could have been updated / deleted during the absence of a peer; How does the peer detect and handle this?

qing3gan commented 9 years ago

1 that is to say, there must be a Peer as a server to maintain the data in the network. In the case of network restart, the TomP2P does not have a mechanism for handling data changes and recovery. 2 If the data is not changed, then there is possibility of restoring data in the case of the network restart?

nicoruti commented 9 years ago

1 that is to say, there must be a Peer as a server to maintain the data in the network. In the case of network restart, the TomP2P does not have a mechanism for handling data changes and recovery.

Yes, or have a large user base such that the chance that one of them is up is very high. But for bootstrapping purpose, you usually have stable peers in the network anyway.

2 If the data is not changed, then there is possibility of restoring data in the case of the network restart?

At least that's what I would expect. I never really tried the disk-based storage.

BTW: Just as another input: A graph-database like neo4j would fit perfectly for persistent storage when using the versioned DHT approach (vDHT). Each Number640 key would be a node, connected to the previous revision. This makes it easy to detect conflicts (forks). @tbocek would probably be a cool extension / student project.

qing3gan commented 9 years ago

I modified the Hive2Hive source code and combined with StorageDisk to achieve the disk storage (on the disk can be seen in chunks and tomp2p, tomp2p.p file). TomP2P uses the Mapdb library to achieve the disk storage, my idea is the first time connected peers of the network will save user data locally(such as UserProfile), and in the case of network restart, the peer can be considered as the second connected to the network(the same network as the first connected), then the peer can recover the data by reading the local user data, so that the network can be restored to the first time.But I don't know if it is feasible, do you have any good suggestions?

tbocek commented 9 years ago

For the user profile, in addition to MapDB, I would store the UserProfile separately on disk, and push it to the network whenever the peer responsible for that UserProfile comes online.

I'm closing this issue, if you have further questions, feel free to repopen.