maidsafe-archive / MaidSafe

This is the super-project in which each MaidSafe library resides. Some information is common to all libraries, and is detailed here. Library-specific information can be found in each library's wiki.
Other
583 stars 102 forks source link

API Abstraction Level #112

Open xtopherbrandt opened 10 years ago

xtopherbrandt commented 10 years ago

HI guys,

This is an design and integration question, not an issue. If there's a better place to post and discuss, let me know.

I'm attempting to design an application that will run on top of Maidsafe. However I'm having some trouble understanding how to best store my app's data.

My main question is, what is the level of abstraction of the Maidsafe storage system? I know that underneath it's a DHT, but I assume there is some abstraction of that through your APIs (which I havn't had time to really dive into). The same with encryption; I know that underneath your self-encrypting the data and scattering it. How much do I need to worry about my data's encryption?

For example, say my application stores a user's profile information (as it relates to my app) and their interaction history. So I've got both resource data (user profiles) and transaction data (interaction history). Should I be thinking about storing my records as blobs to a DHT, basically as key value pairs? Do I need to encrypt the blobs? Assuming that many users are on the system, what's the mechanism or pattern to ensure that each user can only see their own data?

I'd be happy with any kind of answer I can get on this. If you'd like to discuss here or If there are documents explaining this somewhere, please point me to them. Also point me to good code examples that might help me understand the abstraction layer and how to use it.

Thanks.

dirvine commented 10 years ago

No problem Chris. it's a question for the mailing list really but here goes.

The network works like this (the API is in the middle of an API war just now, so will change a little in the next 2-3 weeks).

There are vaults (an application that runs and server routing/disk space etc. to the network), you need to have one of these and it's private key to be able to write anything to the net. There are tests that bring a whole network up and store/delete etc. on it (Mahmoud can talk you through that).

Then the NFS library that connects client to the network. A client must send a register pmid token to the network. You can see this in the tests (client part). After that the network gets space available from the pmid (the vault) and the client can put/store/delete etc. (very soon, message, email etc.)

The we have full blown client library which does this (this is the main API change, good to watch this closely as well alter the API now). This provides an API for a gui driven application (of any type).

Then there is Drive, this is again another application that connects to the network, mounts a cross platform virtual drive and presents the drive to users as a usb type drive. This component encrypts/decrypts the data and does the network Put/Del/Get actions with data.

A Drive can have many identities, so you can have a private data drive, a public data drive (aka the decentralised web) and a private shared drive.

This is something we need to document much more clearly and we will as the API completes.

The best mechanism at the moment is to be on the dev list and let us know what you are doing and we will guide you through. This helps our docs and also gives another view on the API that works best.

We like debate and do so a lot, as long as it's logic we listen, so please feel free to dive in and join us. It's always good to be first to the table and you are right in at launch time so will learn and help an awful lot.

A good thing to do is use our next branches of the libraries and check soe of the tests in Vaults/Routing/Drive for now. The Routing tests can be very scary, but they are incredibly thorough as our DHT needs to be extremely fast and importantly correct at all times, even under incredible churn rates.

I hope this helps.

mmoadeli commented 10 years ago

Hi Chris,

Regarding the Vault tests which David mentioned above, you may find tests at the highest level of abstraction in vault_network_tests.cc file. The tests are under development, however, they can easily provide some basic insight into the expectation one may have from a vault network, how a client or vault can join and, in particular basic Put/Get/Delete operations. The tests are built on top of a network which is created using VaultNetwork class, in vault_network.h and vault_network.cc files. It is important to note that the network setup in the tests is not completely similar to the real network setup. In particular, the way keys are handled/distributed is totally customised for the local test.

Should further clarifications be required, please do not hesitate to ask