Consensus protocol implementation

Details

This is to ensure that local database files maintain their integrity. Raft can be used with cross-platform networking libraries such as libuv and libzmq for ordering and detecting duplications. This means you could use UDP for transmission.

Edit: According to ChatGPT, a p2p marketplace cannot function without any sort of consensus mechanism and the raft consensus protocol cannot scale to the millions, which is why I am proposing that we should choose a more scalable solution.

These are protocols that I am considering:

Sharding-based protocols - likely the most suitable for a decentralized P2P marketplace app that is intended to be scalable to billions of users, as they can significantly increase the network's throughput and transaction processing speed.
Hashgraph - does not require a blockchain and can handle hundreds of thousands of transactions per second and is suitable for applications that require high throughput and low latency.
PBFT - ensures all nodes in the network agree on the same order of transactions, making it well-suited for enterprise applications and private blockchain networks.
PoS - More decentralized and secure than DPoS and can scale in the thousands, but unfortunately, it does require a blockchain or a similar distributed ledger technology to function and may face challenges related to centralization if a small group of validators hold a large percentage of the cryptocurrency supply.

References

https://raft.github.io/ https://raft.github.io/raft.pdf https://thesecretlivesofdata.com/raft/ (ELI5) https://github.com/canonical/dqlite https://github.com/rqlite/rqlite/ https://developer.mozilla.org/en-US/docs/Learn/Server-side/First_steps/Client-Server_overview

Server-Client Communication System and Design

Once we get peer discovery rolling then we'll need to move on to server-client communications.

The neroshop daemon will automatically be ran (as a separate child process) on the client being launched by the user (This is for the sake of convenience. And convenience gives the app a sense of user-friendliness)

I need the daemon to act as a server. Let's call it a daemon-server. The name of the executable will be called neromon (nero + daemon). This is because "neroshopd" and "neroshop" can be confusing at times.

So, the purpose of the daemon-server, neromon is to wait for and accept requests from the client and translate those requests into SQLite statements that write to the database (e.g CREATE, ALTER, INSERT, UPDATE, DELETE, etc.) that will be placed in a queue and then executed once all peers reach a consensus. The daemon-server will return the result of the statement's execution. After the execution succeeds, all peers will have the same data within their local database files. The raft consensus protocol will have a huge part to play in this.

The server will have two threads: a consensus thread which will handle the database replication, and a peer thread which will handle traffic from peers.

The client will send a request message that uses a custom-made format, akin to the HTTP request format, which includes:

A Request-line

Zero or more header (General|Request|Entity) fields followed by CRLF

An empty line (i.e., a line with nothing preceding the CRLF) 
indicating the end of the header fields

Optionally a message-body

But we can decide to use the JSON-RPC format instead.

NOTE: I have zero idea of what I'm talking about. I don't know a thing about network programming or server/clients, so If I made any mistakes let me know.

Comments

anarkiocrypto:

"Database ideas:

Buyers/sellers: monero_address display_name user_signature

user_signature is required to avoid impersonation. If you set your address to 4abcdef... you need to sign this input with your 4abcdef... address.

Signing requires the mnemonic (i.e. it is done locally on the client and the mnemonic is not saved in the database). Verification requires the Monero address only (which is stored in the shared database). No view key required: https://monero.stackexchange.com/questions/6093/how-to-sign-a-message-with-my-public-monero-address-and-verify-it-using-the-cli

Any users/listings/orders with invalid signatures could be ignored by the software.

Listings: listing_guid seller_address product_name product_description product_price product_photo listing_signature

Orders: order_guid seller_address buyer_address listing_guid order_data order_status order_signature

order_data just contains the product name, price, payment address, etc.

For privacy, orders should be encrypted to the buyer's + seller's keys (any other nodes would just see encrypted data). Either seller_address + buyer_address are unencrypted (=metadata) or clients just try to decrypt all orders, to find out which belongs to them."

Scalability

We need the application to be as scalable as possible so that users will not have to use up too much memory/storage space storing data in their nodes. I will now explain the methods we can use to reduce database size.

We could purge orders after a certain frame of time has passed, say 2 years
Another method is pruning but I don't know too much about how pruning works

Bounty reward

layters / testshop

Consensus protocol implementation #12