"invoice" data type and access rules

[michielbdejong]

Let's say an invoice consists roughly of:

• seller identity
• buyer identity
• invoice identity
• items with product id, quantity, gross unit price, tax percentage
• extras and discounts
• shipping details
• payment terms

And let's model how two parties negotiate a deal. We could define the following operations:

Create

Starting with just seller and buyer identity and a unique identifier of this particular invoice

Add-to-cart

Say the buyer starts with adding items to it (at this point it's really still like a shopping cart on an e-commerce website), with just product id and quantity

Remove-from-cart

The buyer can remove from cart

Edit-quantity

The buyer can edit the quantity on an item

Item-out-of-stock

The seller can lower a quantity on one of the items

Quote-item

The seller sets the gross unit price and tax rate for one of the items

Bid-item

The buyer proposes a gross unit price and tax rate for one of the items. The seller can accept the bid by doing a Quote-Item operation with the same gross unit price and tax rate.

Add-extra

Buyer and seller

Remove-extra

Seller can always remove extras. Buyer can only remove it if they were the one to add it

Add-discount

Seller (buyer may request a discount, possibly with a discount token or code, but let's leave that out of band for now)

Remove-discount

Seller

Propose-delivery-date

The seller proposes a delivery date (earliest option given the extras).

Request-postponed-delivery

The buyer can request to postpone delivery, the seller can accept the request by doing a new Propose-delivery-date operation with the same date.

Choose delivery address

Buyer

Set payment terms

Seller

[michielbdejong]

cc @gsvarovsky

[michielbdejong]

Ah, and a status field. It starts in 'draft' status. The buyer can set the status from 'draft' to 'buyer proposes', and the seller from 'draft' to 'seller proposes'. The buyer can update it from 'seller proposes' to 'agreed', and the seller from 'buyer proposes' to 'agreed'. From 'agreed' the seller can set it to 'shipped', to 'paid', and via those or directly, to 'shipped and paid'. From 'shipped' or 'shipped and paid', the buyer can set it to 'received' or 'received and paid', respectively. If something goes wrong (e.g. the payment is late or the goods are damaged), from any state, the buyer can set the state to 'buyer-disagrees' and the seller can set it to 'seller-disagrees', and then buyer and seller can try to use some out-of-band means to settle the dispute.

[michielbdejong]

Both seller and buyer keep a list of operations they did, and of operations the other party did. And this is where it gets interesting... are those operations commutative like in a CRDT?

[michielbdejong]

OK, I think for instance item quantity can be made CRDT by saying the seller sets a max-quantity, and the buyer sets a desired-quantity; the actual item quantity is then min(desired, max). Adding / removing an invoice item is a noop if the item quantity is zero. No two invoice items can have the same value for 'product id'.

For the other parts of the data type we can probably do something similar.

[michielbdejong]

For instance instead of editing the unit price, the buyer appends bids, and the seller appends quotes. The buyer's bids are just informational; from the seller's quotes, the lowest one is used. If the seller made a mistake, they need to set the invoice status to 'seller-disagrees' and create a new invoice with a new unique invoice id.

[gsvarovsky]

There's no problem with canceling an existing quote so long as the cancelation is captured in the data (a tombstone), and with causal delivery it can even be removed.

EDIT: actually I see better what you're proposing. "Appending" bids and quotes is not adding to the state, these are operations (that might happen to be journaled by someone). So your proposed data type resolves conflicts by using the min price – it can't go up because that would break an acceptance, but it can keep going down if the seller insists.

[gsvarovsky]

I think the authorisation scheme here is fine, as long as

• actors are identified, obviously
• operations are signed
• access control ACL is a separate data set (actually, this one can be overcome)

In principle the operations can be checked against the ACL by any participant; misbehaviour simply results in that operation being ignored and probably the whole interaction being thrown out.

[michielbdejong]

Right! If we make the assumption that there is exactly 1 buyer and 1 seller involved in updating the invoices, then authentication can be done by the transport layer, so a Buyer implementation will only accept seller actions from the seller identified in the Seller Identity details of this invoice, and vice versa.

[michielbdejong]

We would just use CRDT theory to prove that any combination of seller actions and buyer actions will lead to consensus and common knowledge of the invoice in its final state.

[michielbdejong]

Ah, but after thinking about it some more, just making the negotiation between buyer and seller work in a provably convergent way between mutually distrustful agents is not ambitious enough. :) First of all, we want both the seller and the buyer to potentially have multiple devices, and not only sync changes seller -> buyer and buyer -> seller, but also seller device x -> seller device y and buyer device x -> buyer device y.

And there may be other players apart from buyer and seller, like a tax authority for instance, who have no edit rights or maybe just 'comment' rights. Or a parcel delivery company who can edit the track&trace details of the shipment, but not edit the price etc.

And there may be internal departments (shipping, stock, production, sales, ...) on the buyer and seller side, for instance an agent that only has permission to edit stock levels, or to mark an order as shipped. Probably, the two ERP systems already have elaborate access controls set up for that? These could be federated as well.

OK, thinking about all these aspects, I think there is a lot of potential value to unlock here!

[gsvarovsky]

Some intuitions and terminology OCD at this stage, related to Conflict-free Replicated Data Types.

CRDTs could also be named "coordination-free". The data in each replica is independent. They are not able (by themselves) to provide consensus or any kind of final state. They are only able to guarantee that the data converges if everyone stops dancing and the network transmits everything successfully. But that's purely hypothetical, because in order to know that everyone has stopped dancing, you need some external coordination. So:

1. State transitions ("draft", "seller-disagrees") are possible, but require that any participant could still be in a prior state and sending operations that are only allowed in that state. Intuitively, I think this will be too complex to model successfully as a pure CRDT (I could be wrong).
2. Some of the state transitions trigger some other real-world actions, like delivery. By the argument above, these absolutely require some coordination (by definition, outside of the CRDT).

So, I think that it may be interesting to try modelling this as an interaction of two decentralised constructs:

1. A CRDT for the invoice, in which every interested party is able to make changes (some of which may be negotiated out of band). These changes are subject to access control but generally allow any authorised change at any time. This does not mean that the changes are untraceable – just that they are modelled as generally disordered.
2. A ledger for making decisions. This contains the major state transitions which represent agreements among the parties. Once a transition has happened, CRDT operations pertaining to the previous state are disregarded (this could even be implemented as a change of document identity). I think it would be good to try and keep the use of the ledger to a minimum, but I also think it's necessary.

However! I admire your ambition to model the whole process as an ongoing distributed negotiation, so these thoughts may just be implementation details.

[michielbdejong]

Very good point, it's fundamentally impossible to use a CRDT to get to a final agreement state and trigger shipping. Let's discuss in our video call today!

[michielbdejong]

Exploring this in https://github.com/pondersource/collaborative-invoice-composition/pull/1

[michielbdejong]

We would just use CRDT theory to prove that any combination of seller actions and buyer actions will lead to consensus and common knowledge of the invoice in its final state.

That's probably impossible due to the Two Generals Problem but when they continue to collaborate, there can probably be something like eventual common knowledge.