[Improvement propsal] Adding matcher to protocol

The basic idea of changes to protocol is that nodes don't add orders of other nodes to their local orderbook. Instead there is matchers which run orderbooks and match orders. This solves the problem about if many takers try to take same maker in same time. Everyone can verify that matcher follows matching rules, so there is no need to trust them. And there is matchers in different levels/delays, and node just send order to one low delay matcher, which sends it to higher delay matcher, which again sends it to higher delay matcher. So if you have chosen different matcher than someone, you can still get match with him, thanks to higher delay matcher.

Node choose probably matcher based on amount of orders in matcher's orderbook, and if nodes do like this, everyone will use same matcher. Even if someone have some reason to use some other matcher, then all matchers probably choose same higher dealy matcher to match orders between their orderbooks. So I think there willbe just few matchers(on every pair) + one higher delay matcher which match orders between other matchers. So one order need just send to one matcher, and matcher send it may still to one higher dealy matcher.

This also make it possible to have more order types, so post only orders and trigger price is added.

Hello @xVETTEx, thank you for your improvement proposal. You clearly are very skilful with cryptocurrency exchange protocols and we are happy you are interested in OpenDEX. Here are my initial reactions to your improvement proposal. Correct me if I have misunderstood something.

The Post and Trigger order types are an interesting idea. Being able to perform these actions on OpenDEX certainly has value, but I think they may belong architecturally in a layer on top of the DEX. The Post order could be an order that is created and then cancelled within a very short timeframe. The Trigger order could be implemented as a bot that watches the market and places an order when a specific price is reached. That said, if a node provides a Trigger order service, i.e. watching the market for specific prices, then it could reasonably expect compensation for this computationally demanding service. We have been considering incorporate micropayments for services into the system, but I still think these types of services might be better as part of a separate service built on top of OpenDEX. If this sort of thing interests you then I can point you toward Exchange Union’s Hedgy and Arby projects.

No discrimination is a core principle of OpenDEX. Any node is allowed to locally ban another node, but letting makers send ban lists to other makers seems contrary to the project vision. Fortunately, removing the ban lists does not change the fundamental nature of your improvement proposal. I also worry that the problem you are solving may not be a problem, given OpenDEX’s reliance on Lightning and Raiden for atomic swaps. You say that everyone can verify that the matchers follow matching rules. How can they do this if they are non-matcher nodes, given that only matchers get to see orders?

Propagating orders to only slower nodes is an unusual idea. Won’t the slowest matchers end up with the most requests? A node’s latency may be due to network distance or hardware resources, but also to being overloaded, so it seems like a hierarchy of nodes where each node forwards traffic to only(and all?) nodes slower than itself would lead to a few powerful nodes participating happily while all the weaker nodes get slower and slower exponentially with the number of orders placed into the system. Basically I think there will be scalability problems. This is just my first reaction to diagramming the system as I understand it, and it is late so I may have misunderstood.

Verifying, and hopefully refining, the OpenDEX protocol, perhaps via simulation or proofs, has been of interest to me in the past and such a verification system could easily be used to model your proposed modifications too. Your proposed improvement might be an excellent solution for OpenDEX, but we definitely need to verify so major an architectural change before merging this PR.

Thank you for feedback @hatmer. Tell me if something in BOLDs is hard to understand or I haven't writed enough clearly, I'm not may best writer.

The Post order could be an order that is created and then cancelled within a very short timeframe

I don't think this would be post only order. Post only should be placed to orderbook as a maker if it won't match immediatly, and if it would matc immediatly, it should be just deleted. So you should cancel it only in case you've got swap request, and that would lead to bad reputation. If my explanation about post only orders wasn't very good, there is from bitmex:

Post Only Orders are Limit Orders that are only accepted if they do not immediately execute. That is, Post Only Orders never take liquidity. Market makers use Post Only Orders in order to only submit passive orders so as to earn the Maker rebate

Trigger order could be implemented as a bot that watches the market and places an order when a specific price is reached.

You are right. But when bot places order after it have seen trigger price is reached, it takes some time before matcher get order. If matcher would watch trigger prices, it would place order to orderbook immediatly when trigger price is reached.

That said, if a node provides a Trigger order service, i.e. watching the market for specific prices, then it could reasonably expect compensation for this computationally demanding service

Matcher's provide Trigger order service to their orderbook, so it would be possible to have some fee for trigger orders. But on the other hand I think it won't be so heavy computationally that price of thousand trigger order would be more than transaction fee.

No discrimination is a core principle of OpenDEX. Any node is allowed to locally ban another node, but letting makers send ban lists to other makers seems contrary to the project vision

Idea of banlists is that when matcher decide other party of your trade, you don't want trade someone that swap always fail with. So you have to tell matcher which nodes you don't want trade. Other solution which still needs banlists is that matcher node make it own banlist(nodes in banlsit can't place orders to orderbook) and nodes that use that matcher just check that all nodes that are banned locally is also in matcher's banlist. This may would be easier to implement to xud, but still need of banlist...

I also worry that the problem you are solving may not be a problem, given OpenDEX’s reliance on Lightning and Raiden for atomic swaps.

I think without matcher nodes "high speed trading" like a binance or bitmex isn't possible, tell me if you see how this could be made. Many nodes would try to take best order in same time, then second best etc.

You say that everyone can verify that the matchers follow matching rules. How can they do this if they are non-matcher nodes, given that only matchers get to see orders

Everyone, also non-matcher nodes can subscribe all orderbook events from matcher. If all nodes want to subscribe all orderbook events to know that matcher follow matching rules, in this case all orders would be needed to send everyone by matcher node, and this may cause scalability problems. I think it is possible that someone who subscribe orderbook events can prove to other nodes if matcher isn't following rules, so may we can figure out somehing that everyones wouldn't need to subscribe orderbook events.

Propagating orders to only slower nodes is an unusual idea. Won’t the slowest matchers end up with the most requests? A node’s latency may be due to network distance or hardware resources, but also to being overloaded, so it seems like a hierarchy of nodes where each node forwards traffic to only(and all?) nodes slower than itself would lead to a few powerful nodes participating happily while all the weaker nodes get slower and slower exponentially with the number of orders placed into the system. Basically I think there will be scalability problems. This is just my first reaction to diagramming the system as I understand it, and it is late so I may have misunderstood.

I think I should find a way to explain this better on BOLDs. Maybe delay is bad term to use with this. Delay of orderbook is time to accept or unaccept swap request, it have nothing to do with node's latency. And matcher always propagate orders to one higher delay matcher, just one. And only if in higher delay orderbook have more orders, which mean that also someone else matcher propagate its orders to this higher delay orderbook. Every matcher have their own orderbooks, so if node can't scale to that, node can just stop running some orderbooks.

Verifying, and hopefully refining, the OpenDEX protocol, perhaps via simulation or proofs, has been of interest to me in the past and such a verification system could easily be used to model your proposed modifications too. Your proposed improvement might be an excellent solution for OpenDEX, but we definitely need to verify so major an architectural change before merging this PR.

Some simulation etc. is good idea. At least I'd like to know how much network bandwitch being matcher would require.

Hi @xVETTEx,

Thank you again for your improvement proposal. Your ideas are directly relevant to the ongoing project of improving the scalability of the OpenDex network. For now I cannot merge your commits because I will need to first perform extensive verification of the scalability and resilience of any modifications to the system architecture. Our existing scaling plan involves a different system model, but it is good to have an alternative idea to compare with. I will leave this PR open and keep you updated.

Just came around to read this a bit more in detail. I think there are great ideas in the concept. Nevertheless, I see multiple levels of matchers relaying orders to each other leading to the exact same problem that we are facing right now: orders not being available if a match happens somewhere else.

The only solution to this indeed: match in one place only

Like @xVETTEx correctly said, there are different trading use cases, in general I would classify into two:

I want to exchange A against B, interested in holding B long-term. Not very time-critical.
I want to exchange A against B, and back to A for short-term profit. Time critical.

For 1. the current system is fine for 2. it is not. So I would suggest to split this Improvement Proposal up into a new one where there is only one matcher, no levels. This matcher can be specified by the trader e.g. in form of an OpenDEX URI (key@host:port). If the trader specifies this, the order will not get matched locally and is sent to this particular matcher instead (and nowhere else). It will get matched only with orders with other nodes who send their orders to the same matcher. Trader accepts matching result and if she is classified as maker or taker. Both (Taker+Maker) receive information from matcher when a match happened along with information to construct the payment.

The matcher should have a way to punish both traders if order settlement fails.

In this way we are creating an opt-in HF centralized matcher which can be used for use case 2.

So I would suggest to split this Improvement Proposal up into a new one where there is only one matcher, no levels. This matcher can be specified by the trader e.g. in form of an OpenDEX URI (key@host:port).

Reason for levels is that orders could be matched even if traders have choosed different matchers, but I think this could be done with some simpler way than levels. Do you see useful that nodes can subscribe orderbook events from matcher to see that matching rules is followed? I would like this feature as a trader, to see that matcher don't match first its own orders or something. However system could work also without this feature. Without this feature it is very easy to match orders between matchers, and way to it don't need to be specified in opendex protocol. If we decide to include subscribing orderbook events feature to opendex, we just need add to matching rules how to match orders between matchers.

Reason for levels is that orders could be matched even if traders have choose different matchers, but I think this could be done with some simpler way than levels.

Honestly, I can't imagine how this would work. For me the logic "if matches can happen in more than one place, there is a race condition" still holds.

Do you see useful that nodes can subscribe orderbook events from matcher to see that matching rules is followed? I would like this feature as a trader, to see that matcher don't match first its own orders or something. However system could work also without this feature. Without this feature it is very easy to match orders between matchers, and way to it don't need to be specified in opendex protocol. If we decide to include subscribing orderbook events feature to opendex, we just need add to matching rules how to match orders between matchers.

I would expect to have a way to publicly see what the matcher does, like an order match event feed, yes. But this won't eliminate all ways a matcher can be dishonest. I think we'd need a game theory incentive scheme for the matcher to stay honest. I'd have to think about this some more though.

opendexnetwork / opendex.network

[Improvement propsal] Adding matcher to protocol #13