search

waku-org / nwaku

Waku node and protocol.
Other
201 stars 53 forks source link

feat: Pluggable validator/QoS interface #1700

Open jm-clius opened 1 year ago

jm-clius commented 1 year ago

Problem

Waku currently implements and plans to implement various message "evaluation" mechanisms. Two examples of this would b:

Both these examples tie into the Waku Relay validation mechanism and drops messages that violate some configured rule criteria within a Waku network, subnetwork or application. Although message validation (especially for DoS protection) is an obvious use case for evaluating messages against some criteria, we can imagine more general evaluation scenarios (such as determining a preferred QoS for a message) which wouldn't strictly be a simple validation. For this issue I'll just refer to "validation" as it matches what we have already.

From the examples it's also clear that message validation is configured somewhat separately from the protocol (Waku Relay in both cases) and even sometimes belong to the application. Implementing this in nwaku therefore requires introducing (sometimes complex) validator logic which could in theory be more cleanly maintained separately or application-level constructs which more appropriately belongs elsewhere. There are more good reasons to want to separate validation logic, including communicating about Waku's internal vs external DoS protection capabilities, allowing third parties to provide their own validation/QoS, etc.

Suggested solution

Define a plugin interface and strategy for Waku validators. To keep things simple, this could for now be:

UPDATE: see also suggestion on defining this as a JSON-RPC API that will allow running validation/QoS as a web service in the comments below

cc @arnetheduck @fryorcraken @alrevuelta

rymnc commented 1 year ago

We could probably look into using wasm too, in the future

jm-clius commented 1 year ago

Another suggestion was to define this as a JSON-RPC API. This way we get the benefits of separation of application-level concerns as set out in the description, but the validation can also be run as a web service that can be provisioned (we can, for example, provide RLN-validation-as-a-service).

fryorcraken commented 1 year ago

In js-libp2p, gossipsub topic validators return 3 possible values:

https://github.com/libp2p/js-libp2p-interfaces/blob/4c83268890ea521a2731d6c4bb0d18c5fd3dae7e/packages/interface-pubsub/src/index.ts#L136

Reject messages affect peer scoring:

https://github.com/ChainSafe/js-libp2p-gossipsub/blob/b6225d62fa3f372acee31f3ad71579693e0fcdbe/src/score/peer-score.ts#L448

Is that something we want to consider for this? ie, under a given threshold it rejects, and in the mid section it ignores.

fryorcraken commented 1 year ago

In js-libp2p, gossipsub topic validators return 3 possible values:

https://github.com/libp2p/js-libp2p-interfaces/blob/4c83268890ea521a2731d6c4bb0d18c5fd3dae7e/packages/interface-pubsub/src/index.ts#L136

Reject messages affect peer scoring:

https://github.com/ChainSafe/js-libp2p-gossipsub/blob/b6225d62fa3f372acee31f3ad71579693e0fcdbe/src/score/peer-score.ts#L448

Is that something we want to consider for this? ie, under a given threshold it rejects, and in the mid section it ignores.

jm-clius commented 1 year ago

Yes, I think it could fit this paradigm. Perhaps messages with a validation score lower than 0.5 should be considered similar to a "Reject" (but with possibility to affect gossipsub scoring more harshly the lower the score) and 1.0 would be an unqualified "Accept". It's then TBD whether 0.5-1.0 should be "ignore" or simply affect the QoS in some way.

chair28980 commented 9 months ago

@jm-clius @fryorcraken We are discussing whether to keep this issue open or closed, now that RLN has been implemented, is this issue still relevant/pertinent?

jm-clius commented 9 months ago

Yes, this approach would not change the RLN implementation, but make it "pluggable" so that other Waku users could potentially define their own validation rules. This issue remains relevant.

mfw78 commented 9 months ago

Definitely from an application perspective I can see validity in this pluggable design. I can think of a few off-chain services / protocols being run where spam protection may be minimised / reduced without the need for RLN, so having this pluggable / replaceable would be great.

arnetheduck commented 9 months ago

I'm hoping to see this one too - I see it as a fundamental customization point and a more simple and flexible way to deploy RLN - also for rln itself which in this architecture gains significant deployment flexibility

fryorcraken commented 9 months ago

https://forum.vac.dev/t/message-validation-marketplace-waku-network-of-networks/262 @arnetheduck @mfw78

rymnc commented 9 months ago

https://forum.vac.dev/t/message-validation-marketplace-waku-network-of-networks/262 @arnetheduck @mfw78

I like this idea! applications should be able to incentivize node operators to relay traffic for their app (and by extension, their rln membership set)

fryorcraken commented 9 months ago

I like this idea! applications should be able to incentivize node operators to relay traffic for their app (and by extension, their rln membership set)

https://forum.vac.dev/t/message-validation-marketplace-waku-network-of-networks/262/4?u=fryorcraken