Make Gnorkle Agents Gas-Less

Villaquiranm commented 2 months ago

Related issues and PRs

Motivation

In issue #1568, @deelawn implemented a Gno-based oracle and provided an example of how to use Gnorkle to link a Gno wallet address with a GitHub handle.

Upon reviewing the current process, we identified two key points:

First, only whitelisted addresses can view pending verification requests. Every time the GitHub Agent retrieves active verification requests, it incurs gas fees.
Second, after the agent verifies off-chain that the user owns the GitHub handle, it must pay additional gas fees to register the verification on the ghverify (Gnorkle instance).

The challenge with this approach is that Gnorkle bears the entire cost of the GitHub verification process. While this is feasible, we must ensure that the agent consistently has enough tokens to cover these expenses.

Proposition

Allow Gnorkle to execute messages from any sender, whether they are whitelisted or not, but only if the message is proven to be signed by a whitelisted address.

An GnoEvent should be published each time a user requests verification (this covers first point). This event can then be indexed by the off-chain agent, which will attempt to verify the ownership of the account. If ownership is confirmed, the agent will issue a payload to the user in the following format:

GNORKLE_FUNC,PUBLIC_KEY,RELAYED_PAYLOAD_SIGNATURE,RELAYED_PAYLOAD

GNORKLE_FUNC = 'relay' // always relay for relayed messages
PUBLIC_KEY ='gpub...'  // Get it from gnokey list

// identical to a non-relayed ingestion message
RELAYED_PAYLOAD='ingest,FEED_ID,OK' 
RELAYED_PAYLOAD_SIGNATURE=Sign(RELAYED_PAYLOAD, privKey)

Delivrables

Implement the native function verifySignature, which will take the public key, message, and signature as input, and return a boolean to indicate whether the signature corresponds to the provided public key, along with the address associated with that public key. #2776
Emit an event each time a user invokes the GitHub verification method.
Modify the Gnorkle code to handle relayed messages by adding support for message.FuncTypeRelay. Update the relevant section of the code, to include this new case and perform all necessary verifications.

Villaquiranm commented 2 months ago

@deelawn I would really appreciate your input here 💯

deelawn commented 2 months ago

Thanks @Villaquiranm; I'll take a look at this soon. @leohhhn because he said he'd be looking at gnorkle this week as well.

leohhhn commented 2 months ago

Taking a look; can you please name the issue more appropriately? It's not only about optimizing gas fees.

Villaquiranm commented 2 months ago

Taking a look; can you please name the issue more appropriately? It's not only about optimizing gas fees.

Thanks you're right maybe Enhance Gnorkle with Relayed Messages to Shift Gas Fees to Users for GitHub Verification would do ?

n0izn0iz commented 2 months ago

I propose: "Make Gnorkle Agents Gas-Less" although the initial title was already pretty accurate

The event emission is about avoiding to pay gas for retrieving the requests list
The off-chain signing and relaying by user is also about removing gas cost for the agent
verifySignature is a dependency of relaying

thehowl commented 2 months ago

Implement the native function verifySignature, which will take the public key, message, and signature as input, and return a boolean to indicate whether the signature corresponds to the provided public key, along with the address associated with that public key.

I'm not convinced by this approach.

I'd prefer if we changed std to have a method like GetMessage(), which returns a Gno representation of the corresponding std.Msg used to run the transaction, with maybe the following info for now:

type Message interface {
    Caller() Address
    Signers() []Address
}

(This could be an eventual substitute for GetOrigX functions; but I digress).

This way we can have gas-less gnorkle; by having the oracle multi-sig a message sent by the user. ghverify can "trust" the verification if the whitelisted address is in the signers, and by making sure std.AssertOriginCall (so that the verification function is not called indirectly).

thehowl commented 2 months ago

As for viewing verification requests, I agree it should be a gas-free query, but it should not work with events.

It should be a simple function, queryable using vm/qeval.

n0izn0iz commented 2 months ago

It should be a simple function, queryable using vm/qeval.

it could be also, but I think an event should be available too, so the nodes can push to the agents instead of the agents pulling repeatedly

thehowl commented 2 months ago

it could be also, but I think an event should be available too, so the nodes can push to the agents instead of the agents pulling repeatedly

If the problem we're trying to tackle is an off-chain oracle asking the chain "What users do I need to verify?", events do nothing to answer it; because they don't reflect what the chain has already stored. On the other hand, a queryable function does this.

I am starting to see that you and other contributors are beginning to use events for just about everything. I understand where you're coming from; working with the result of MsgCall is garbage for now, until we have #1776 and #1842. But understand that stuffing things into events to pass them off-chain is not good, as this is information that at least for now cannot be retrieved on-chain. We should look to make working off-chain just about as useful as working on-chain.

To me, in this specific case, there is no good reason to use events:

They add no more useful information than what you can already fetch using the indexer
They don't reflect the "current state" of all non-verified accounts; just the history of which verifications have been requested.
They add data to the transaction result which is frankly not necessary.

so the nodes can push to the agents instead of the agents pulling repeatedly

Understand this is already possible and easy to do; you can use the websocket endpoint on the indexer and listen for events on the realm+function combination. Once you receive them, you can vm/qeval the node. Yes, it's an added step, but it also means that the oracle can work more statelessly, and use the chain as the source of truth rather than keeping an internal database of who's verified and who's not.

n0izn0iz commented 2 months ago

We should look to make working off-chain just about as useful as working on-chain.

I agree that we should also have the eval query

They add no more useful information https://github.com/gnolang/gno/pull/2778#pullrequestreview-2300064523

I disagree and answered there https://github.com/gnolang/gno/pull/2778#issuecomment-2346650831 Basically event search allow nested calls queries where function search only allows query of EOA calls

They don't reflect the "current state" of all non-verified accounts; just the history of which verifications have been requested.

if the indexer is up to date it does It's like the chain rpc you query, if is not up to date, the query does not reflect "current state"

They add data to the transaction result which is frankly not necessary.

See my answer to the second point

Understand this is already possible and easy to do; you can use the websocket endpoint on the indexer and listen for events on the realm+function combination

Again, this will ignore non-EOA calls

Villaquiranm commented 2 months ago

Implement the native function verifySignature, which will take the public key, message, and signature as input, and return a boolean to indicate whether the signature corresponds to the provided public key, along with the address associated with that public key.

I'm not convinced by this approach.

I'd prefer if we changed std to have a method like GetMessage(), which returns a Gno representation of the corresponding std.Msg used to run the transaction, with maybe the following info for now:
type Message interface {
    Caller() Address
    Signers() []Address
}
(This could be an eventual substitute for GetOrigX functions; but I digress).

This way we can have gas-less gnorkle; by having the oracle multi-sig a message sent by the user. ghverify can "trust" the verification if the whitelisted address is in the signers, and by making sure std.AssertOriginCall (so that the verification function is not called indirectly).

Hello thanks a lot for your review on this issue :) I think I got what you're trying to say.

We do have sdk.Msg Msg on stdlibs.ExecContext so we could sort of create a native function that would get all the signers of the current transaction. Just something I'm not seeing currently is the orange dotted section on the image of the simplified flow down here.

Before I was thinking of having a sort of API on the off-chain whitelisted agent returning the payload to relay. So the user would just need to sign the transaction with the signed payload already included on the transaction. Now I'm just having trouble visualising the multisign process you're proposing, do we have some examples of how to 'multi-sig a message' ?

Thanks again 👍

deelawn commented 2 months ago

I believe @thehowl suggested the correct approach in regards to retrieving feed tasks without incurring any gas fees: use qeval.

In regards to emitting events -- this could be added in the gnorkle package, but it seems like the event data the oracle realm may want to emit is highly specific to the particular application. I think it is easier and more reliable to query the realm rather than subscribe to events -- what if the agent crashes? It will need to query the realm anyway to see what it might have missed. In any case, leaving this up to the oracle realm to emit events partially solves this because the agents can choose their own method of obtaining the feed tasks that need to be completed.

As for the proposed strategy to offload the verification gas fees from the agent, how does the agent communicate its signed message with the user that should pay the fee? By storing it in the ghverify realm? It seems unneessary.

I propose that we establish a pattern that allows oracle realms to pay out agents for data provider services rendered. In this case, the fee would be required by the ghverify realm when verification is requested, say 5gnot (I currently have no idea what a reasonable fee would be, so use this as an example). Upon receipt of the verification message from the agent, the fee can be paid out to the caller during the PostMessageHandler.Handle execution, defined in the ghverify realm.

This is a very simple solution for a very simple case -- a single agent doing the verification for a static feed with a single task value ingester. In the future, the agent incentive payout mechanism can be expanded by defining an Incentive() method on the Task interface. That incentive gets distributed to participating agents when the ingester commits a value to the feed's storage (or maybe just to agents in the majority when a feed's ingester accepts multiple values and aggregates them).

So one interesting exercise might be to figure out the best way to work in an incentive mechanism into the gnorkle package. It might involve modifying the Task interface and also allowing a banker to be stored as part of the gnorkle Instance. We have to be very careful any time we start passing a banker around created by the oracle realm that creates the Instance, but it would enable a more seamless way for ingesters to trigger payouts when feed values are committed.

That was a lot. Let me know what needs clarification.

Villaquiranm commented 2 months ago

@deelawn

As for the proposed strategy to offload the verification gas fees from the agent, how does the agent communicate its signed message with the user that should pay the fee? By storing it in the ghverify realm? It seems unneessary.

I was thinking mostly as the off-chain whitelisted agent does have access as a bridge to realms and to internet. Let's imagine the I want to verify my account. 1- I will query one the Off-chain agent with my address and handle as parameters. 2- The off-chain agent will qeval the ghverify realm in order to check if the RequestValidation was indeed requested by that address/handle. 3- Assuming the request was done, the agent will then fetch github to make all the vérifications 4- Assuming it is verified, the API will then return the signed payload, (with the private key whose public key is whitelisted on Gnorkle). 5- the soon to be verified user will maketx to Gnorkle (Paying for gas fees) to finalize the account vérification, The message signature is verified and it cannot be tampered with because then the verification would fail.

Maybe this is not at all what we want but I just wanted to better explain what was the flow I had thought about :)

leohhhn commented 2 months ago

I started working on a simple agent for GH verify; it's still very WIP, but thought I'd share it here. It bases the logic on a simpler approach - see the PR description.

I like @deelawn's idea to just pay a fee to the gfverify realm when requesting verification, so that those fees can be given to the agent as compensation.

Two main points:

If we add a compensation system, what happens when we have multiple agents verifying the same thing?
What does the agent do if it goes down? See if there are any missed+unresolved verify tasks? We would need the indexer for this.

gnolang / gno