Open igormcoelho opened 3 years ago
So these two transactions must be included in the same block?
The idea is that this is a single and atomic transaction.
One way to implement this is to put some new verification script field on tx header (I'm avoiding this approach)
Another way is to have the script part that emits the GAS.scheduleTransfer(sender, GetTransactionHash(), value)
to be embedded in a co-signer (I prefer this one, because it's supported by current mechanisms)
So user attaches a sender witness, and another "transfer witness" that generates collateral effect as a "promiseNotification" (just like a notification, but that vanishes as long as invocation part of transaction begins).
With this, Neo system just needs to catch the "promiseNotification" from GAS for "transaction balance" (and maybe the specific contract scopes of that witness), and this can be done deterministically (I hope so), before the transactions is put on block (the same as fee calculation currently being done). This way, we don't interfere much with Neo system, and for user, it just needs to implement NEP-TTT together with NEP-17:
function scheduleTransfer(HASH160, HASH256, BigInteger)
And maybe another way (not sure now if it's necessary):
function consumeNotification(NotificationObject)
- called after notification is "consumed" (begin of invocation part)
So this is a solution to let the contract pay for the fee?
So this is a solution to let the contract pay for the fee?
I think so @erikzhang
Then why not use the contract hash as the sender of the transaction?
:thinking:
Then why not use the contract hash as the sender of the transaction?
[EDIT]: sender
means signers[0]
That's a very good question. Using the sender looks nicer indeed.
I think that using the balance from sender requires permission, and it could indeed provide permission to spend its GAS assets. I just don't know how it could give "partial permission" over a limited amount of assets (imagining some "unbounded" verify()
returns true). Another trick is to prevent the actual movement of assets, to forbid non-determinism (that's why I think we should do all this on Witness processing level with special function "scheduleTransfer", not real "transfer").
But I think you're right, Neo System could automatically invoke this "scheduleTransfer" directly from sender (as a contract), to launch "promiseNotification" and guarantee the existence of GAS to make whole its future operation, and store it on current transaction (this is much easier for everyone as no new script is required, all automatic). One trick of using "sender" is that maybe the remote contract wants to give partial allowance according to the "user" (but who is the user now?). That's why I still miss a role of the "sender" (who actually wants that tx processed) and maybe a "payer" (which is typically the sender, but not in this case).
Let me just highlight the most fundamental points trying to be accomplished here, in my opinion:
This NEP considers a transaction as a valid (and temporary) token holder (as HASH256 identifier, not usual HASH160).
IIUC this requires changes to NEP-17 and if we're using "transactioned" GAS for fees then it also requires senders to be 256-bit wide which is quite invasive.
"partial permission" over a limited amount of assets
Verification context has access to transaction and it can get fee values to decide whether it wants to allow spending a particular amount of GAS for fees.
That's why I still miss a role of the "sender" (who actually wants that tx processed) and maybe a "payer" (which is typically the sender, but not in this case).
Isn't it all controlled by witness scopes?
funds are stored on transaction, for direct usage, with limited scope (respecting the given witness scope)
I think I don't quite understand this "stored on transaction" part. If actual contract storage isn't affected then it's a kind of "ephemeral" storage and this then raises questions like what happens if 0.5 GAS was requested and 0.4 spent, what transfer events should be generated for this storage/spending (especially for fees). Also IIRC notifications are forbidden at the moment in verification context.
this makes it easier to consume one-time from transaction (such as in minting)
But do we have any problem with that? I think we have any minting/staking/depositing case covered by simple NEP17 transfers with additional data to contract's address, then the contract in question can do anything it needs to in onNEP17Payment
handler.
this makes it possible for contracts to give away GAS that can only be used within a given scope, meaning that it won't be used to sponsor third-party operations
I think this case can be perfectly handled by Notary subsystem from neo-project/neo#1573. Contract's address can still be used as a sender (but some other address can be used if needed), but user will create an incomplete transaction and then the decision on whether this transaction should be completed or not can be made by dApp backend instead of (inevitably limited) contract's verification method. Backend can have some state-dependent logic for which addresses can use this feature and how much GAS they could spend, it can also parse the entry script and for example only allow some specific calls to be made.
Thanks for taking the time to evaluate this proposal @roman-khimov. I agree that neo-project/neo#1573 is an interesting approach, specially as it deals with P2P operations, so I don't see any conflict with this proposal here. This aims at providing temporary storage at current transaction and allow scoped-acess to this temporary storage. I'll try to clarify some points here and presenting a possible "implementation-driven" approach.
this requires changes to NEP-17 ...
This is important: NEP-17 isn't affected in any way (and certainly no address change!). The main intention at the moment is to having GAS contract implementing NEP-17 and NEP-TTT (an extra method), although other contracts may benefit from it as well.
I think I don't quite understand this "stored on transaction" part.
That's the core of this proposal, I'll try to clarify. Currently, user informs that tx "fee" on GAS will be, and then Neo System automatically takes this from user "GAS account" and uses this to makewhole its operational costs.
The idea here is to create a memory map/dictionary like this (this represents the "promiseNotification" I mentioned eariler): (TransactionHash, ContractHash, Scope) -> Data
Since data can only be read from inside Transaction itself, TransactionHash part can be ommitted on practice (not a global information, not even block-level, just single-transaction-level).
[EDIT]: sender
means signers[0]
The workflow I imagine is:
remainingFee
the non-paid GAS execution part (remainingFee="fee"
)PubkeyScript
is "co-signer", and sender
is some GAS-sponsoring ContractY through verify()
, as pointed out in this discussion (I'm not fully confident that this will work, so I'll propose not using it, then you can explain to me if it's the same as below)PubkeyScript
is sender
, and "co-signer" is third-party GAS-sponsoring ContractY.
sender
Pubkey is validated and method GAS.scheduleTransfer(sender, GetTransactionHash(), remainingFee)
is invoked (but it fails because there is no GAS in sender
, this is ok)remainingFee
is kept the same, as no funds were extracted from sender
ContractY.verify()
is invoked, so during this process it observes that Transaction.remainingFee > 0
, so it manually invokes GAS.scheduleTransfer(ContractY.id, GetTransactionHash(), remainingFee)
, according to some business logic that checks that sender
"deserves" such remainingFee
(it could even give it partially, but let's suppose now it's 100% or 0%, to simplify)GAS.scheduleTransfer(ContractY.id, GetTransactionHash(), remainingFee)
succeeds, and remainingFee
is allocated into the following Transaction field (NEP-TTT): Transaction[GAS.id, ContractY.scope_hash] += remainingFee
.remainingFee
now is zeroremainingFee
is zero, if not, abort contract verificationscheduleTransfer
are effectively performed here (now Storage is changed and Transaction effectively starts holding the assets)Transaction[GAS.id, ContractY.scope]
. This respects the willingness of ContractY to allow executions only for its contract scope or its group scope (or even globally? we can build general GAS faucet with this...), with the exception of Entry costs (should we allow ContractY to cover them or not?).finalizeTransfer
to handle and clear them.Note that if sender
had funds, these would be allocated according to its own scope, for example Transaction[GAS.id, GLOBAL]
if they can be used anywhere during invocation (same logic to its witness).
then raises questions like what happens if 0.5 GAS was requested and 0.4 spent, ...
Another interesting feature is that we can have an extra method (like I mentioned before, finalizeTransfer
), to be called after Transaction processing. This method would ensure that Transaction ephemeral storage is cleared after execution... meaning that it could be used to claim back assets put on Transaction. Something like GAS.scheduleTransferFrom(GetTransactionHash(), ContractY.id)
(with no value, so it takes all available content and clears transaction memory).
But do we have any problem with that?
No, we can solve minting using OnPayment strategy. But it's powerful to have some "shared space" for assets during contract invocation, so that one can give assets to another one, or even cover system fees directly (it could even re-fill system fees during runtime, but we decide how much flexibility and complication we want to provide).
Does it makes it clearer or more confusing? :joy:
@erikzhang when you have some time, please take a look at the workflow above. Afterall, I don't think using sender
solves the problem.
The issue I see with using sender
as a third-party GAS provider, is that it's not clear to who it is providing GAS to (but I agree that's a problem that needs to be somehow solved by the verify()
logic, but personally I don't know how to do it).
I think that some explicit operation GAS.scheduleTransfer(ContractY.id, GetTransactionHash(), remainingFee)
could explicitly load Transaction with GAS, and we may also give this ability during verify()
method, to be used by any contract co-signer.
Another solved issue is the ability to manually give back unused tokens, something that secondary method finalizeTransfer
can do (during invocation time).
@roman-khimov please take a look at the workflow I've put in details. @shargon do you think this works?
General Outline: What do you think of the idea of Transactions holding temporary assets (including scoped-execution GAS)?
Asset.scheduleTransfer
(during Verification time, no Storage changed!)Asset.finalizeTransfer
is invoked after contract execution, to clear non-empty TTT fields (and possibly to give-back unused tokens)Asset.finalizeTransfer is invoked after contract execution, to clear non-empty TTT fields (and possibly to give-back unused tokens)
If the account balance is insufficient, it will cause the transaction that did not pay the network fee to be included in the block. This can lead to attacks.
If the account balance is insufficient, it will cause the transaction that did not pay the network fee to be included in the block. This can lead to attacks.
I've accounted for that already @erikzhang , and this won't happen with GAS as the existing accounting model will be immediately mirrored into this new one. As soon as a GAS.scheduleTransfer
is emitted on verification, Neo System is wise enough to deduce it from balance, precisely how it happens nowadays.
This variable gas_amount
and this var GasLeft
are the ones being "internalized" into Transaction "ephemeral storage".
gas_amount on ApplicationEngine
No attacks can happen, because no non-determinism on balance is generated from this (as no Storage access is performed, besides intrinsic GAS.GetBalance
operation that was already managed on Neo). It's exactly how it happens now, just giving a place to store the available GAS on operation (now it can be stored on the Transaction).
Let me give a numerical example, because this reasoning you mentioned is fundamental @erikzhang .
Please inform me of the possible mistakes I'm making here!
[EDIT]: sender
means signers[0]
(thanks @erikzhang)
Scenario:
Current mechanism:
sender
, fills in the GAS fees (netfee=0.2 sysfee=0.3) and invocation TKN.transfer(Alice,Bob,2) is added to transaction (costs 0.3 GAS)Proposed mechanism:
sender
, fills in the GAS fees (netfee=0.2 sysfee=0.3) and invocation TKN.transfer(Alice,Bob,2) is added to transactionContractTKN.verify()
is executedsender
) is worth 0.5 GAS, due to its 10 TKN balance, and then it executes GAS.scheduleTransfer(TKN, TX, 0.5)
Why not put contractTKN as the sender?
Why not put contractTKN as the sender?
I knew you were going to ask @erikzhang :joy:
[EDIT]: sender
means signers[0]
Just because the real sender
is Alice, and if we put ContractTKN as sender
, how would it know that Alice is the "real sender" (from the co-signers list)?
I think sender
is to pay for the fee, there is no other meaning.
And don't forget about "scope"... why should ContractTKN give GAS to some TX that doesn't even execute ContractTKN?
[EDIT]: sender
means signers[0]
I put ContractTKN as sender
, it gives me GAS, but I execute ContractXYZ during invocation... so maybe we should first agree that "scope" is important for GAS under execution, and there could be "multiple GAS kinds" available during a single execution (some GAS is global, some GAS is for ContractTKN scope, and some other GAS is for ...).
Instead of loading ApplicationEngine with such advanced logic, I would rather put a Dictionary/Map into Transaction, and consider that execution GAS is part of the Transaction Balance, instead of considering it part of "ApplicationEngine Balance".
In fact there is no sender
field in Transaction
. We only have signers
. And the sender is the first signer and pay for the fee.
why should ContractTKN give GAS to some TX that doesn't even execute ContractTKN?
You add it as one of the signers, the verify
method will be invoked. You can check the transaction in verify
and decide whether to pay for it.
In fact there is no sender field in Transaction. We only have signers.
Ok, I'll fix description as signers[0]
.
You can check the transaction in verify and decide whether to pay for it.
That's the trickiest of all... there could be perhaps some theoretical way of doing this, but it's not possible "on practice" to deduce contract logic just by inspecting invocation script. It's simply so much easier to consider that, as long as signers[0]
is giving GAS, at least limit that GAS into signers[0]
scope. Do you agree? As long as its signature/verification is valid, you can spend its GAS.
If at least this change is made, ContractTKN logic for verify() becomes simple: it just checks that user put it as signers[0]
with "an interesting" scope (itself or its own contract group).
Besides the scoped GAS, which I believe now to be very important, there's another fundamental contribution of this NEP: GAS reloading during contract invocation.
As long as some contract has the ability to invoke GAS.scheduleTransfer(...)
during invocation, it could load transaction with very little GAS (just to cover network fee costs) and then periodically invoke method to put more into Transaction GAS Balance.
This is very good for dynamic applications that are hard to estimate on GAS costs, and it's much better than sending a lot of GAS just to get them back as refund (that could lock multiple tx invocations).
I still think this NEP is valid, and honestly, I find it quite beautiful to be able to store assets on a transaction. Specially, the idea of a transaction having a temporary balance, that expires once it's finished, and then it automatically invokes then cleaning method to handle non-zero pending balances. That's why I think we should keep it, and allow non-GAS contracts to have such feature. This could allow, in some future, to have other tokens behaving in a similar manner to GAS, and like I said before, having contracts interacting with each other through Transaction Balance.
From a practical perspective on GAS, I'll open two issues directly on Neo: https://github.com/neo-project/neo/issues/2442 and https://github.com/neo-project/neo/issues/2443 For these issues, it doesn't matter if GAS is stored on ApplicationEngine or in Transaction, as on practice, every ApplicationEngine instance is bound to a distinct Transaction (for me, it's the same). So feel free to evaluate the possibility of having such features on GAS, and then we evaluate if these should apply to general assets as well.
Does it makes it clearer or more confusing? :joy:
Certainly clearer, thanks @igormcoelho.
Technically I'd be concerned about these things:
remainingFee is allocated into the following Transaction field (NEP-TTT): Transaction[GAS.id, ContractY.scope_hash] += remainingFee.
and Now it may proceed with tx allocation in block and perform script invocation
. Verification is not supposed to have any side-effects as of now and C# node doesn't reverify in-block transactions, so these side-effects either need to be stored in transaction itself or recalculated before block processing. Actually, verification side-effects could be a blocker for this, it's a huge change.Also, there is a huge difference going from
remainingFee is allocated into the following Transaction field (NEP-TTT): Transaction[GAS.id, ContractY.scope_hash] += remainingFee
to
The idea here is to create a memory map/dictionary like this (this represents the "promiseNotification" I mentioned eariler): (TransactionHash, ContractHash, Scope) -> Data
The first one tries to solve local contract sponsoring problem (and it has other solutions), the second one is much more generic and raises a question of what data and how could be stored in transaction. This generic case of this proposal probably is interesting in that it can allow limiting the amount of assets approved to use in some situations, but I'm not sure we can reliably implement this more generic case and I'm also not sure it's to be used a lot, current scoping and onNEPXXPayment
mechanisms cover most of needs.
Very interesting technical points @roman-khimov.
Regarding re-verification, it's not needed for the reason described in this comment here: https://github.com/neo-project/proposals/issues/137#issuecomment-823716366
On essence, precisely the same existing strategy used now for GAS Native Contract (that controls in real-time the assets allocated to fees, thus preventing re-verification) could be done using scheduleTransfer
operations. They don't cause side effects "on general" during scheduleTransfer
and the only contract to trace such "effects" is GAS, since it's native and it requires that to put transactions on block while covering network fees. The other tokens that implement this (except GAS) are assumed to fail during invocation, as this operation will effectively happen before the first invocation operation on contract.
Another interesting behavior of this proposal is that:
scheduleTransfer
only "prepares ground" for the transfer (checking balance for instance), but no notification is not effectively launched at that momentscheduleTransfer
and executed, thus moving assets into Transaction. At this moment, "some sort" of Notification could be launched here, but that would have origin as HASH160 and destination HASH256, the tx.id itself. Personally, I don't think it's necessary (but it could)finalizeTransfer
will be automatically be launched if TTT fields are not empty (so, funds will never rest on transaction after execution)It's like creating a pool of funds that can be used during transaction execution. I think it's nice, but I agree we must find some "killer application" using this mechanism, to justify such trouble implementing it (maybe it helps on minting, maybe it helps with managing execution gas, ...).
@erikzhang @roman-khimov we have a problem... (or maybe my head is not quite right at this moment)
How can verify()
do anything useful, if it cannot access storage during verification (or can it)? :joy:
I mean, contract has an internal storage for vip customers, for token ratios, but none of this could be used during verify, is that correct?
So if it can't read storage during verification, then this NEP-TTT is absolutely necessary, in my opinion, as a way to provide accounting mechanism similar to GAS to other NEP-17 tokens, otherwise it's not possible to do contract sponsoring onchain.
I'll begin drafting a document to properly explain this to the community. If you have any questions, I'm willing to debate this as long as it is necessary, because I really think this is important for N3.
I recall that the essence of this NEP-TTT is:
scheduleTransfer
operation, because:Now it's clear to me that this is precisely what GAS does, and that's how it manages to escape non-determinism and still avoid a second re-verification. We need that for other tokens and this NEP can provide that. If we have that, users will be able to truly enforce collateral guarantees on any NEP-17 token before Invocation happens, without breaking non-determinism and avoiding re-verification, by the virtue of fact that balances are additive (we have talked about that sooooome time ago https://github.com/neo-project/neo/issues/814). As long as it is additive, mempool can keep on memory the updated balances for any token that has GetBalance(), besides GAS, and perform these transfers for ALL transactions in block, before any invocation happens (that's the way to prevent double spending and that's exactly how GAS manages to survive it, right?). So, I really don't see much alternatives now, if we want to do this process onchain (P2P alternatives will certainly work, but I just want to give all NEP-17 tokens the same treatment that GAS currently has).
How can verify() do anything useful, if it cannot access storage during verification (or can it)? :joy:
It can.
It can.
It can do something useful or it can access storage? :joy: Last time I seen there was a discussion on this, but if it has access to storage, we can manage many things.
It has access to storage and therefore it can do many useful things.
It has access to storage and therefore it can do many useful things.
Thanks @roman-khimov , I got confused about that.
On the other hand, this means that transactions are still being re-verified (on the witness part) after every block is put... I thought that only GAS parts would require updates by now (thus preventing any re-verification). Worse, it should re-verify transactions after each transaction is executed, otherwise it risks breaking verify()
sponsoring logic (as storage may have changed...). It justs not risk attacking Neo System, as GAS would be paid anyway, but from verify()
perspective, it may not be a wise choice. My point here is that, if we pursue this path of forbidding storage access @erikzhang , this NEP-TTT would allow same funcionality as GAS to any NEP-17, without any re-verification after blocks or tx execution.
Summary: this NEP called Transfer to Transaction tries to provide a practical manner to redistribute assets during Verification time on transaction, thus allowing implementation of practical mechanisms for "free transactions".
[EDIT 1] - This doesn't affect NEP-17 in any case. At the time of writing, this is mainly intended for adoption only on GAS contract, as NEP-17 and NEP-TTT. [EDIT 2] - Two methods are proposed in this NEP:
scheduleTransfer
andfinalizeTransfer
.[ORIGINAL PART]
=====
Transfer to Transaction
This NEP considers a transaction as a valid (and temporary) token holder (as HASH256 identifier, not usual HASH160). It can be done with operation "contract.scheduleTransfer(from, target_tx, value)" that launches a "promiseNotification" (consumed before actual contract invocation). Contracts can easily extract funds from transaction, for example, in a mint operation. Neo system can also consume funds from Transaction, for example, GAS funds to cover fee for operations. User can also specify a "contract scope" or "group scope" as the temporary holder (instead of global "transaction scope") and funds are consumed according to the scope.
Example 1 (usual operation):
fee
field on transaction and script runs directly from WitnessExample 2:
This "request" operation at ContractY can do some interesting thing such "redistributing" GAS for user operation, in that specific scope, with some simple rule as:
This means that giveaway operation only works after few blocks, and with limited gas supply (limited per block hold time and user shares on that contract).