ERC1538: Transparent Contract Standard

[mudgen]

---

eip: 1538 title: Transparent Contract Standard author: Nick Mudge nick@perfectabstractions.com status: Draft type: Standards Track category: ERC created: 31 October 2018

None: An EIP has been written for this standard and is here: https://github.com/ethereum/EIPs/blob/master/EIPS/eip-1538.md

Simple Summary

This standard provides a contract architecture that makes upgradeable contracts flexible, unlimited in size, and transparent.

A transparent contract publicly documents the full history of all changes made to it.

All changes to a transparent contract are reported in a standard format.

Abstract

A transparent contract is a proxy contract design pattern that provides the following:

1. A way to add, replace and remove multiple functions of a contract atomically (at the same time).
2. Standard events to show what functions are added, replaced and removed from a contract, and why the changes are made.
3. A standard way to query a contract to discover and retrieve information about all functions exposed by it.
4. Solves the 24KB maximum contract size limitation, making the maximum contract size of a transparent contract practically unlimited. This standard makes the worry about contract size a thing of the past.
5. Enables an upgradeable contract to become immutable in the future if desired.

Motivation

A fundamental benefit of Ethereum contracts is that their code is immutable, thereby acquiring trust by trustlessness. People do not have to trust others if it is not possible for a contract to be changed.

However, a fundamental problem with trustless contracts that cannot be changed is that they cannot be changed.

Bugs

Bugs and security vulnerabilities are unwittingly written into immutable contracts that ruin them.

Improvements

Immutable, trustless contracts cannot be improved, resulting in increasingly inferior contracts over time.

Contract standards evolve, new ones come out. People, groups and organizations learn over time what people want and what is better and what should be built next. Contracts that cannot be improved not only hold back the authors that create them, but everybody who uses them.

Upgradeable Contracts vs. Centralized Private Database

Why have an upgradeable contract instead of a centralized, private, mutable database? Here are some reasons:

1. Because of the openness of storage data and verified code, it is possible to show a provable history of trustworthiness.
2. Because of the openness, bad behavior can be spotted and reported when it happens.
3. Independent security and domain experts can review the change history of contracts and vouch for their history of trustworthiness.
4. It is possible for an upgradeable contract to become immutable and trustless.
5. An upgradeable contract can have parts of it that are not upgradeable and so are partially immutable and trustless.

Immutability

In some cases immutable, trustless contracts are the right fit. This is the case when a contract is only needed for a short time or it is known ahead of time that there will never be any reason to change or improve it.

Middle Ground

Transparent contracts provide a middle ground between immutable trustless contracts that can't be improved and upgradeable contracts that can't be trusted.

Purposes

1. Create upgradeable contracts that earn trust by showing a provable history of trustworthiness.
2. Document the development of contracts so their development and change is provably public and can be understood.
3. Create upgradeable contracts that can become immutable in the future if desired.
4. Create contracts that are not limited by a max size.

Benefits & Use Cases

This standard is for use cases that benefit from the following:

1. The ability to add, replace or remove multiple functions of a contract atomically (at the same time).
2. Each time a function is added, replaced or removed, it is documented with events.
3. Build trust over time by showing all changes made to a contract.
4. Unlimited contract size.
5. The ability to query information about functions currently supported by the contract.
6. One contract address that provides all needed functionality and never needs to be replaced by another contract address.
7. The ability for a contract to be upgradeable for a time, and then become immutable.
8. Add trustless guarantees to a contract with "unchangeable functions".

New Software Possibilities

This standard enables a form of contract version control software to be written.

Software and user interfaces can be written to filter the FunctionUpdate and CommitMessage events of a contract address. Such software can show the full history of changes of any contract that implements this standard.

User interfaces and software can also use this standard to assist or automate changes of contracts.

Specification

Note: The solidity delegatecall opcode enables a contract to execute a function from another contract, but it is executed as if the function was from the calling contract. Essentially delegatecall enables a contract to "borrow" another contract's function. Functions executed with delegatecall affect the storage variables of the calling contract, not the contract where the functions are defined.

General Summary

A transparent contract delegates or forwards function calls to it to other contracts using delegatecode.

A transparent contract has an updateContract function that enables multiple functions to be added, replaced or removed.

An event is emitted for every function that is added, replaced or removed so that all changes to a contract can be tracked in a standard way.

A transparent contract is a contract that implements and complies with the design points below.

Terms

1. In this standard a delegate contract is a contract that a transparent contract fallback function forwards function calls to using delegatecall.
2. In this standard an unchangeable function is a function that is defined directly in a transparent contract and so cannot be replaced or removed.

Design Points

A contract is a transparent contract if it implements the following design points:

1. A transparent contract is a contract that contains a fallback function, a constructor, and zero or more unchangeable functions that are defined directly within it.
2. The constructor of a transparent contract associates the updateContract function with a contract that implements the ERC1538 interface. The updateContract function can be an "unchangeable function" that is defined directly in the transparent contract or it can be defined in a delegate contract. Other functions can also be associated with contracts in the constructor.
3. After a transparent contract is deployed functions are added, replaced and removed by calling the updateContract function.
4. The updateContract function associates functions with contracts that implement those functions, and emits the CommitMessage and FunctionUpdate events that document function changes.
5. The FunctionUpdate event is emitted for each function that is added, replaced or removed. The CommitMessage event is emitted one time for each time the updateContract function is called and is emitted after any FunctionUpdate events are emitted.
6. The updateContract function can take a list of multiple function signatures in its _functionSignatures parameter and so add/replace/remove multiple functions at the same time.
7. When a function is called on a transparent contract it executes immediately if it is an "unchangeable function". Otherwise the fallback function is executed. The fallback function finds the delegate contract associated with the function and executes the function using delegatecall. If there is no delegate contract for the function then execution reverts.
8. The source code of a transparent contract and all delegate contracts used by it are publicly viewable and verified.

The transparent contract address is the address that users interact with. The transparent contract address never changes. Only delegate addresses can change by using the updateContracts function.

Typically some kind of authentication is needed for adding/replacing/removing functions from a transparent contract, however the scheme for authentication or ownership is not part of this standard.

Example

Here is an example of an implementation of a transparent contract. Please note that the example below is an example only. It is not the standard. A contract is a transparent contract when it implements and complies with the design points listed above.

pragma solidity ^0.5.7;

contract ExampleTransparentContract {
  // owner of the contract
  address internal contractOwner;
  event OwnershipTransferred(address indexed previousOwner, address indexed newOwner);

  // maps functions to the delegate contracts that execute the functions
  // funcId => delegate contract
  mapping(bytes4 => address) internal delegates;

  // maps each function signature to its position in the funcSignatures array.
  // signature => index+1
  mapping(bytes => uint256) internal funcSignatureToIndex;

  event CommitMessage(string message);
  event FunctionUpdate(bytes4 indexed functionId, address indexed oldDelegate, address indexed newDelegate, string functionSignature);

  // this is an example of an "unchangeable function".
  // return the delegate contract address for the supplied function signature
  function delegateAddress(string calldata _functionSignature) external view returns(address) {
    require(funcSignatureToIndex[bytes(_functionSignature)] != 0, "Function signature not found.");
    return delegates[bytes4(keccak256(bytes(_functionSignature)))];
  }

  // add a function using the updateContract function
  // this is an internal helper function
  function addFunction(address _erc1538Delegate, address contractAddress, string memory _functionSignatures, string memory _commitMessage) internal {    
    // 0x03A9BCCF == bytes4(keccak256("updateContract(address,string,string)"))
    bytes memory funcdata = abi.encodeWithSelector(0x03A9BCCF, contractAddress, _functionSignatures, _commitMessage);
    bool success;
    assembly {
      success := delegatecall(gas, _erc1538Delegate, add(funcdata, 0x20), mload(funcdata), funcdata, 0)
    }
    require(success, "Adding a function failed");   
  }

  constructor(address _erc1538Delegate) public {
    contractOwner = msg.sender;
    emit OwnershipTransferred(address(0), msg.sender);

    // adding ERC1538 updateContract function
    bytes memory signature = "updateContract(address,string,string)";
    bytes4 funcId = bytes4(keccak256(signature));
    delegates[funcId] = _erc1538Delegate;
    emit FunctionUpdate(funcId, address(0), _erc1538Delegate, string(signature));
    emit CommitMessage("Added ERC1538 updateContract function at contract creation");

    // associate "unchangeable functions" with this transparent contract address
    // prevents function selector clashes with delegate contract functions
    // uses the updateContract function
    string memory functions = "delegateAddress(string)";
    addFunction(_erc1538Delegate, address(this), functions, "Associating unchangeable functions");

    // adding ERC1538Query interface functions
    functions = "functionByIndex(uint256)functionExists(string)delegateAddresses()delegateFunctionSignatures(address)functionById(bytes4)functionBySignature(string)functionSignatures()totalFunctions()";    
    // "0x01234567891011121314" is an example address of an ERC1538Query delegate contract
    addFunction(_erc1538Delegate, 0x01234567891011121314, functions, "Adding ERC1538Query functions");

    // additional functions could be added at this point
  }

  // Making the fallback function payable makes it work for delegate contract functions 
  // that are payable and not payable.
  function() external payable {
    // Delegate every function call to a delegate contract
    address delegate = delegates[msg.sig];
    require(delegate != address(0), "Function does not exist.");
    assembly {
      let ptr := mload(0x40)
      calldatacopy(ptr, 0, calldatasize)
      let result := delegatecall(gas, delegate, ptr, calldatasize, 0, 0)
      let size := returndatasize
      returndatacopy(ptr, 0, size)
      switch result
      case 0 {revert(ptr, size)}
      default {return (ptr, size)}
    }
  }
}

As can be seen in the above example, every function call is delegated to a delegate contract, unless the function is defined directly in the transparent contract (making it an unchangeable function).

The constructor function adds the updateContract function to the transparent contract, which is then used to add other functions to the transparent contract.

Each time a function is added to a transparent contract the events CommitMessage and FunctionUpdate are emitted to document exactly what functions where added or replaced and why.

The delegate contract that implements the updateContract function implements the following interface:

ERC1538 Interface

pragma solidity ^0.5.7;

/// @title ERC1538 Transparent Contract Standard
/// @dev Required interface
///  Note: the ERC-165 identifier for this interface is 0x61455567
interface ERC1538 {
  /// @dev This emits when one or a set of functions are updated in a transparent contract.
  ///  The message string should give a short description of the change and why
  ///  the change was made.
  event CommitMessage(string message);

  /// @dev This emits for each function that is updated in a transparent contract.
  ///  functionId is the bytes4 of the keccak256 of the function signature.
  ///  oldDelegate is the delegate contract address of the old delegate contract if
  ///  the function is being replaced or removed.
  ///  oldDelegate is the zero value address(0) if a function is being added for the
  ///  first time.
  ///  newDelegate is the delegate contract address of the new delegate contract if 
  ///  the function is being added for the first time or if the function is being 
  ///  replaced.
  ///  newDelegate is the zero value address(0) if the function is being removed.
  event FunctionUpdate(
    bytes4 indexed functionId, 
    address indexed oldDelegate, 
    address indexed newDelegate, 
    string functionSignature
  );

  /// @notice Updates functions in a transparent contract.
  /// @dev If the value of _delegate is zero then the functions specified 
  ///  in _functionSignatures are removed.
  ///  If the value of _delegate is a delegate contract address then the functions 
  ///  specified in _functionSignatures will be delegated to that address.
  /// @param _delegate The address of a delegate contract to delegate to or zero
  ///        to remove functions.      
  /// @param _functionSignatures A list of function signatures listed one after the other
  /// @param _commitMessage A short description of the change and why it is made
  ///        This message is passed to the CommitMessage event.          
  function updateContract(address _delegate, string calldata _functionSignatures, string calldata _commitMessage) external;  
}

Function Signatures String Format

The text format for the _functionSignatures parameter is simply a string of function signatures. For example: "myFirstFunction()mySecondFunction(string)" This format is easy to parse and is concise.

Here is an example of calling the updateContract function that adds the ERC721 standard functions to a transparent contract:

functionSignatures = "approve(address,uint256)balanceOf(address)getApproved(uint256)isApprovedForAll(address,address)ownerOf(uint256)safeTransferFrom(address,address,uint256)safeTransferFrom(address,address,uint256,bytes)setApprovalForAll(address,bool)transferFrom(address,address,uint256)"
tx = await transparentContract.updateContract(erc721Delegate.address, functionSignatures, "Adding ERC721 functions");

Removing Functions

Functions are removed by passing address(0) as the first argument to the updateContract function. The list of functions that are passed in are removed.

Source Code Verification

The transparent contract source code and the source code for the delegate contracts should be verified in a provable way by a third party source such as etherscan.io.

Function Selector Clash

A function selector clash occurs when a function is added to a contract that hashes to the same four-byte hash as an existing function. This is unlikely to occur but should be prevented in the implementation of the updateContract function. See the reference implementation of ERC1538 to see an example of how function clashes can be prevented.

ERC1538Query

Optionally, the function signatures of a transparent contract can be stored in an array in the transparent contract and queried to get what functions the transparent contract supports and what their delegate contract addresses are.

The following is an optional interface for querying function information from a transparent contract:

pragma solidity ^0.5.7;

interface ERC1538Query {

  /// @notice Gets the total number of functions the transparent contract has.
  /// @return The number of functions the transparent contract has,
  ///  not including the fallback function.
  function totalFunctions() external view returns(uint256);

  /// @notice Gets information about a specific function
  /// @dev Throws if `_index` >= `totalFunctions()`
  /// @param _index The index position of a function signature that is stored in an array
  /// @return The function signature, the function selector and the delegate contract address
  function functionByIndex(uint256 _index) 
    external 
    view 
    returns(
      string memory functionSignature, 
      bytes4 functionId, 
      address delegate
    );

  /// @notice Checks to see if a function exists
  /// @param The function signature to check
  /// @return True if the function exists, false otherwise
  function functionExists(string calldata _functionSignature) external view returns(bool);

  /// @notice Gets all the function signatures of functions supported by the transparent contract
  /// @return A string containing a list of function signatures
  function functionSignatures() external view returns(string memory);

  /// @notice Gets all the function signatures supported by a specific delegate contract
  /// @param _delegate The delegate contract address
  /// @return A string containing a list of function signatures
  function delegateFunctionSignatures(address _delegate) external view returns(string memory);

  /// @notice Gets the delegate contract address that supports the given function signature
  /// @param The function signature
  /// @return The delegate contract address
  function delegateAddress(string calldata _functionSignature) external view returns(address);

  /// @notice Gets information about a function
  /// @dev Throws if no function is found
  /// @param _functionId The id of the function to get information about
  /// @return The function signature and the contract address
  function functionById(bytes4 _functionId) 
    external 
    view 
    returns(
      string memory signature, 
      address delegate
    );

  /// @notice Get all the delegate contract addresses used by the transparent contract
  /// @return An array of all delegate contract addresses
  function delegateAddresses() external view returns(address[] memory);
}

See the reference implementation of ERC1538 to see how this is implemented.

The text format for the list of function signatures returned from the delegateFunctionSignatures and functionSignatures functions is simply a string of function signatures. Here is an example of such a string: "approve(address,uint256)balanceOf(address)getApproved(uint256)isApprovedForAll(address,address)ownerOf(uint256)safeTransferFrom(address,address,uint256)safeTransferFrom(address,address,uint256,bytes)setApprovalForAll(address,bool)transferFrom(address,address,uint256)"

How To Deploy A Transparent Contract

1. Create and deploy to a blockchain a contract that implements the ERC1538 interface. You can skip this step if there is already such a contract deployed to the blockchain.
2. Create your transparent contract with a fallback function as given above. Your transparent contract also needs a constructor that adds the updateContract function.
3. Deploy your transparent contract to a blockchain. Pass in the address of the ERC1538 delegate contract to your constructor if it requires it.

See the reference implementation for examples of these contracts.

Wrapper Contract for Delegate Contracts that Depend on Other Delegate Contracts

In some cases some delegate contracts may need to call external/public functions that reside in other delegate contracts. A convenient way to solve this problem is to create a contract that contains empty implementations of functions that are needed and import and extend this contract in delegate contracts that call functions from other delegate contracts. This enables delegate contracts to compile without having to provide implementations of the functions that are already given in other delegate contracts. This is a way to save gas, prevent reaching the max contract size limit, and prevent duplication of code. This strategy was given by @amiromayer. See his comment for more information. Another way to solve this problem is to use assembly to call functions provided by other delegate contracts.

Decentralized Authority

It is possible to extend this standard to add consensus functionality such as an approval function that multiple different people call to approve changes before they are submitted with the updateContract function. Changes only go into effect when the changes are fully approved. The CommitMessage and FunctionUpdate events should only be emitted when changes go into effect.

Security

This standard refers to owner(s) as one or more individuals that have the power to add/replace/remove functions of an upgradeable contract.

General

The owners(s) of an upgradeable contract have the ability to alter, add or remove data from the contract's data storage. Owner(s) of a contract can also execute any arbitrary code in the contract on behalf of any address. Owners(s) can do these things by adding a function to the contract that they call to execute arbitrary code. This is an issue for upgradeable contracts in general and is not specific to transparent contracts.

Note: The design and implementation of contract ownership is not part of this standard. The examples given in this standard and in the reference implementation are just examples of how it could be done.

Unchangeable Functions

"Unchangeable functions" are functions defined in a transparent contract itself and not in a delegate contract. The owner(s) of a transparent contract are not able to replace these functions. The use of unchangeable functions is limited because in some cases they can still be manipulated if they read or write data to the storage of the transparent contract. Data read from the transparent contract's storage could have been altered by the owner(s) of the contract. Data written to the transparent contract's storage can be undone or altered by the owner(s) of the contract.

In some cases unchangeble functions add trustless guarantees to a transparent contract.

Transparency

Contracts that implement this standard emit an event every time a function is added, replaced or removed. This enables people and software to monitor the changes to a contract. If any bad acting function is added to a contract then it can be seen. To comply with this standard all source code of a transparent contract and delegate contracts must be publicly available and verified.

Security and domain experts can review the history of change of any transparent contract to detect any history of foul play.

Rationale

String of Function Signatures Instead of bytes4[] Array of Function Selectors

The updateContract function takes a string list of functions signatures as an argument instead of a bytes4[] array of function selectors for three reasons:

1. Passing in function signatures enables the implementation of updateContract to prevent selector clashes.
2. A major part of this standard is to make upgradeable contracts more transparent by making it easier to see what has changed over time and why. When a function is added, replaced or removed its function signature is included in the FunctionUpdate event that is emitted. This makes it relatively easy to write software that filters the events of a contract to display to people what functions have been added/removed and changed over time without needing access to the source code or ABI of the contract. If only four-byte function selectors were provided this would not be possible.
3. By looking at the source code of a transparent contract it is not possible to see all the functions that it supports. This is why the ERC1538Query interface exists, so that people and software have a way to look up and examine or show all functions currently supported by a transparent contract. Function signatures are used so that ERC1538Query functions can show them.

Gas Considerations

Delegating function calls does have some gas overhead. This is mitigated in two ways:

1. Delegate contracts can be small, reducing gas costs. Because it costs more gas to call a function in a contract with many functions than a contract with few functions.
2. Because transparent contracts do not have a max size limitation it is possible to add gas optimizing functions for use cases. For example someone could use a transparent contract to implement the ERC721 standard and implement batch transfer functions from the ERC1412 standard to help reduce gas (and make batch transfers more convenient).

Storage

The standard does not specify how data is stored or organized by a transparent contract. But here are some suggestions:

Inherited Storage

1. The storage variables of a transparent contract consist of the storage variables defined in the transparent contract source code and the source code of delegate contracts that have been added.

2. A delegate contract can use any storage variable that exists in a transparent contract as long as it defines within it all the storage variables that exist, in the order that they exist, up to and including the ones being used.

3. A delegate contract can create new storage variables as long as it has defined, in the same order, all storage variables that exist in the transparent contract.

Here is a simple way inherited storage could be implemented:

1. Create a storage contract that contains the storage variables that your transparent contract and delegate contracts will use.
2. Make your delegate contracts inherit the storage contract.
3. If you want to add a new delegate contract that adds new storage variables then create a new storage contract that adds the new storage variables and inherits from the old storage contract. Use your new storage contract with your new delegate contract.
4. Repeat steps 2 or 3 for every new delegate contract.

Unstructured Storage

Assembly is used to store and read data at specific storage locations. An advantage to this approach is that previously used storage locations don't have to be defined or mentioned in a delegate contract if they aren't used by it.

Eternal Storage

Data can be stored using a generic API based on the type of data. See ERC930 for more information.

Becoming Immutable

It is possible to make a transparent contract become immutable. This is done by calling the updateContract function to remove the updateContract function. With this gone it is no longer possible to add, replace and remove functions.

Versions of Functions

Software or a user can verify what version of a function is called by getting the delegate contract address of the function. This can be done by calling the delegateAddress function from the ERC1538Query interface if it is implemented. This function takes a function signature as an argument and returns the delegate contract address where it is implemented.

Best Practices, Tools and More Information

More information, tools, tutorials and best practices concerning transparent contracts need to be developed and published.

Below is a growing list of articles concerning transparent contracts and their use. If you have an article about transparent contracts you would like to share then please submit a comment to this issue about it to get it added.

ERC1538: Future Proofing Smart Contracts and Tokens

The ERC1538 improving towards the “transparent contract” standard

Inspiration

This standard was inspired by ZeppelinOS's implementation of Upgradeability with vtables.

This standard was also inspired by the design and implementation of the Mokens contract from the Mokens project. The Mokens contract has been upgraded to implement this standard.

Backwards Compatibility

This standard makes a contract compatible with future standards and functionality because new functions can be added and existing functions can be replaced or removed.

This standard future proofs a contract.

Implementation

A reference implementation of this standard is given in the transparent-contracts-erc1538 repository.

Copyright

Copyright and related rights waived via CC0.

[mattlockyer]

First Questions: 1) what else is out there? 2) how come the only delegate is payable? What about non-payable? or did I miss something? 3) any security issues? 4) how come you don't inherit Ownable.sol, is this because the owner of root will never change?

[mudgen]

Great questions. I'll answer the last three questions first:

First Questions:

1. what else is out there?
2. how come the only delegate is payable? What about non-payable? or did I miss something?
3. any security issues?
4. how come you don't inherit Ownable.sol, is this because the owner of root will never change?

2. The fallback function delegates every function call to a delegate contract. The fallback function is payable because some of the functions in a delegate contract may be payable. Making the fallback function payable makes it work for delegate contract functions that are payable and not payable.

3. See the newly added Security section in the standard.

4. A transparent contract could inherit Ownable.sol if someone wants the owner() function to be unchangeable. The above implementation is just an example of implementing ERC1538.

[mudgen]

First Questions:

1. what else is out there?

There are various ideas about different designs and implementations of upgradeable contracts. These are found on the web.

I didn't find any that provide the same list of benefits that this standard provides. The closest I found was zepplin's vtable implementation. ERC1538 was inspired by the vtable implementation, specifically from the idea of managing an upgradeable contract by function, instead of by contract address.

[AC0DEM0NK3Y]

Looks pretty good on the face of it. I'll have to go deeper and think any caveats when I get a little more time to jump in but yeah, nice format to allow flexibility while making things have some standard visibility on change history/reason.

I think there could be some improvements or cuts to ERC1538Query perhaps, quick example being in what format should functionSignatures() return, comma/semi-colon delimited for eg. but that is minor detail.

Question on this: "Delegate contracts can be small, reducing gas costs. Because it costs more gas to call a function in a contract with many functions than a contract with few functions."

Do you have numbers or have links to numbers?

edit: One thing you should perhaps think about adding/talking about is this is not something that should be done by default, and if it is used then people should be ideally putting in capability to pull authority out on being able to change the functions and where they delegate to.

[mudgen]

@AC0DEM0NK3Y thanks for the great feedback!

When a function is called on a contract there is a linear search to find the function in the contract. So every function costs a different amount of gas to be called. I did a simple test with a simple function. I made a contract with one function. The gas cost to execute it was 384. I made a second contract with the same exact function 26 times but with slightly different function names. The most expensive function to call was 934.

[mudgen]

@AC0DEM0NK3Y Yes, people can call the updateContract function to remove the updateContract function. That would remove the ability to change the contract, making it immutable. I added that in.

[mudgen]

From solidity documentation:

The low-level call, delegatecall and callcode will return success if the called account is non-existent, as part of the design of EVM. Existence must be checked prior to calling if desired.

This is not a security vulnerability in a transparent contract because the code above does check that a contract address exists before making a delegate call. It is possible for a user to submit an invalid address to the updateContract function, but that function could also check for the existence of code for the submitted address, and throw if the address is not a contract address.

[mwherman2000]

Why not simply create a generic worklow engine SC that understands BPMN workflow templates that have been transcompiled into a BPMN specific byecode?

[mudgen]

@mwherman2000 Sorry, I am not familiar with how that works so I can't answer that question.

[theblockstalk]

I would suggest that you get the Aaragon and zepplinOS team involved in this EIP, they are leading upgradeable contract researchers.

There are in fact multiple ways to implement upgradeable contracts using delegate call and the would not all be compatible.

• Some include a method for initialization which is not considered in this EIP.
• Some of them implement an owner contract in different ways
• Some of them delegate calls to one contract, instead of different contracts for each function

I would closely check that this is in line with the zepplinOS unstructured storage pattern, as I think this will be the most widely adopted.

[satyamakgec]

@mudgen It is a very nice approach to make the size of the contract unlimited. My worry point is how you can make communication between two different implementation contracts. AFAICT It only works when we have independent functions that don't depend on the other functions or internal functions.

Ex - some X function is used to change the state that used by other functions and it is implemented in the contract1 and then some Y function is implemented in the contract2 and want to use the X function the how do contract1 and contract2 function will communicate (communication between two implementation contract).

[bitcoinwarrior1]

@mudgen this seems like something quite handy, however it seems that a big bottleneck is the admin key which could be compromised to lead to a dodgy contract or simply lost. Seems you would need a sort of consortium to manage it...

[adibas03]

@mudgen One thing I think is missing is a default delegate address, so in a case where the first deployment has all the contracts in one address, this becomes important, this can always be removed by upgrading a function to address(0) if the function is no longer supported. In a scenario where there is no default delegate, it is simply set as address(0). @James-Sangalli Adding a consortium needs not be part of the standard, as that is no longer miimum Viable. As for the issue with the ownership, the delegate of the contractOwner() can be set to zero to return null for the contractOwner and remove the compromise issue, or in another option is to make the contract non-upgradable if that is the case just as @AC0DEM0NK3Y mentioned

[mudgen]

@jackandtheblockstalk Thanks for your feedback. I will check with the Aaragon and zepplinOS teams.

To be clear, multiple functions from the same delegate contract can be added to a transparent contract at the same time with the updateContracts function. Some people have the idea that each function that is added has to be from a different delegate contract.

[mudgen]

@satyamakgec communication between functions from different contracts is not a problem.

If contract2 needs to use a function that is defined in contract1 then the solution is to add the same function, with the same definition, to contract2. Or if contract2 is already deployed then create a contract3 contract that has all the needed functionality, deploy it, and use that contract to add the functions to the transparent contract.

The problem can also be avoided by putting all the related functionality into one contract and then adding all the external/public functions to the transparent contract with one call to updateContract.

[mudgen]

@James-Sangalli yes, I think it is possible to build various kinds of systems on top of a transparent contract, such as governance, decentralized schemes, authentication schemes etc.

I modified the proposal, saying that the scheme for ownership or authentication is not part of the standard.

[mudgen]

@adibas03 I see that a default delegate address could be convenient. Would it be necessary?

The initial functions of a transparent contract could be added to it in the transparent contract's constructor. Various kinds of initialization of a transparent contract could be done in the constructor. Here is an example that adds initial functions to a transparent contract in the constructor.

 constructor(address _erc1538Delegate) public {
    contractOwner = msg.sender;
    emit OwnershipTransferred(address(0), msg.sender);

    // adding ERC1538 updateContract function
    bytes memory signature = "updateContract(address,string,string)";
    bytes4 funcId = bytes4(keccak256(signature));
    delegates[funcId] = _erc1538Delegate;
    emit FunctionUpdate(funcId, address(0), _erc1538Delegate, string(signature));
    emit CommitMessage("Added ERC1538 updateContract function at contract creation");

    // add initial functions
    // uses the updateContract function to add initial functions
    bytes memory initialFunctions = "myFirstFunction()mySecondFunction(string)";
    address initialDelegate = 0x343434353222222;
    bytes memory calldata = abi.encodeWithSelector(signature, initialDelegate, initialFunctions, "Adding initial functions");
    bool success;
    assembly {
      success := delegatecall(gas, _erc1538Delegate, add(calldata, 0x20), mload(calldata), calldata, 0)
    }
    require(success, "Adding initial functions failed.");   
  }

@adibas03 I agree with you that a consortium need not be part of the standard.

[Droopy78]

@mudgen This is great. There is no chance of selector clashing exploit with this implementation.

As far as storage goes, the reference implementation is the way to go as best practice (i.e. inherited storage). All delegates will probably also want to inherit from the ERC1538Delegate reference implementation for updateContract().

I'm trying to remember... assume a transparent contract's storage variables are all in the inherited base contract, and delegate contract 1 and delegate contract 2 also inherit these variables from the same base contract. Can delegate contract 1 and 2 then define their own separate storage vars on top of that or do they also need to line up in memory? I was assuming the former, as that is how it is worded in the description (also, it would be a pretty clear, hassle-free implementation at that point).

[mudgen]

@Droopy78 Yes! The lack of functions directly in a transparent contract completely avoids selector clashing and the complexity of dealing with that issue.

Yes, the variable storage of contract1 and contract2 need to line up. I found this sort of thing easiest to manage by creating a new storage contract for each new delegate contract that inherits the storage class from the previous delegate contract. This strategy keeps the storage variables lined up without duplicate code and without the chance of creating bugs.

Here's some examples of storage scenarios:

Let's say that contract1 was created and added to a transparent contract. Now you want to add contract2 to the transparent contract.

1. If contract2 does not use variables defined by contract1 (or later contracts) then it does not need to define variables created in contract1.
2. If contract2 uses variables defined in contract1 then contract2 also needs to define those variables.
3. If contract2 defines new variables then it also needs to define all the variables defined by contract1.

I am also interested in unstructured storage. The advantage of this approach is that later contracts can create new storage locations without having to define previous ones that it doesn't use.

[adibas03]

@mudgen You are right, it seems not necessary, but it might help to have it as part of the standard. Another use case, might be if a another proxy or Identity contract is used to handle the call, and therefore should handle all calls by default.

[wighawag]

That's a cool pattern.

One thing to add would be to add a security mechanism that invalidate transaction if the contract changed while the transaction was in transit, or simply if the user was not yet aware of the changes.

Whenever the contract change, a version number is generated and that version is always used as part of the transaction data.

Implementation could then check the version number matches the current contract version. If it matches, the transaction is allowed. If not it is rejected since the sender was not aware of the changes at that point.

This would prevent frontrunning attack where the owner of the contract could change the contract logic just in time to compromise a wealthy sender.

[mudgen]

@wighawag Yes, you bring up a good point here. Thanks for this insight. It would definitely be good to prevent frontrunning attacks.

[mudgen]

@wighawag One way to handle frontrunning attacks is to verify the expected state after the call to the function you care about in the same transaction. If the state is not as expected then revert. This is how Project Wyvern handles frontrunning. Project Wyvern or its protocol handles the exchange contracts for OpenSea.io and other exchanges.

So here is an example of this. Let's say that ContractA is a transparent contract and you want to call function myFunction(). So you create ContractB with a function verifyMyFunction(). The verifyMyFunction() function calls myFunction() in ContractA and then it checks that state has been changed correctly and throws if not.

Handling frontrunning like this works. No change to ERC1538 is needed to implement this.

[adibas03]

Another implementation of this is openZeppelin's re-entrancy guard. Though it is still a little tricky how to implement in this.

[mudgen]

@adibas03 how does a re-entry guard prevent a frontrunning attack?

[wighawag]

@mudgen Interesting, that would indeed works but for some state changes that might be tricky or maybe not even possible to check, no ?

The other solution I was thinking is for all function call to have a version as first argument and the ERC1538 default function would simply check if it matches the current version. If not it reject.

The drawback is that you always have to have this extra parameter for every call. Making it incompatible with existing contract standard.

You could use a wrapper contract that add that extra param though.

Another way is to add a mapping (address => version) that block user from ever calling function unless they previously authorized the newest version.

In other word, if the contract update, users need to call approveVersion(versionNumber) first before being able to call any other function

[cwgoes]

The other solution I was thinking is for all function call to have a version as first argument and the ERC1538 default function would simply check if it matches the current version. If not it reject.

I think you can just use the query functions in the calling function to check the delegate call target, a la https://github.com/ProjectWyvern/wyvern-ethereum/blob/master/contracts/exchange/ExchangeCore.sol#L713 (similar pattern).

@mudgen This looks pretty solid to me. A few possibilities to consider:

• More granular upgradability - adding the ability to lock specific functions (instead of locking the whole contract by replacing the upgradeContract target). This would require an additional method and a table of function selectors to booleans, but it would a mix of guarantees for contract users that particular functions will not change with flexibility for other functions.
• Consider combining the two events so that a function update must correspond to exactly one explanatory message (also makes querying easier).

[wighawag]

@cwgoes

I think you can just use the query functions in the calling function to check the delegate call target, a la https://github.com/ProjectWyvern/wyvern-ethereum/blob/master/contracts/exchange/ExchangeCore.sol#L713 (similar pattern).

I am not sure I follow you, what is the query function? Remember there could be multiple implementation contract in ERC1538 and each of the implementation could call further implementation. As such the contract state (links to the different implementations) should be considered as a whole. Else you need to pass the full list of implementation's address to be checked.

[cwgoes]

I am not sure I follow you, what is the query function? Remember there could be multiple implementation contract in ERC1538 and each of the implementation could call further implementation. As such the contract state (links to the different implementations) should be considered as a whole. Else you need to pass the full list of implementation's address to be checked.

It would need to be the case that the query function could not be upgraded and that the delegatecall target was not also an ERC1538 proxy.

[mudgen]

This looks pretty solid to me.

@cwgoes Thanks.

Consider combining the two events so that a function update must correspond to exactly one explanatory message (also makes querying easier).

Yes, I agree. I have looked at this. I considered one event like this: event ContractUpdate(address[] oldDelegates, address[] newDelegates, bytes4[] functionIds, string functionSignatures, string commitMessage); That one event could hold all the changes of one call to the updateContract function. But there is no parameter indexed in this event so no way to filter the events based on the parameters. This one event also has its own complexities of parsing the data. I don't know of other event designs that might work better.

[mudgen]

@wighawag @cwgoes Any security solution that depends on an upgradable contract's storage variables does not work because the owner(s) of the contract can always add a new function and call it to modify any storage state in the contract.

The dirty truth is that an upgradeable contract requires that the users of the contract trust the owner(s) of the contract. Note that this trust issue doesn't come from the design of the transparent contract standard. This is a problem for any upgradeable contract that uses delegatecall.

That is why a big part of this standard is about ways to make a contract earn trust, or be more trustworthy.

Here are ways the standard mentions to earn trust or be more trustworthy:

1. Standard events to show what functions are added, replaced and removed from a contract, and why the changes are made.
2. Create upgradeable contracts that earn trust by showing a provable history of trustworthiness.
3. It is possible to extend the standard to decentralize the authority to add/replace/remove functions. For example there could be an approve function that multiple different people have to execute.
4. To be compliant with the standard the source code of the transparent contract and all delegate contracts used by it need to be publicly viewable and verified.
5. An upgradeable contract can become immutable in the future by removing all update/upgrade functions.
6. A standard way to query a contract to discover and retrieve information about all functions exposed by it.

I am interested in more ways that an upgradeable contract can have, get or earn more trust.

[Droopy78]

@mudgen Is it possible to do a variation of the ZeppelinOS vtable implementation you linked to above? It has a version mapping to each function implementation. It includes a registry of versions, so if you called a specific function by version, there would not be a possibility of front-running since upgrading a function would not modify the existing version you are calling (it would simply creates a new version of the function):

// Mapping of versions to implementations of different functions mapping (string => mapping (bytes4 => address)) internal versions;

[mudgen]

@Droopy78 Yes I think it is possible. Thanks for your thoughts about this.

I don't think it make sense to try to prevent front-running attacks from the owner(s) of an upgradeable contract. Because you already have to trust them. They can add a function and execute it to manipulate any of the contract storage at any time. The emitting of function events exist to make transparent this sort of bad activity so that nobody trusts the contract and its owners again.

I do like the idea of allowing the user to choose which version of a function to execute. Would every function have a parameter that specifies the version of function to call? This could be done but it wouldn't be possible to implement some standards this way like ERC721 and ERC20 because the functions in them don't take a version parameter.

Another way to give a user a choice on old and new functions is to give new functions new names and leave the old functions as they are. If a new function is the same as an old function but has some additional functionality then it makes sense to give the new function a new name and leave the old one as it is. No reason to update or remove an existing function if there is nothing wrong with it and users are using it or might want to use it. If the new function fixes a bug or something wrong then it makes sense to replace the old function and not let users call the old function.

[adibas03]

@mudgen the re-entrancy guard is a form of state checker, to ensure expected state before allowing completion , but upon looking at it again, it is definitely not suitable for the use-case, as it does not protect from the front-running from the upgradeContract function. Having a version number as described by @wighawag , might be the solution for the update contract, as even having a consortium required to upgrade does not solve the problem, as the different keys could be compromised, and I don't think adding that, is worth the gas and complexity expenditure to add such.

@Droopy78 the addition of another mapping increases the overall cost, and could lead to storage issues in an edge case scenario, since there could hypothetically exist unlimited number of versions for unlimited number of functions, till the limit of the contract, which could cause memory issues on smaller devices.

@mudgen I think the importance of upgrade goes beyond adding features, as while you are right that once the function content changes, it is a different function, there is still a requirement for such for case like security. For instance if the 2017 parity bug, was called through a transparent contract, you would want to be able to update the said function to avoid any other contracts being exposed.

Using the version on the other hand, enforces real transparency, as if you last vetted a version and it has since changed, it could revert with an error, that a more recent version is now available, and it is up to you to vet the update or proceed with the knowledge of the update. Finally, I would prefer only the most recent version is kept in the contract, that way, you either do not proceed to use the contract, or you vet the most recent version and go ahead with the update and there is no worry of the possibility of storage or memory issues.

[spalladino]

Hey @mudgen! Santiago from ZeppelinOS here. Just got the time to review the proposal. Looks pretty solid, and thanks a lot for picking up the vtable experiment and pushing it forward!!

FWIW, for the current version of ZeppelinOS, we decided to go with a more simple approach (a proxy that delegates to a single address) since it removes a few complications when coding upgradeable contracts (such as having to coordinate storage, or not being able to share internal functions). But we do envision supporting other upgradeability patterns in the future, and it's nice to see the community pushing for alternatives :-)

An improvement I'd strongly suggest is using unstructured storage for storing the contract owner and the delegates. This removes the need for any implementation contract to extend from a UpgradeStorage base contract. From our experience, the fewer requirements you impose on your users, the better the chances for adoption.

As for allowing users to decide which version of each function they want to call (which was mentioned in the last comments), I'm not very sold on the idea to be honest: I feel it may open the door to certain incompatible calls in a wider system. I'd keep it simple for the time being, until we have a better understanding of this pattern.

Also, I'm not sure this implementation does prevent the selector clashing exploit, as was mentioned before. Registering a delegate which has a different function name but the same function identifier as updateContract may end up in accidentally removing upgradeability. I'd be particularly careful in protecting that function.

Last but not least, we have a clash on the naming itself! We are pushing for the transparent proxy naming from ZeppelinOS to describe proxies that are indistinguishable from the implementation for anyone but their owner. You can read more about that here, and we'll be probably releasing a blogpost soon. I'd suggest renaming this proposal to something that better describes its main selling points, which to me are being able to upgrade functions individually, and having the proxy dispatch to the correct implementation. I personally like vtable, but that's just because I came up with that name and are somewhat attached to it :-P

[mudgen]

@spalladino Thank you so much for your feedback here. I really appreciate and like the work you and your team are doing with upgradeable contracts.

Yes, I very much like the unstructured storage approach and I think it is better. How variable storage and ownership is done is not part of the standard but I think people are likely to follow the example given. So I'll update the example to use the unstructured storage approach. Thanks for pointing this out.

Update: To keep the example given simple I decided to keep it how it is.

Yes, I agree with your thoughts on the versioning. One thing that can happen is that a bug is written into a function. When the contract is upgraded to fix the bug you never want users to be able to call the buggy version of the function.

You are right that the standard itself does not prevent selector clashing. But because the function signatures are passed into the updateContract function it is possible to prevent selector clashes within the implementation of updateContract. I will add a note about this to the standard.

I also like the vtable name. The standard could be renamed to "Transparent vTable Contract Standard". But does this clash with your vtable implementation name? Would you prefer this standard was named that rather than "Transparent Contract Standard"? I think "Transparent" is a key part of the name of this standard because a very major thing this standard does is report information about the change of every function.

By the way, with this standard it is possible for delegate contracts to share internal functions by duplicating the internal functions in the delegate contracts or by making the delegate contracts inherit the same contracts that contain the internal functions.

[spalladino]

I also like the vtable name. The standard could be renamed to "Transparent vTable Contract Standard". But does this clash with your vtable implementation name? Would you prefer this standard was named that rather than "Transparent Contract Standard"? I think "Transparent" is a key part of the name of this standard because a very major thing this standard does is report information about the change of every function.

It's fine with me if you pick up the vtable name, as we won't be pursuing that pattern in ZeppelinOS in the short term. And to be honest I love the fact that an experiment was picked up by a member of the community and driven forward until it becomes a standard!

As for transparent being a key part of the name, maybe I have a different idea in my mind about what "transparent" implies. To me it was about the proxy being invisible to a client, who would not be able to distinguish it from the actual contract, and not so much about reporting the changes of all functions. In that sense, something along the lines of "registry", "logging", "history", "reporting" may be better suited (though I don't really like any of those!). Anyway, to make a long story short, feel free to grab the vtable name for this standard! I'd be happy if you do :-)

[spalladino]

And something about the standard, which your last comment about internal functions reminded me: it's critical to have a way to atomically upgrade multiple functions, in case there is a dependency between two of them, or they share the same internal functions. You definitely don't want a tx that calls two public functions to use the new version for one of them, and the old one for the another.

That said, this feature does not necessarily need to be built into the standard. Using any forwarder that can receive multiple meta txs and call them in a single eth tx would do the trick.

[mudgen]

@spalladino I understand. This is built into the standard. The updateContract function adds, updates or removes multiple functions at the same time, atomically. The second parameter of updateContract takes a list of multiple functions signatures so that multiple functions are added/updated/removed atomically. I will make this more clear in the standard. The format of the string of signatures is given above in the standard and the reference implementation shows how to parse it.

This is why it is difficult to have one event that shows all function changes and a single commit message for all them.

[adibas03]

@mudgen @spalladino will an option for latest version, where the absence of a version value means a call to the latest version, as the default behavior cover the effect of adding the version variable.

Two cases

• Calls made with 0x0 passed as desired version, indicates the latest version is always chosen, so large scale, non sensitive calls can be made without any worries
• calls made with a non-0x0 version value, indicating a specific version and throws when said version is updated, with notification of the update, which can be used to cover sensitive transactions

[mudgen]

I am looking for people who are interested in writing a tutorial that shows an example of implementing ERC1538 or shows other things with regards to it. I have added a section to the standard where links to such articles can be added.

[maxsam4]

@mudgen It's great that you are pushing this forward and the write up is pretty mature as well. Thanks for sharing.

The only concern I have for this is the use case. Rather than having different function call implementation in different contracts, I would prefer to split the contracts to literally different upgradable contracts.

We don't want implementation contracts to unknowingly overwrite each other's data. Implementation contracts will have to be aware of what the other Implementation contract's data structure looks like so that they don't mess it up. We can just inherit a common storage structure in all but then we won't be able to change storage structures hence limiting the upgradability. On the other hand, In unstructured storage, we can add storage structures to the contract even after deployment (need to be done carefully, though).

I can see the benefits your implementation brings and I won't mind having this as a standard but I don't think it is worth the overhead required to feed and manage the function names.

Just my two cents.

[mudgen]

@maxsam4 Thanks for your comments and sharing your concern!

Instead of inheriting a common storage structure for all, you can just inherit the storage structure that is needed for a contract.

The problem can be solved this way:

1. Create a storage contract that contains the storage variables that your transparent contract and delegate contracts will use.
2. Make your delegate contracts inherit the storage contract.
3. If you want to add a new delegate contract that adds more storage variables then create a new storage contract that adds the new storage variables and inherits from the old storage contract. Use your new storage contract with your new delegate contract.
4. Repeat steps 2 or 3 for every new delegate contract.

The above is a simple way to handle the problem. There are other organizational ways to handle it. I like to have my storage contracts more granular so for my Mokens transparent contract I created storage contracts that fit exactly or almost exactly each delegate contract. Each storage contract inherits the appropriate prior storage contract so that variable storage is aligned. You can see my storage contracts here: https://gist.github.com/mudgen/aa626da30fdc2dedfac640a8e063255f

[maxsam4]

@mudgen Yes, aligning is what I meant by same storage structure. Using the same structure for every contract is just an easier way to do it. Also, you're right that we can add more just like we do with unstructured storage. I don't know why I thought we couldn't here. My bad.

Good luck!

[mudgen]

@maxsam4 Sorry, I changed around my answer a lot, I wasn't sure if you knew that more variables could be added or not, or if you were just mentioning an organizational problem with inheriting storage.

[Amxx]

The reference implementation doesn't support function containing tuples as it doesn't track the nested parentheses. Their are different solutions to that:

1. Track the number of nested parentheses
2. Change the signature format to be an array of strings
3. Add a special character (for example a semicolumn) between the different signature

[mudgen]

@Amxx good point about the tuples.

@amiromayer I think the wrapper contract for transparent contracts is useful enough to add to the standard as a possible strategy for handling the use case you gave. Great idea.

[mudgen]

Two new sections have been added to the standard to address points made by @satyamakgec and @amiromayer.

The sections are: Wrapper Contract for Delegate Contracts that Depend on Other Delegate Contracts

And:

Removing Functions

[benzap]

@mudgen Hello!

I was wondering if you or anyone implementing this EIP have run into issues with byte conversions involving the function signatures when using the latest version of solidity (0.5.0+)?

Solidity added quite a few more restrictions, and i've noted that some of the contracts that i've been developing required a new approach when implementing.

I ask because i'm interested in attempting to implement a variation on this approach, since i've been struggling with contract sizes, and i'm trying to keep everything within the latest versions of solidity.

Thanks,

Ben

[mudgen]

Hi @benzap!

I have yet to use solidity 0.5.0. If you come across any interesting/useful issues then please let us know.