EIP-2535: Diamonds

[mudgen]

EIP-2535 Diamonds exists here: https://eips.ethereum.org/EIPS/eip-2535

Below is a feedback and discussion of the standard.

[androolloyd]

This is great.

Would be curious to see a uniswap build using the Diamond pattern.

[mudgen]

@androolloyd yes, me too!

[pi0neerpat]

I think you need to add the proper table to the top of this file. Wait, this is an issue not a PR. I'm confused are you trying to make a new EIP?

(Putting discussion here since I don't see an external link)

This looks interesting, albeit very ahead of it's time. You've clearly put a lot of thought into it, and it is very well written. I'm curious to see how this intersects with governance. My jerk reaction is that this is only useful in a specific application that includes permission-based "cut" control. But even that would introduce lots of systemic risk. For now I think we are all stuck with upgrades being controlled by a single source of control, like Maker and rDAI, which publish documentation to record the history of changes. I look forward to the day where this fine-grained control is reality, and small changes can be made on-the-fly without introducing additional risk!

Am I missing another use-case for this?

Edit: (follow-up) how can we isolate storage on the same permissioned system that we use to allow cuts?

[mudgen]

@pi0neerpat thanks for your kind comments! The discussion is here. An official EIP for this standard is coming soon. It will be EIP 2535.

Companies are already using this kind of architecture (ERC1538) for their contracts, such as Caesar's Triumph, Enjin and others. It is real and in use today.

Enjin's NFT standard EIP 1155 recommends using an architecture like ERC1538 for upgrades.

I don't know why you think a diamond can't be controlled by a single source. The reference implementation of the standard is implemented that way -- only the single owner of the diamond can make changes. Maybe I am understanding you wrong? Authentication is not part of the standard, it can be fine grained or not.

A big use case is contracts with designs that exceed the max size of contracts, since diamonds don't have a max size. It is also very nice that while diamonds can be large their functionality can still be compartmented by facets. Particularly NFT contracts tend to exceed the max size limit, such as implementations of ERC721 and ERC998 and others that implement an NFT but also need to implement custom functionality for the application.

Permissions and authentication for cuts and access to storage variables can be handled in the same way as any other contract. Yes, if you wanted to, you could add various different permissions/authentications for different changes and handling different storage variables. Or you could keep it simple and not do that.

[leonardoalt]

The owners(s) of an upgradeable contract have the ability to alter, add or remove data from the contract's data storage. Owner(s) of a contract can also execute any arbitrary code in the contract on behalf of any address. Owners(s) can do these things by adding a function to the contract that they call to execute arbitrary code. This is an issue for upgradeable contracts in general and is not specific to diamonds.

Well, exactly. How is this standard any different from the centralized owned upgradeable smart contracts out there? Why not a standard that abstracts upgrades being opt-in only by default?

[leonardoalt]

This is great.

Would be curious to see a uniswap build using the Diamond pattern.

To me Uniswap is great exactly because it's not upgradeable/centralized.

[mudgen]

@leonardoalt It is different in a lot of ways, but it is not different in regards to ownership/authentication. Ownership/authentication is not part of this standard. The ownership/authentication can be implemented in a diamond any way anybody wants.

I realize now that the way EIP-2535 Diamonds is currently written it misleads people into thinking that EIP-2535 Diamonds specifies how authentication/ownership is implemented or should work. It doesn't. So I think I will change the text you quoted because it is misleading. It certainly does not have to be that way, it just could be that way, depending on how ownership/authentication is implemented.

But note that the standard does suggest a different way to do ownership or authentication. See the "Decentralized Authority" section.

Also, the upgrade functionality can be removed in a diamond making a diamond immutable. This could be done by removing the cut function. And it possibly could be added back depending on certain cases dictated by the implementation of the diamond -- there are so many possibilities to what could be implemented and EIP-2535 Diamonds does not limit what can be done.

Why not a standard that abstracts upgrades being opt-in only by default?

Standards or tutorials or implementations that build on top of EIP-2535 Diamonds to provide different ownership/authentication/upgrade schemes are very much wanted!

EIP-2535 Diamonds provides basic architecture and structure and points of interoperability with user interfaces and software, and it leaves up to the implementer what the diamond does and how it works.

[pi0neerpat]

@mudgen

I don't know why you think a diamond can't be controlled by a single source.

The only reason I say this is because the single source would just publish documentation whenever they made a change. There's no real reason to emitt changes as events or capture this on-chain. Maker and other protocols already have governance processes to document this already.

A big use case is contracts with designs that exceed the max size of contracts, since diamonds don't have a max size.

That's a very good feature!

Companies are already using this kind of architecture (ERC1538) for their contracts, such as Caesar's Triumph, Enjin and others. It is real and in use today

I'll have to look more closely at these. This could be really useful for @austintgriffith DAOG game where the rules of the game are updated on-the-fly. Right now the rules are limited in scope, but this could allow more open-ended rules to be added or removed.

Permissions and authentication for cuts and access to storage variables can be handled in the same way as any other contract.

I'm glad you think this is possible because I really think this is the best potential use-case for this system. For instance, in the DAOG game, you could allow players to add/remove rules, without letting them interfere with the core game logic (i.e. the logic and storage deciding who wins and can withdraw the prize pot). Or with a Moloch, you could set certain thresholds for high/low impact changes to the dao contract itself. Dao members could pass smaller rules more easily, to run segregated and quick experiments, without risking the dao's funds.

[leonardoalt]

the upgrade functionality can be removed in a diamond making a diamond immutable.

The upgrade functionality can also be removed by the contract not being upgradeable in the first place.

there are so many possibilities to what could be implemented and the diamond standard does not limit what can be done

Exactly my point. It is basically a really complicated way to make contracts as mutable as simply delegating everything to a dynamic address, which is actually a lot more transparent and simpler to read and check what it's doing. Complicated upgrade standards enable backdoors by obfuscation, especially when you can literally change everything.

[mudgen]

@leonardoalt Its is really not very complicated. It is new and the standard does provide a lot of information to help use and implement the standard.

EIP-2535 Diamonds is this

1. One function for adding/replacing/removing functions.
2. One event for logging changes.
3. Some functions for showing what functions the contract has.

A big part of the standard is transparency (logging changes), which removes obfuscation. I plan to make a user interface that shows and visualizes all changes to a diamond.

[mudgen]

Some good discussion of EIP-2535 Diamonds here: https://ethereum-magicians.org/t/diamond-contract-standard/4038

[spalladino]

Awesome work @mudgen!! I like this new version of the EIP, and have a few comments:

• I really like the fact that cut can modify several functions atomically, and you don't need multiple calls, potentially leaving your contract in an inconsistent state. That said, I'd look into simplifying it a bit: instead of having the concept of add/remove/replace action, why not just have cut accept an array of pairs selector,implementation, directly setting the implementation for each selector? Add and replace would be similar, and removal can be done by setting implementation to zero. Function clashes can be managed via appropriate tooling here.

• In line with the previous one, I'd also simplify the event. Having a big event with a bunch of actions is harder to consume. I'd just set up events with the selector and implementation arguments, and fire as many as needed. You could even make selector indexed, so clients can monitor for changes to a specific function (I'd look into gas tradeoffs here though).

• There is another potential clash that is event clashing. The DiamondCut event could have the same selector as an event from a facet, potentially fooling a client who is listening to these events. That said, a client could just query the contract to verify the facet address if needed.

• I'd also simplify the loupe interface. I'd remove any enumerable features, and just keep a getFacet(bytes4 selector) method. I'm not sure in which situations another contract would need to iterate over the facets of a diamond, and an offchain client could look into past events. I think this should also simplify the storage structures needed to support a diamond, making it overall more gas efficient.

[mudgen]

@spalladino Thank you very much. This is great feedback! I appreciate that you take the time to look at this and write this feedback.

Awesome work @mudgen!! I like this new version of the EIP, and have a few comments:

• I really like the fact that cut can modify several functions atomically, and you don't need multiple calls, potentially leaving your contract in an inconsistent state. That said, I'd look into simplifying it a bit: instead of having the concept of add/remove/replace action, why not just have cut accept an array of pairs selector,implementation, directly setting the implementation for each selector? Add and replace would be similar, and removal can be done by setting implementation to zero. Function clashes can be managed via appropriate tooling here.

I like the idea of simplifying the argument to cut. Your suggestion omits the string message part of the argument that describes a set of changes. It is a commit message. Removing it would simplify the argument to cut and reduce some gas. Do you think its disadvantages outweigh its benefits? The argument is as complicated as it currently is because it gives the ability to have different change messages for different sets of changes in one transaction. This helps document the changes, but maybe it isn't worth the added complexity of the data structure.

• In line with the previous one, I'd also simplify the event. Having a big event with a bunch of actions is harder to consume. I'd just set up events with the selector and implementation arguments, and fire as many as needed. You could even make selector indexed, so clients can monitor for changes to a specific function (I'd look into gas tradeoffs here though).

The argument to the cut function and the argument to the DiamondCuts event is exactly the same. The cut function literally passes its argument directly to the DiamondCuts event without any manipulation. This is a nice simplicity. You are right, it would be nicer if it was a simpler data structure. It associates the change messages with their changes. Again, if we remove the string message then this data structure could be simplified. So I guess the important question is how important is it to have descriptive text describing changes to a diamond emitted in events? I suppose the datastructure could also be simplified some by only allowing one change message for all changes in one transaction. With the way it is now you can have multiple change messages for different sets of changes in one transaction. That is why the data structure is as complicated as it is.

• There is another potential clash that is event clashing. The DiamondCut event could have the same selector as an event from a facet, potentially fooling a client who is listening to these events. That said, a client could just query the contract to verify the facet address if needed.

Can you explain this more? I don't understand the scenario you are describing here.

• I'd also simplify the loupe interface. I'd remove any enumerable features, and just keep a getFacet(bytes4 selector) method. I'm not sure in which situations another contract would need to iterate over the facets of a diamond, and an offchain client could look into past events. I think this should also simplify the storage structures needed to support a diamond, making it overall more gas efficient.

Wow, that is an interesting idea. I see what you are saying, the loupe isn't needed if software can simply look at the events to determine which functions exist. I'm not sure why getFacet(bytes4 selector) would be needed by the standard. The main purpose of the loupe is to provide interoperability with user interface software and tooling, but if events already give that then they could be used. People can always add in their own functions like getFacet(bytes4 selector) for their own purposes. I am considering removing the loupe completely from the standard.

@spalladino What do you think now?

[spalladino]

The commit message is definitely interesting, but shouldn't it be handled off chain? Perhaps the contract should just have a small identifier that points to an actual commit or release off-chain, for traceability from the source to the deployment?

Still, one option (following the address,selector pairs approach) would be for the cut event to publish the array of address,selector pairs, and the commit message, similar to what you originally proposed - though easier to parse for the client. You lose indexability of selectors, but it may not be too critical.

Can you explain this more? I don't understand the scenario you are describing here.

Sorry, forget about this one. I mixed up event topics and function selectors. The first topic for an event, which is derived from a hash of its name and args and identifies the type of event, is 32 bytes long -not 4 bytes like a function selector. So clashes between event names are not possible.

People can always add in their own functions like getFacet(bytes4 selector) for their own purposes. I am considering removing the loupe completely from the standard.

Maybe there is an opportunity to have this standard be automatically ERC165 compliant...? Haven't looked at it in-depth.

[mudgen]

The commit message is definitely interesting, but shouldn't it be handled off chain? Perhaps the contract should just have a small identifier that points to an actual commit or release off-chain, for traceability from the source to the deployment?

Have the identifier where? The commit messages are not stored in the contracts, they are just emitted with the event that shows the changes. I'd rather just emit the commit message in the event than emit a hash of a commit stored somewhere because I don't think this would be useful for user interfaces that show people all the changes to a diamond, but the commit messages describing the changes could be useful. The idea of the user interface is that it would pull all the verified source code from somewhere like etherscan so people could easily see all the source code of all the facets used by a diamond, and in addition people could see the verified source code of how a diamond was in the past if it was cut. And people would see the commit messages describing the upgrades, why they were done etc.

Maybe there is an opportunity to have this standard be automatically ERC165 compliant...? Haven't looked at it in-depth.

I don't see how it could be automatically compliant. I do like ERC165 and I think it is good for people to use it.

[fulldecent]

I recommend that this standard use ERC-165, just for the cut function. This will help tools, such as Etherscan, recognize compliant contracts.

---

Because this contract is VERY general in purpose, the cut function should be renamed to a more qualified name such as diamondCut. Also I would like to see a discussion of the proposed name and any potential function selector conflicts in the rational section.

---

The event has arrays in it, so this limits the ability to search logs.

---

The API is overly complex:

enum CutAction {Add, Replace, Remove}
struct FacetCut {
  address facet;
  CutAction action;
  bytes4[] functionSelectors;
}
struct DiamondCut {
  FacetCut[] facetCuts;
  string message;
}
interface Diamond {
  function cut(DiamondCut[] calldata _diamondCuts) external;
  event DiamondCuts(DiamondCut[] _diamondCuts);    
}

It can be:

struct DiamondBatchCuts {
  bytes4[] functionSelectors;
  address[] implementations;
}
interface Diamond {
  function diamondCut(DiamondBatchCuts calldata diamondCuts) external;
  event DiamondCut(bytes4 functionSelector, address oldImplementation, address newImplementation);
}

It is unnecessary to categorize adding, changing and removing. Simply, a zero address corresponds to no implementation and a non-zero address is an implementation.

---

Use extra bytes.

Nobody asked for this feature but I'll suggest it any way.

I assume the standard contract is implemented like:

contract DiamondImplementation {
  mapping (bytes4 => address) implementations;
  function diamondCut(DiamondBatchCuts calldata diamondCuts) external {
    for (uint i = 1; i < diamondCuts.functionSelectors.length, I++) {
      address old = diamondCuts.functionSelectors[I];
      implementations[diamondCuts.functionSelectors[i]] = diamondCuts.implementations[i];
      emit DiamondCut(diamondCuts.functionSelectors[I], old, diamondCuts.implementations[I]);
    }
  }
}

So this means the storage is mapping (bytes4 => address) implementations;. That is wasteful because you are only putting 160 bits in a 256 bit register.

You can store more... but what? The selector in the target contract!

So you can have a function selector a on your Diamond contract be implemented by function b on the implementation contract. Why would you want to do that? Because of code reuse.

OR you can ignore this suggestion entirely if all the implementation contracts are implemented using the fallback function, which is better.

E.g. specify that all implementations are:

interface DiamondFunctionImplementation {
    fallback () external {
      // code goes here
    }
}

^^ this will be more efficient.

I didn't actually read the EIP some maybe you already specified this.

---

Documentation on storage mutability is insufficient. This is a major design consideration. And as somebody that audits contracts I'll hate auditing this kind of contract :-~~~

---

Mutability is bad. As stated before in my prior related review. I'm still not a fan of using this EIP or Zeppelin OS for upgradeable contracts. If you want to upgrade your contract then best practice is to deploy a new contract and spend your marketing budget to inform everybody of the new version.

This is what I did, multiple times, when working on ERC-721. Since I published the first "ERC-721 compliant contract" (it's Su Squares, check it out) that means I needed to redeploy it every time there was a new ERC-721 draft. That's okay, and all of the wallet providers know me because I kept having to bother them to update the contract address in MetaMask, MyEtherWallet, etc. And that's a good thing.

An exception is zero-knowledge proof contracts. These require an enormous amount of code. And these require a limited caveat to my note above. It might be reasonable for a ZKP contract to be deployed in multiple stages. But the contract should not be open to the public until deployment is completed (dependent contracts are loaded) and no further changes should be possible after deployment. Even still, the functionality of Diamond contracts should not be necessary for this deployment strategy.

[mudgen]

@fulldecent I appreciate your feedback on this.

I recommend that this standard use ERC-165, just for the cut function. This will help tools, such as Etherscan, recognize compliant contracts.

Yes, I'll add this to EIP-2535 Diamonds.

Because this contract is VERY general in purpose, the cut function should be renamed to a more qualified name such as diamondCut.

I agree that cut should be renamed to a more qualified name. I'll rename it to diamondCut.

Also I would like to see a discussion of the proposed name and any potential function selector conflicts in the rational section.

There is a section about function selector conflicts in the security section.

The API is overly complex:

The argument to cut is complex because it supports descriptive messages describing changes, like commit messages. And different messages can be associated with different sets of changes. The argument can be simplified if we throw out the descriptive messages of the changes, but I think it is worth it to keep them. I understand that documentation messages are not sufficient for a security audit but I think they can help people understand diamonds and why/what changes are made.

An important part of EIP-2535 Diamonds is creating user interfaces that pull all the verified source code that is used and displaying it in such a way that a person can see and understand all the code that is currently used by a diamond and also look at past code that was used before it was cut.

It is unnecessary to categorize adding, changing and removing. Simply, a zero address corresponds to no implementation and a non-zero address is an implementation.

It is necessary to prevent function selector conflicts. That's why it is there. The alternative is to let the user or off-chain software first verify that they aren't making any function selector clashes before calling cut but a diamond can't enforce they do this and it may be easier for them to specify add,replace,remove and the diamond can enforce they do this.

contract DiamondImplementation {

I understand your DiamondImplementation and I understand that an address is 160 bits and that additional data can be stored in the 256 slot. But after that I am lost. But I am interested. Can you explain more?

As stated before in my prior related review.

I think I kinda remember your prior review but for some reason I can't find it. Do you happen to know where it is? Because I'd like to review it if we can find it. Nevermind I found it! It was an email to me and I found it.

I remember you working and handling Su Squares and I thought you did a good job with it and the way you did things with it was good.

[mudgen]

This change is complete: I renamed the cut function to diamondCut in the standard.

[mudgen]

Wow, that is an interesting idea. I see what you are saying, the loupe isn't needed if software can simply look at the events to determine which functions exist. I'm not sure why getFacet(bytes4 selector) would be needed by the standard. The main purpose of the loupe is to provide interoperability with user interface software and tooling, but if events already give that then they could be used. People can always add in their own functions like getFacet(bytes4 selector) for their own purposes. I am considering removing the loupe completely from the standard.

Okay, here's the truth, I just don't totally trust 100 percent that events will be available all the time, forever, and with good performance all the time, forever. Maybe that is dead wrong and hope it is wrong and I'd love someone to prove it to me that it is wrong so I am totally convinced. I want to be overly safe until then -- after all we are dealing with diamonds!. Having the loupe functions implemented is a very good guarantee that you will be able to inspect your diamonds for facets and functions. If it is implemented right and it doesn't work then that means ethereum contracts don't work anymore and we have bigger problems.

So I'm keeping the loupe functions in the standard. Actually I removed two of them: functionSelectorByIndex and totalFunctions, because I don't see the need for them with the other ones. Also, by having events and loupe functions it provides two different ways to get the functions and facets of a diamond. If one way breaks for some impossible reason, the other way is available.

[GNSPS]

I am going to support @leonardoalt fully here. Rare are the occasions where upgradeability on-chain cannot be replaced with off-chain mechanisms to achieve the same end result.

As an added problem, the more you complicate on-chain upgradeability mechanisms, the more obfuscated and less auditable these become. This means that clients' trust on the system is greatly reduced.

If everything is mutable why not just delegate execution?

[androolloyd]

I am going to support @leonardoalt fully here. Rare are the occasions where upgradeability on-chain cannot be replaced with off-chain mechanisms to achieve the same end result.

As an added problem, the more you complicate on-chain upgradeability mechanisms, the more obfuscated and less auditable these become. This means that clients' trust on the system is greatly reduced.

If everything is mutable why not just delegate execution?

I feel like this approach takes the stance that we're never going to improve the way we do things today.

The audit log itself ensures that no central party has to prove what the state is, as it's self managed.

As a user if you want any real trust with proxies, you want to have your own, anytime an app needs a proxy they deploy one for you, as a user, you could have A proxy that you trust, that you cut with any features that you need, without having to extend that trust to anyone else to verify what the state of your proxy is.

Governance maintained Diamonds with the ability to cut in new features seems like a huge boon in terms of how we manage and maintain upgradeability.

No question there are new security challenges to deal with, but these types of patterns work well in other application development to date.

[leonardoalt]

As a user you should be asked to opt-in an upgrade, not be forced to trust obfuscated code.

[androolloyd]

As a user you should be asked to opt-in an upgrade, not be forced to trust obfuscated code.

No disagreements there, which is why its great for user owned contracts.

[mudgen]

@leonardoalt @androolloyd @GNSPS One thing to keep in mind, which @fulldecent pointed out, is that this standard is very general, which also means that it is extremely flexible. It is easy to make the error of pegging this standard to a particular use case or to make assumptions about it.

As a user you should be asked to opt-in an upgrade, not be forced to trust obfuscated code.

@leonardoalt This standard is probably flexible enough to accommodate that. I'm interested in more details about how that could work.

I myself am very guilty of pegging this standard to a particular use case: "upgradeable contracts". This standard is useful for creating very useful immutable contracts that can't be upgraded. How so? Well there might be many ways (being so general and all) but I think of two really good use cases. But before I tell you the use cases let me tell you how to make useful immutable contracts with this standard.

The standard has been carefully edited to say that a diamond "uses" the diamondCut function. It does not say that the diamondCut function has to be added to the diamond as one of its functions. So this means that you can delegatecall to the diamondCut function in the constructor of the diamond to add all the functions needed by the diamond. The example in the standard shows how to call diamondCut in the constructor of a diamond. Since the diamondCut function is never actually added to the diamond the diamond is immutable with the same immutability and trust guarantees as a vanilla contract. My two use cases illustrate why this is useful:

1. If you have a contract that hits the max contract size limit and there is too much code or dependency upon storage variables to separate it out into regular contracts, then you can make it an immutable diamond (which has no max size). Cool thing about making it a diamond is that you still break your big contract into smaller contracts, modularizing your code to a degree. A good description of a diamond is this: A group of contracts that share the same storage variables and address.

2. You can start with an upgradeable diamond in your development and testing and upgrade it to your heart's delight. Reap the advantages of easy upgrading and a stable address as you work out new features, bugs and kinks. Release the upgradeable contract on a test network with your application for beta testing and upgrade it when needed. This is iterative development. When it is all solid then make it an immutable diamond and launch it on the main network.

Obfuscation exists when there are no tools to make something transparent and clear. This standard standardizes diamonds so that tools can be written for them so they are transparent and clear.

[mudgen]

@androolloyd I love this use case:

As a user if you want any real trust with proxies, you want to have your own, anytime an app needs a proxy they deploy one for you, as a user, you could have A proxy that you trust, that you cut with any features that you need, without having to extend that trust to anyone else to verify what the state of your proxy is.

I want everyone to have their own diamond.

[mudgen]

The only reason I say this is because the single source would just publish documentation whenever they made a change. There's no real reason to emitt changes as events or capture this on-chain. Maker and other protocols already have governance processes to document this already.

@pi0neerpat I realized that you are right. It is better to let people document their contracts and their upgrades off chain the way they want to. Thank you for this useful feedback. I updated the standard to reflect this. Specifically I removed the documentation message part of the input and event.

---

The commit message is definitely interesting, but shouldn't it be handled off chain? Perhaps the contract should just have a small identifier that points to an actual commit or release off-chain, for traceability from the source to the deployment?

@spalladino I realized that you are right. I removed the commit message functionality. I think that the facet addresses can be used as pointers to facet and upgrade documentation, in the same way they are used as pointers to verified source code on etherscan. Thank you for this very useful feedback.

I really like the fact that cut can modify several functions atomically, and you don't need multiple calls, potentially leaving your contract in an inconsistent state. That said, I'd look into simplifying it a bit: instead of having the concept of add/remove/replace action, why not just have cut accept an array of pairs selector,implementation, directly setting the implementation for each selector? Add and replace would be similar, and removal can be done by setting implementation to zero. Function clashes can be managed via appropriate tooling here.

@spalladino You are right about simplifying the argument to the diamondCut function. After removing the message part I was able to simplify it and I updated the standard.

The add/remove/place action is used to prevent function selector clashes. I don't feel comfortable with delegating that responsibility to tools because some tools might not do it. Also, having the action makes explicit what is happening which is good. I know it is redundant with events but someone can look at past transactions and know whether functions were added or replaced.

[mudgen]

The event has arrays in it, so this limits the ability to search logs.

@fulldecent You are right about the events. I changed the events in the standard to something very similar to what you suggested.

---

In line with the previous one, I'd also simplify the event. Having a big event with a bunch of actions is harder to consume. I'd just set up events with the selector and implementation arguments, and fire as many as needed. You could even make selector indexed, so clients can monitor for changes to a specific function (I'd look into gas tradeoffs here though).

@spalladino You are right about events. I updated the standard with events like you suggested. I am leery about making the function selector indexed because of the extra gas cost and how useful it might be.

[pi0neerpat]

The add/remove/place action is used to prevent function selector clashes. I don't feel comfortable with delegating that responsibility to tools because some tools might not do it. Also, having the action makes explicit what is happening which is good. I know it is redundant with events but someone can look at past transactions and know whether functions were added or replaced.

Not sure if this will help, but I made a tool to assist in stripping out variables for both turning a regular contract into a proxy, and to help when performing an upgrade. It gathers all variables from all contracts (so dev can write new code as normal, without thinking of proxy) and throws them into Storage.sol. The dev can then just git diff to make sure variables are appendend and not reorderes. In this case, it's the EIP1822 Universal Upgradeable Proxy Standard.

https://www.npmjs.com/package/sol-proxy

[mudgen]

@pi0neerpat sol-proxy is interesting, thank you.

[mudgen]

How can we isolate storage on the same permissioned system that we use to allow cuts?

@pi0neerpat I am glad you ask this because it is a very good question. I've thought about it a lot. The answer is we can't. If a person can add/replace functions then he/she can alter storage willy nilly. But the story does not end there. There are things that can be done to eliminate, reduce or limit the danger to storage. I just added a new subsection to the standard called, "Security of Diamond Storage" to address this.

[Droopy78]

Love this. One quick comment: I don't see any issue with leaving the upgrade message string in the event. If worried about gas, simply leave it blank (I did this many times when upgrading my ERC-1538 functions). Reasons you may want the message logged on chain is if you are upgrading/cutting a number of functions in different contracts one at a time over months. Maybe you are fixing a minor bug in a function and forget to comment the in the function header the reason for the upgrade, or maybe from an operational perspective you may not remember to log all new code updates to etherscan for example. Thus the upgrade message may become useful tool for documenting the reason for the upgrade (and again, you have the option to leave the message blank if you choose).

[mudgen]

Note: @Droopy78 is an experienced developer with ERC1538 in production.

@Droopy78 yes, I hear you. I have gone back and forth about it a lot. I also developed ERC1538 contracts and I liked having the commit messages. I take what you say with strong consideration.

An earlier version of the diamond standard had the message string. Here's what it looked like:

struct FacetCut {
    address facet;
    CutAction action;
    bytes4[] functionSelectors;
}

struct DiamondCut {
    FacetCut[] facetCuts;
    string message;
}

interface Diamond {
    event DiamondCuts(DiamondCut[] _diamondCuts);    
    function cut(DiamondCut[] calldata _diamondCuts) external;
}

This was really nice because it enabled you to put in documentation messages in multiple places within the cut.

The diamond standard is different than ERC1538 in that the updateContract function can only work with one delegate contract at a time, but the diamondCut can work with multiple facets at a time. I've found that adding the message string to the diamondCut argument in a good way complicates the argument. But this in itself is not a good reason not to do it.

@spalladino and @pi0neerpat have mentioned that documentation of upgrades could be handled off chain. I've really considered each side of this. My personal opinion is that contract documentation and documentation of upgrades should be done off chain and should be editable so it can be added and improved over time. Just like there is an etherscan API for retrieving the verified source code and ABI of a contract I think there should be an API for retrieving the documentation, including upgrade documentation for contracts and diamonds. And the documentation should be able to be edited so it can be improved and updated, not fixed in time.

I don't want this standard to enforce people to do something that isn't absolutely necessary for interoperability with tools. At the same I don't want the standard to stop people from having things the way they want them. So I'm making the standard give what I think is absolutely necessary to be standardized but at the same time giving the freedom to do more and enable people to make things the way they want them.

Freedoms of note:

1. The standard does not prevent people from adding and using their own custom functions for adding/replacing/removing functions from a diamond. They should still provide and implement the standard diamondCut function if their diamond is not immutable for interchangeability reasons if the person ever wants their diamond to work with other tools that add/replace/remove functions.
2. People can emit other events in the implementation of the diamondCut function and add whatever custom functionality they want to it.

[Droopy78]

@mudgen My comment was making the assumption that each update (even one that contains multiple diamandCuts across multiple contracts) should be in relation to only one change, and thus only one commit message would be needed per update/commit (instead of one per diamondCut, which I agree is too complicated). It is similar to committing changes to github. Good practice is to commit a group of files that relate to one feature or bug fix at a time. Yes, you can commit 5 unrelated features or bug fixes with one commit, but it's tough to unravel later if you have to roll back one of those changes.

That being said, I am ok with the decision to leave the commit message out of the standard.

[mudgen]

@Droopy78 Yes, that make sense. I see what you mean.

[mudgen]

The event has arrays in it, so this limits the ability to search logs.

@fulldecent How so? Please explain.

[wjmelements]

The event has arrays in it, so this limits the ability to search logs.

@fulldecent How so? Please explain.

You can only filter by indexed event topics. You can index up to 4 event topics, and arrays are not indexable. Usually the event type is the first (0th) topic.

[mudgen]

@wjmelements Thanks, that is what I was thinking but wanted to make sure that @fulldecent wasn't referring to something else.

[nfurfaro]

I'm not certain about this, but I believe the Argent wallet has taken a modular approach to upgrades, where a small piece of the wallet functionality can be upgraded while leaving most of the contracts untouched. In addition, wallet users need to opt in to use the upgraded portions of code. While this is probably out of scope for this standard, it does demonstrate a way to make use of this type of architecture, while providing a guarantee to users that things won't change unexpectedly.

[mudgen]

@nfurfaro I think that is a great example.

[mudgen]

Somebody asked me if they should use the Diamond Standard or ERC1538. Here is my answer:

Definitely go with the Diamond Standard. Here are some reasons why:

1. The function that is used by the Diamond Standard to add/replace/remove functions is more capable than in ERC1538. In the Diamond Standard this function, called diamondCut, can add/replace/remove multiple functions from multiple facets in one transaction. This ability enables you to make all changes necessary without your diamond being left in a temporary incomplete state. In contrast the same function in ERC1538 can add/replace/remove multiple functions from only one facet in one transaction.

2. The diamondCut function uses less gas than the equivalent function in ERC1538.

3. More tools and support can be expected for the Diamond Standard. For example there is a plan to add special deployment support to Truffle for diamonds: https://twitter.com/mudgen/status/1246991530478501889 My personal marketing, support and work is with the Diamond Standard. I consider ERC1538 deprecated. The author of ERC1538 agrees with this.

4. ERC1538 was and is great. Lessons learned from it and the improvements in the Solidity language have been applied to the Diamond Standard.

That is awesome what you are doing. I can definitely see a diamond as the glue that connects everything together.

[hiddentao]

Just implemented this for our project and it went quite well -> https://hiddentao.com/archives/2020/05/28/upgradeable-smart-contracts-using-diamond-standard

One suggestion I'd like to make is that instead of DiamondExample manually configuring the method selectors, have an interface for fetching method selector info from facets such that facets themselves tell the proxy what methods they expose (thereby reducing coupling between these components):

interface IDiamondFacet {
  function getSelectors () external pure returns (bytes memory);
}  

Example facet using it:

contract TestEntityFacet is IDiamondFacet {
  function getSelectors () public pure override returns (bytes memory) {
    return abi.encodePacked(
      TestEntityFacet.getNumPolicies.selector
    );
  }
  function getNumPolicies() public view override returns (uint256) {
    return 666;
  }
}

Then you could have a method which iterates through passed-in facets to aggregate this info and pass it onto diamondCut():

  function _registerFacets (address[] memory _facets) internal {
    bytes[] memory changes = new bytes[](_facets.length);
    for (uint i = 0; i < _facets.length; i += 1) {
      IDiamondFacet f = IDiamondFacet(_facets[i]);
      bytes memory selectors = f.getSelectors();
      changes[i] = abi.encodePacked(_facets[i], selectors);
    }

    bytes memory cutFunction = abi.encodeWithSelector(Diamond.diamondCut.selector, diamondCut);
    (bool success,) = address(diamondFacet).delegatecall(cutFunction);
    require(success, "Adding functions failed.");        
  }
}

As outlined in the blog post we also had to make adjustments to how diamondCut() gets called in order to control access to the upgrade mechanism.

Perhaps the above ideas can be incorporated into the spec to make implementing the standard easier for people?

[fulldecent]

I had this sitting around for some reason...

---

1. Here's an interface:

interface ERC2535 /* is ERC165 */ {
  struct implementation {
    bytes4 selector;
    address target;
  }

  // Emitted for each changed implementation
  event DiamondCut(bytes4 functionSelector, implementation oldImplementation, implementation newImplementation);

  // Message is not in the event. Intentional. You could look it up in the chain data if needed. Expected not needed.
}

3. Here is THE implementation:

contract Diamond is ERC2535 {
  struct implementation {
    bytes4 selector;
    address target;
  }
  mapping (bytes4 => implementation) functionImplementations;
  address reserved;

  fallback() external payable {
    require(functionImplementations[msg.sig], "Function is not implemented");
    assembly {
      //TODO: Call the function...
      switch result
      case 0 {revert(ptr, size)}
      default {return (ptr, size)}
    }
  }

  function diamondCut(
    bytes4[] functionSelectors,
    implementation[] oldImplementations,
    implementation[] newImplementations
    string calldata message
  ) internal {
    for (uint i = 1; i < functionSelectors.length, I++) {
      bytes4 selector = functionSelectors[i];
      implementation old = oldImplementations[i];
      implementation new = newImplementations[i];
      require(old == functionImplementations[selector]);
      functionImplementations[selector] = new;
      emit DiamondCut(selector, old, new);
    }
  }
}

[mudgen]

@fulldecent Thanks for sharing that. Your implementation is the basic idea of the standard.

This standard is moving along. It was recently popular on Reddit: https://www.reddit.com/r/ethereum/comments/gze6k3/a_diamond_is_a_set_of_contracts_that_can_access/

Some people are using the standard and its precursor (ERC1538) and some are expressing interest in using it. Today a new Python tool for automating the upgrade process for diamonds was released. Here is an article about that: http://joeyzacherl.com/2020/06/diamond-setter-ethereum-smart-contract-manager/

And here is the tool: https://github.com/lampshade9909/DiamondSetter

Here's a recent article about the Diamond Standard by someone who is using it in production: https://hiddentao.com/archives/2020/05/28/upgradeable-smart-contracts-using-diamond-standard

[iamdefinitelyahuman]

I understand that marketing can be important, but I feel it's in poor taste to open near-identical issues on so many repos in such a short period. This is the first time I've seen what feels like spam on Github. I think you might be doing yourself a disservice here.

[mudgen]

@iamdefinitelyahuman You are right, I am sorry about this. Any ideas on what I should do to fix this and find a better way to let tools know about the Diamond Standard?

[fulldecent]

Here is how to properly "spam" other projects to adopt a standard:

1. Identify projects that might benefit from your standard
2. Pay somebody to implement on each of those projects
3. Submit and PRs and start discussion

Do things that don't scale...

[mudgen]

That makes sense. Thanks @fulldecent.

[mudgen]

Rare are the occasions where upgradeability on-chain cannot be replaced with off-chain mechanisms to achieve the same end result.

As an added problem, the more you complicate on-chain upgradeability mechanisms, the more obfuscated and less auditable these become. This means that clients' trust on the system is greatly reduced.

If everything is mutable why not just delegate execution?

@GNSPS The Diamond Standard has been modified and improved since you first wrote about it. Please take another look.

Diamonds are not just an upgrade mechanism. They are a way to organize Solidity code to achieve cohesion and modularity. Check out my new article about diamonds: https://dev.to/mudgen/understanding-diamonds-on-ethereum-1fb

Diamonds don't have to be upgradeable, or the upgrade capability can be removed. The Single Cut Diamond pattern that is written about in the standard is a nice way to do iterative contract development and/or testing on a test network. Do all your upgrades during development and testing and then deploy an immutable Single Cut Diamond on mainnet. It has the same immutability guarantees as a regular contract.

Diamonds are flexible and capable when it comes to upgrading them. But the DiamondCut event tracks and records every upgrade. So when you are auditing a diamond you can use this list of events to see what has changed and when it has changed.

I think that how much work it takes to audit an upgradable contract is related to how much is changed over time. If many things are changed then there is more work to see those changes and how they have affected contract storage.

So a diamond could be easy to audit if it hasn't changed much or more difficult to audit if it has changed a lot. But this is the same for any kind of upgradeable contract.

But with diamonds you are working with a proxy contract that has its contract storage modified by multiple facets. Each facet has its own source code. So when auditing a diamond you have to see how the source code of the different facets affects the contract storage of the proxy and you have to see how the different facets might interact with each other via the proxy contract storage. I agree this is more complicated to audit than a single contract. With more flexibility more complexity is possible. There are tradeoffs here.

[JulesGoddard]

If I have assimilated this correctly, the diamond standard implements most of the following model of a router with 2 contracts.

Each contract may have its own data (i.e. DataR, Data1, Data2) and share data with other contracts (i.e. Data12) or with other contracts and the router (DataX).

Any shared data can be located in memory using the mapping of a hash technique which is imported into all the using contracts.

This is very much like the memory model of an embedded system with a scheduler and 2 processes.

Data12 would contain the inter-process queues.

DataX would contain system gauges/whiteboards and logging.

Data1 and Data2 would contain contract specific data.

I think that the similarity effectively validates your design pattern.

I see the big difference being the DiamondCut function you are proposing, which is an implementation choice.

[mudgen]

@JulesGoddard Yes, this is a fantastic perspective and example of a diamond.

Curious, did you make that diagram or is that part of an existing system?

The purpose of the diamondCut function is to provide compatibility among tools that help make and form diamonds.