Soft-fork Mechanism

[Isaac-DeFrain]

RChain blessed contract upgrades (i.e. soft-fork mechanism)

Contract standard

• all contract state elements must have an accompanying extraction method
  • e.g. the Registry contract uses TreeHashMap to store contract uris, we will need to make this uri map available for peeks
• must take old state elements as arguments and initialize corresponding new state elements with them
  • e.g. when updating the Registry, we need to take the current uri map and make it available in the updated Registry
• for a given contract, all methods should have the same unforgeable name and be differentiated by a string in the first arg
  • e.g. instead of having one method with unforgeable name a and one with unforgeable name b, we make them contract unf(@"a", ...) = {...} and contract unf(@"b", ...) = {...}, respectively, for some fixed unforgeable name unf
  • this will give us the ability to dispatch all method calls through unf
• must provide api (client method names and number of args)
  • e.g. when updating the Registry, we must ensure that every method used on the Rev vault map in other contracts is also available in the new contract
  • this will give us a soft guarantee that the new contract is backwards-compatible

Dynamic dispatch

Each system (blessed) contract will exist as data on a fixed location channel corresponding to the contract. E.g. if C is a blessed contract, then we add a level of indirection through dynamic dispatch by

contract C(arg1, ..., argN) = {
  for (realC <<- cLocation) {
    realC!(arg1, ..., argN)
  }
}

The cLoction channel will be accessible only through a multisig contract in the registry which the coop will have keys to. This indirection buys us the flexibility to update a contract by simply extracting all state elements from the old contract, initializing the new contract's state elements with them, and updating the data stored on the location channel through a quorum of multisig public key agreements.

In the registry uri map, instead of directly mapping a blessed contract's shorthand to the contract's uri, we map the shorthand to a dispatcher contract which gets the data from the corresponding location channel and calls that contract with the supplied arguments. E.g. if C is a blessed contract, then it will have an accompanying dispatcher contract to dispatch calls

contract C(arg1, ..., argN) = { ... } |
cLocation!(*C) |
contract cDispatcher(arg1, ..., argN) = {
  for (realC <<- cLocation) {
    realC!(arg1, ..., argN)
  }
}

Previously, when we added this contract to the registry, we simply did an insertSigned with bundle+{*C}. Now, we will do an insertSigned with bundle+{*cDispatcher}. Hence, the registry uri map will contain the key rho:registry:c (for example) and value (max_int, bundle+{*cDispatcher}).

Requiring all methods in a blessed contract to be of the form

contract contractName(@"methodName", arg1, ..., argN) = {...}

for a fixed unforgeable name contractName, will make it so that we only need to manage contractName. All method calls will be dispatched in the same way.

Location channels

The channels which serve as a protected store for blessed contract data will be generated as unforgeable names in the original instance of the corresponding contract. In this original instance, the location channels will be passed to the multisig contract for further management via insertBlessed. Calling insertBlessed simply updates the blessedContractLocationMap which the multisig controls. There is one insertBlessed consume for each blessed contract to prevent any other contracts from being added.

MultiSig

The multisig contract is declared in the registry and gives privileged access to propose, agree, and update methods. This contract is used to manage the data stored on the blessed contract location channels. The methods

• propose: allows any of the privileged public keys to propose new data (con, meth) to store on a blessed contract location channel (i.e. an update) where con is the unforgeable name of the new contract and meth is the unforgeable name for the new contract's methods
  • propose(@pubKey, @uri, @con, @meth, @sig, ret)
• agree: allows the privileged public keys to "agree" with a proposal
  • agree(@pubKey, @uri, @con, @meth, @sig, ret)
• update: once there is a quorum of privileged keys agreeing on a proposal, this method will update the data on the corresponding location channel
  • update(@uri, ret)

Proof of Concept

// -----------------------------------------
// --- Blessed Contract Update Mechanism ---
// -----------------------------------------
new
  a, b,                   // a few blessed contracts
  aDispatcher,            // a's dispatcher contract
  newA,                   // an update for contract a
  newAMethod,
  insertBlessed,
  blessedContractLocMapCh,
  MultiSig,
  msMethodsRet,
  msRet,
  stdout(`rho:io:stdout`)
in {
  match Set("A", "B", "C") {
    pubKeys => {
      // blessedContractLocMap: uri-shorthand -> location
      blessedContractLocMapCh!({}) |
      // initialize blessed contract `a` data
      for (@uri, loc, @data, @sig, ack <- insertBlessed;
           @blessedContractLocMap      <- blessedContractLocMapCh) {
        // link uri with location channel
        blessedContractLocMapCh!(blessedContractLocMap.set(uri, *loc)) |
        // store contract data on location channel
        loc!(data) |
        ack!()
      } |
      // initialize blessed contract `b` data
      for (@uri, loc, @data, @sig, ack <- insertBlessed;
           @blessedContractLocMap      <- blessedContractLocMapCh) {
        // link uri with location channel
        blessedContractLocMapCh!(blessedContractLocMap.set(uri, *loc)) |
        // store contract data on location channel
        loc!(data) |
        ack!()
      } |
      // -------------------------------------------------------------------------------------
      // MultiSig enables similiar functionality to a multisig vault. 
      // pubKeys = set of public keys which have the privilege to propose and approve upgrades
      // quorumSize = number of pubKeys member approvals needed to upgrade a contract's data
      // -------------------------------------------------------------------------------------
      MultiSig!(pubKeys, 2, *msMethodsRet, *msRet) |
      contract MultiSig(@pubKeys, @quorumSize, methodsRet, msRet) = {
        new
          multisig,       // MultiSig contract's method entry point
          agreementMapCh, // channel on which the agreement map is stored
          proposeMapCh    // channel on which the propose map is stored
        in {
          // Initialize agreement map
          // (uri, contractData, methodData) -> agreement set
          agreementMapCh!({}) |
          // Initialize propose map
          // uri -> (contractData, methodData)
          proposeMapCh!({}) |
          // -------
          // Propose
          // --------------------------------
          // Privileged public keys, i.e. members of `pubKeys`, can propose contract updates.
          // There is only one proposal per uri possible at a time.
          contract multisig(@"propose", @pubKey, @uri, @con, @meth, @sig, ret) = {
            // TODO verify sig of (uri, con, meth)
            if (pubKeys.contains(pubKey)) {
              // `pubKey` has the privilege to propose updates
              for (@bcMap <<- blessedContractLocMapCh) {
                if (bcMap.contains(uri)) {
                  // `uri` belongs to a blessed contract
                  match (uri, con, meth) {
                    key => {
                      for (@proposeMap <- proposeMapCh) {
                        if (not proposeMap.contains(uri)) {
                          // the update proposal is unique
                          proposeMapCh!(proposeMap.set(uri, (con, meth))) |
                          for (@agreeMap <- agreementMapCh) {
                            // the proposer is the first to agree with a proposal
                            agreementMapCh!(agreeMap.set(key, Set(pubKey))) |
                            ret!((true, uri, con, meth))
                          }
                        } else {
                          // an update has already been proposed for this uri
                          proposeMapCh!(proposeMap) |
                          ret!((false, "location already exists"))
                        }
                      }
                    }
                  }
                } else {
                  // `uri` does not belong to a blessed contract
                  ret!((false, "uri does not exist"))
                }
              }
            } else {
              // `pubKey` does not have the privilege to propose updates
              ret!((false, "invalid public key"))
            }
          } |
          // -----
          // Agree
          // --------------------------------------------------------------------------------------
          // Privileged public keys can agree with update proposals.
          // Manages the `agreementMap`: (uri, contractData, methodData) -> set of agreeing pubKeys
          // --------------------------------------------------------------------------------------
          contract multisig(@"agree", @pubKey, @uri, @con, @meth, @sig, ret) = {
            match (uri, con, meth) {
              agreeTruple => {
                if (pubKeys.contains(pubKey)) {
                  // TODO verify sig of (uri, con, meth)
                  for (@map <- agreementMapCh) {
                    match map.getOrElse(agreeTruple, Set()).add(pubKey) {
                      agreeing => {
                        agreementMapCh!(map.set(agreeTruple, agreeing)) |
                        ret!((true, uri, con, meth, agreeing))
                      }
                    }
                  }
                } else {
                  // pubKey is not in pubKeys
                  ret!((false, "invalid public key"))
                }
              }
            }
          } |
          // ------
          // Update
          // ------------------------------------------------------------------------------------
          // if there is a quorum of privileged public keys agreeing on the update for `uri`
          // then this method updates the contract data and manages the internal maps accordingly
          // ------------------------------------------------------------------------------------
          contract multisig(@"update", @uri, ret) = {
            for (@proposeMap <- proposeMapCh) {
              if (proposeMap.contains(uri)) {
                for (@blessedContractLocMap <<- blessedContractLocMapCh) {
                  match (blessedContractLocMap.get(uri), proposeMap.get(uri)) {
                    (loc, (con, meth)) => {
                      for (@agreementMap <- agreementMapCh) {
                        if (agreementMap.getOrElse((uri, con, meth), Set()).size() >= quorumSize) {
                          // sufficiently many keys agree to update
                          // consume data on location channel in order to replace contract data
                          for (oldData <- @loc) {
                            new tmp, newARet in {
                              oldData!("extractState", *tmp) |
                              for (@oldState <- tmp) {
                                // launch new contract instance with initial state extracted from the old instance
                                @con!(oldState, *newARet) |
                                // manage agreement and propose maps
                                agreementMapCh!(agreementMap.delete((uri, con, meth))) |
                                proposeMapCh!(proposeMap.delete(uri)) |
                                // write new method entry point to location channel
                                @loc!(meth) |
                                ret!((true, *newARet))
                              }
                            }
                          }
                        } else {
                          agreementMapCh!(agreementMap) |
                          proposeMapCh!(proposeMap) |
                          ret!((false, "quorum does not exist"))
                        }
                      }
                    }
                  }
                }
              } else {
                proposeMapCh!(proposeMap) |
                ret!((false, "invalid proposal uri"))
              }
            }
          } |
          // Read
          // --------------------------------------------
          // Returns a map containing the current maps:
          // "blessed"   - blessed contract location map
          // "agreement" - agreement map
          // "propose"   - proposals map
          // --------------------------------------------
          contract multisig(@"read", ret) = {
            for (@agreementMap <<- agreementMapCh;
                 @blessedMap   <<- blessedContractLocMapCh;
                 @proposeMap   <<- proposeMapCh) {
              ret!({ "blessed" : blessedMap, "agreement" : agreementMap, "proposals" : proposeMap })
            }
          } |
          methodsRet!(bundle+{*multisig})
        } |
        msRet!((bundle+{*MultiSig}, { "pubKeys" : pubKeys, "quorumSize" : quorumSize }))
      } |
      // a blessed contract in the registry which will not updated in this example
      b!() |
      contract b() = {
        new bMethod, bLoc, ack in {
          insertBlessed!(`rho:registry:b`, *bLoc, bundle+{*bMethod}, Nil, *ack)
          // insert arbitray contract code...
        }
      } |
      // a blessed contract in the registry which we intend to update
      contract a(@val1, @val2, ret) = {
        new aMethod, aDispatcher, aLoc, state1, state2 in {
          // original instantiation of contract data
          state1!(val1) |
          state2!(val2) |
          contract aMethod(@"set1", @val, ack) = {
            for (_ <- state1) {
              state1!(val) |
              ack!()
            }
          } |
          contract aMethod(@"set2", @val, ack) = {
            for (_ <- state2) {
              state2!(val) |
              ack!()
            }
          } |
          contract aMethod(@"read", ret) = {
            for (@val1 <<- state1; @val2 <<- state2) {
              ret!((val1, val2))
            }
          } |
          contract aMethod(@"extractState", ret) = {
            for (@st1 <<- state1; @st2 <<- state2) {
              ret!((st1, st2))
            }
          } |
          // Dispatcher contract for `a`
          contract aDispatcher(@arg1, @arg2) = {
            for (realA <<- aLoc) {
              realA!(arg1, arg2)
            }
          } |
          contract aDispatcher(@arg1, @arg2, @arg3) = {
            for (realA <<- aLoc) {
              realA!(arg1, arg2, arg3)
            }
          } |
          // initialize original contract data
          new ack in {
            insertBlessed!(`rho:registry:a`, *aLoc, bundle+{*aMethod}, Nil, *ack) |
            for (<- ack) {
              ret!(bundle+{*aDispatcher})
            }
          }
        }
      } |
      // updated contract to replace the old one
      // - contract updates do not get a location or dispatcher
      // - location and dispatcher are created in the original contract instance
      contract newA(@oldState, ret) = {
        new state1, state2 in {
          // initialize new state channel with old state
          state1!(oldState.nth(0)) |
          state2!(oldState.nth(1)) |
          contract newAMethod(@"set1", @val, ack) = {
            for (_ <- state1) {
              state1!(val) |
              ack!()
            }
          } |
          contract newAMethod(@"set2", @val, ack) = {
            for (_ <- state2) {
              state2!(val) |
              ack!()
            }
          } |
          contract newAMethod(@"modify", ack) = {
            for (_ <- state1; _ <- state2) {
              state1!("new state 1") |
              state2!("new state 2") |
              ack!()
            }
          } |
          contract newAMethod(@"read", ret) = {
            for (@val1 <<- state1; @val2 <<- state2) {
              ret!((val1, val2))
            }
          } |
          contract newAMethod(@"extractState", ret) = {
            for (@val1 <<- state1; @val2 <<- state2) {
              ret!((val1, val2))
            }
          } |
          // upon successful multisig operations, the data on `aLoc` is replaced with bundle+*{newAMethod}
          ret!(bundle+{*newAMethod})
        }
      } |
      // Scenario
      // --------------------------------------------------------------
      // 1. The pubKeys member "A" will propose an update: bundle+{*newA}, bundle+{*newAMethod}, to contract `a`.
      // 2. Then the pubKeys member "B" will agree with the proposal.
      // 3. Then some "Rando" proposer makes a proposal and it is rejected.
      // 4. Then `a` is updated to the data originally proposed by "A".
      // --------------------------------------------------------------
      new
        ack,
        aCh,
        randoContract,
        randoMethod
      in {
        // instantiate original `a` contract
        // this would happen during the creation of the genesis block
        // or during an update
        a!("old state 1", "old state 2", *aCh) |
        for (_ <<- aCh) {
          for (@oldBcMap <<- blessedContractLocMapCh) {
            // get the MultiSig method entry point
            for (ms <- msMethodsRet) {
              new ret, ret1, tmp in {
                match (`rho:registry:a`, bundle+{*newA}, bundle+{*newAMethod}) {
                  (uri, newContractData, newMethodData) => {
                    // "A" proposes an update for `a`
                    ms!("propose", "A", uri, newContractData, newMethodData, Nil, *ret) |
                    for (@(true, _, _, _) <- ret) {
                      ms!("read", *ret1) |
                      for (@m <- ret1) {
                        match m.get("agreement") {
                          am => {
                            // Only "A" has made a proposal and hence only "A" has agreed on any proposal
                            stdout!(("proposal implies agreement", am.get((uri, newContractData, newMethodData)) == Set("A")))
                          }
                        }
                      } |
                      // "B" agrees with the proposal
                      ms!("agree", "B", uri, newContractData, newMethodData, Nil, *ret) |
                      for (@(true, _, _, _, _) <- ret) {
                        new oldMapsCh, newMapsCh in {
                          ms!("read", *oldMapsCh) |
                          for (@oldMaps <- oldMapsCh) {
                            // just for fun: some rando tries to propose an update for `a`
                            ms!("propose", "Rando", uri, bundle+{*randoContract}, bundle+{*randoMethod}, Nil, *ret) |
                            for (@(false, _) <- ret) {
                              // "A" attempts to agree with their own proposal.
                              // This vote is not counted again; "A" voted for the proposal by proposing it.
                              ms!("agree", "A", uri, newContractData, newMethodData, Nil, *ret) |
                              for (_ <- ret) {
                                ms!("read", *newMapsCh) |
                                for (@newMaps <- newMapsCh) {
                                  // check that invalid proposals and agreements do not the corresponding maps
                                  stdout!(("invlaid proposer does not change the proposal map", oldMaps.getOrElse("proposals", 0) == newMaps.getOrElse("proposals", 1))) |
                                  stdout!(("pubKeys cannot agree more than once with a proposal", oldMaps.getOrElse("agreement", 0) == newMaps.getOrElse("agreement", 1)))
                                } |
                                // both "A" and "B" have agreed to update `a`, quorumSize = 2
                                // so we can update `a`
                                ms!("update", `rho:registry:a`, *ret) |
                                for (_ <- ret) {
                                  for (@bcMap <<- blessedContractLocMapCh) {
                                    // get updated contract's method data
                                    for (newMeth <<- @{bcMap.get(`rho:registry:a`)}) {
                                      // contract data is correctly updated
                                      stdout!(("after update new data is stored on location channel", *newMeth == {bundle+{*newAMethod}})) |
                                      // no contract locations should be changed during the update
                                      stdout!(("blessed location map is unchanged", oldBcMap == bcMap)) |
                                      // the proposal and agreement maps should be empty after the update
                                      ms!("read", *ret) |
                                      for (@m <- ret) {
                                        stdout!(("multisig maps should be empty", m.get("proposals") == {} and m.get("agreement") == {}))
                                      } |
                                      // check that the new contract's state is correctly initialized
                                      // and apply a method which was not available in the old contract
                                      newMeth!("read", *tmp) |
                                      for (@v <- tmp) {
                                        stdout!(("correct initial state", v == ("old state 1", "old state 2"))) |
                                        // since the contract's data has been updated,
                                        // we can call a method that's only available in the new contract
                                        newMeth!("modify", *tmp) |
                                        for (<- tmp) {
                                          newMeth!("read", *tmp) |
                                          for (@v <- tmp) {
                                            stdout!(("correct new state", v == ("new state 1", "new state 2")))
                                          }
                                        }
                                      }
                                    }
                                  }
                                }
                              }
                            }
                          }
                        }
                      }
                    }
                  }
                }
              }
            }
          }
        }
      }
    }
  }
}

Drawbacks

Extra comm events are required to interact with the affected contracts.

[zsluedem]

After some attempts at trying the solutions above, the current solution seems to be an OK temporary solution for now because we don't have too many libraries to look into and we don't have too many rholang developers who have real contracts on-chain and the developers use the library in the wrong way.

However, there are still great limits and difficulties(dependency hell would happen).

The more perfect way to is have a build system which could help on building rholang like sbt, yarn and even babel-like tools to generate linking rholang.