search

MystenLabs / sui

Sui, a next-generation smart contract platform with high throughput, low latency, and an asset-oriented programming model powered by the Move programming language
https://sui.io
Apache License 2.0
5.98k stars 11.09k forks source link

[Move] Third-Party Package upgrades #2045

Open sblackshear opened 2 years ago

sblackshear commented 2 years ago

Principles

Package Upgrades offer builders a way to evolve packages, bringing value to users by adding features or fixing bugs. However, if not carefully designed, those same tools can be used to exploit user trust. Package Upgrades on Sui are designed according to the following principles to limit the potential for exploits without limiting the features available to builders:

Technical Constraints

There are also technical details specific to Sui and Move that impact the design:

Summary of Protocol Changes

A package upgrade will introduce a new object on-chain (T1) which can access objects that were created by older versions of that package. Although it is an object at a new ID, its version field will track its package version.

An upgraded package must be compatible with past versions of itself (See "What can I Upgrade?" below), to allow packages that were originally published calling an older version of that package to be able to run against newer versions.

Upgrades are purely additive, they do not modify any existing on-chain state:

Upgrades are be governed by policies which are defined as Move code that works with newly introduced types (see "Summary of Framework Changes", below):

Summary of Framework Changes

 Upgrades will introduce a new package to the framework package.move, containing `UpgradeCap`, `UpgradeTicket`, `UpgradeReceipt`, and their associated API: ```rust module sui::package { use sui::object::{Self, ID, UID}; /// Capability controlling the ability to upgrade a package. struct UpgradeCap has key, store { id: UID, /// (Mutable) ID of the package that can be upgraded. package: ID, /// (Mutable) The number of upgrades that have been applied /// successively to the original package. Initially 0. version: u64, /// What kind of upgrades are allowed. policy: u8, } /// Permission to perform a particular upgrade (for a fixed version of /// the package, bytecode to upgrade with and transitive dependencies to /// depend against). /// /// An `UpgradeCap` can only issue one ticket at a time, to prevent races /// between concurrent updates or a change in its upgrade policy after /// issuing a ticket, so the ticket is a "Hot Potato" to preserve forward /// progress. struct UpgradeTicket { /// (Immutable) ID of the `UpgradeCap` this originated from. cap: ID, /// (Immutable) ID of the package that can be upgraded. package: ID, /// (Immutable) The policy regarding what kind of upgrade this ticket /// permits. policy: u8, /// (Immutable) Digest of the bytecode and transitive dependencies /// that will be used in the upgrade. digest: vector, } /// Issued as a result of a successful upgrade, containing the /// information to be used to update the `UpgradeCap`. This is a "Hot /// Potato" to ensure that it is used to update its `UpgradeCap` before /// the end of the transaction that performed the upgrade. struct UpgradeReceipt { /// (Immutable) ID of the `UpgradeCap` this originated from. cap: ID, /// (Immutable) ID of the package after it was upgraded. package: ID, } /// Update any part of the package (function implementations, add new /// functions or types, change dependencies) const COMPATIBLE: u8 = 0; /// Add new functions or types, or change dependencies, existing /// functions can't change. const ADDITIVE: u8 = 1; /// Only be able to change dependencies. const DEP_ONLY: u8 = 2; /// Tried to set a less restrictive policy than currently in place. const ETooPermissive: u64 = /* ... */; /// This `UpgradeCap` has already authorized a pending upgrade. const EAlreadyAuthorized: u64 = /* ... */; /// This `UpgradeCap` has not authorized an upgrade. const ENotAuthorized: u64 = /* ... */; /// Trying to commit an upgrade to the wrong `UpgradeCap`. const EWrongUpgradeCap: u64 = /* ... */; /// The most recent version of the package, increments by one for each /// successfully applied upgrade. public fun version(cap: &UpgradeCap): u64 { cap.version } /// The most permissive kind of upgrade currently supported by this /// `cap`. public fun upgrade_policy(cap: &UpgradeCap): u8 { cap.policy } /// Restrict upgrades through this upgrade `cap` to just add code, or /// change dependencies. public entry fun only_additive_upgrades(cap: &mut UpgradeCap) { restrict(cap, ADDITIVE) } /// Restrict upgrades through this upgrade `cap` to just change /// dependencies. public entry fun only_dep_upgrades(cap: &mut UpgradeCap) { restrict(cap, DEP_ONLY) } /// Discard the `UpgradeCap` to make a package immutable. public entry fun make_immutable(cap: UpgradeCap) { let UpgradeCap { id, package: _, version: _ } = cap; object::delete(id); } /// Issue a ticket authorizing an upgrade to a particular new bytecode /// (identified by its digest). A ticket will only be issued if one has /// not already been issued, and if the `policy` requested is at least as /// restrictive as the policy set out by the `cap`. /// /// The `digest` supplied and the `policy` will both be checked by /// validators when running the upgrade. I.e. the bytecode supplied in /// the upgrade must have a matching digest, and the changes relative to /// the parent package must be compatible with the policy in the ticket /// for the upgrade to succeed. public fun authorize_upgrade( cap: &mut UpgradeCap, policy: u8, digest: vector ): UpgradeTicket { let id_zero = object::id_from_address(@0x0); assert!(cap.package != id_zero, EAlreadyAuthorized); assert!(policy >= cap.policy, ETooPermissive); let package = cap.package; cap.package = id_zero; UpgradeTicket { cap: object::id(&cap), package, policy: cap.policy, digest, } } /// Consume an `UpgradeReceipt` to update its `UpgradeCap`, finalizing /// the upgrade. public fun commit_upgrade( cap: &mut UpgradeCap, receipt: UpgradeReceipt, ) { let UpgradeReceipt { cap: cap_id, package } = receipt; assert!(object::id(&cap) == cap_id, EWrongUpgradeCap); assert!(object::id_to_address(&cap.package) != @0x0, ENotAuthorized); cap.package = package; cap.version = cap.version + 1; } fun restrict(cap: &mut UpgradeCap, policy: u8) { assert!(cap.policy <= policy, ETooPermissive); cap.policy = policy; } } ```

What can I Upgrade?

Upgraded packages must maintain compatibility with all their previous versions, so a package published to run with version V of a dependency can run against some later version, V + k. This property is guaranteed by doing a link and layout compatibility check during publishing, enforcing the following:

Compatible Changes

Incompatible Changes

How will I...?

Examples of how common operations work with the introduction of package upgrades.

Publish and distribute a package

 Publishing and distribution works similarly to how it did before, but with the following changes: - The manifests for on-chain **dependencies need to include a `published-at` address** -- this is the address at which the package is found, which could now be different from the self-address of the dependency. - **Publishing will produce an `UpgradeCap` object** which gives permission to the bearer to make future updates to the package. By default this will be returned to the sender, but in more complex use cases, it can be restricted, wrapped, or discarded as part of the publish transaction (using #7790) to set the upgrade policy up-front. - If the package is going to be used as a dependency of other packages, **the `published-at` field in its own manifest needs to be updated** to point to the address it was just published to. - This is in addition to updating the `[addresses]` section replacing the `0x0` entries for package self-address(es) with the newly published address.

Upgrade a package

 In the simplest case, you own an `UpgradeCap` for a package, and will be able to call `upgrade` on a move package with it: ```bash $ sui client upgrade --gas-budget 0x... --cap 0x... ``` This behaves similarly to a publish, except that it makes sure that the `--cap` argument points to a valid `UpgradeCap` that you have permission to modify, that the bytecode being published is compatible with the version of the package that is being guarded by the cap, and registers the newly published package as an upgrade of this old package. In cases where the `UpgradeCap` is guarded by a more complex upgrade policy (See "Choose when upgrades are allowed", below), the upgrade transaction will need to be built using #7790, using the following steps: 1. **Build the bytecode** for the upgraded package. 2. **Calculate the `digest`** of this bytecode (this is a sha256 hash of the module bytecode, calculated by hashing each module separately, sorting the resulting hashes, and then rehashing them to form a stable digest). 3. Add a command to the programmable transaction to **invoke the upgrade policy**, which holds the `UpgradeCap`, requesting permission to perform the upgrade. The precise interface exposed by the policy is implementation-specific, but you will at least need to supply the `digest` calculated in the previous step. If the request is successful, this command will return an `UpgradeTicket` as a result. 4. Add the **`Upgrade` command** next, taking the ticket as the first input, and the bytes for the upgrade as the following input. If the upgrade is successful, it will produce an `UpgradeReceipt` as a result. 5. The final command in the programmable transaction is a move call, to finalize the upgrade by **updating the `UpgradeCap` with the `UpgradeReceipt`** by calling another function on the upgrade policy which accepts the receipt (again which function is specific to the policy). 6. Execute the programmable transaction on the network.

Assess risk from package upgrades

 It's important that a package's upgrade policy is easily auditable, so that potential stakeholders can verify that the package they are agreeing to will not change in a way that they did not expect, after they lock value in it. Because a package's upgrade policy is controlled by its `UpgradeCap`, auditability will be provided by tracking the relationship between different versions of the same package and their `UpgradeCap`, and display this information in the Explorer. When viewing a package it will be possible to identify whether its `UpgradeCap` is still available, and if so, where it is: - If the `UpgradeCap` has been **deleted**, the **package is not upgradeable**. - If it is not deleted, and is **owned by an individual address**, trust in the package is based on **trust in the entity that controls that address**, because they could modify the package. - If it is not deleted, and is **wrapped in another object** that is part of an upgrade policy, trust in the package depends on **trust in the code that defines the upgrade policy** (and in turn whether that is upgradeable and if so what its upgrade policy is, and so on). **Package use limited to purely objects that you own confers minimal risk** because you will always be able to continue using the old package. But if your use of a package involves shared objects, or handing ownership of your objects to other parties, then care should be taken when it comes to the relevant packages' upgrade policies: As a general rule of thumb, packages used in production by a large number of third parties should have very restrictive upgrade policies. I.e. they are either not upgradable at all, or the right to upgrade is controlled by more than one individual address (e.g. a K-of-N policy). As an extreme example, **actively used upgrade policies should not be upgradeable**. In comparison it is much less risky for packages that act as entry points to individual apps, or that are in the early stages of development to have permissive upgrade policies to allow for bugfixes.

Choose what can be upgraded

 The owner of an `UpgradeCap` can restrict future upgrades to one of the following policies: - `COMPATIBLE`: any change as long as it maintains link and layout compatibility. - `ADDITIVE`: only new types, functions and modules, as well as changes to dependencies. - `DEP_ONLY`: only changes to dependent packages. Packages start off with upgrade caps that permit all compatible upgrades, and can be restricted using the following functions in `UpgradeCap`: ```rust module sui::package { /// Restrict upgrades through this upgrade `cap` to just add code, or /// change dependencies. public entry fun only_additive_upgrades(cap: &mut UpgradeCap) { restrict(cap, ADDITIVE) } /// Restrict upgrades through this upgrade `cap` to just change /// dependencies. public entry fun only_dep_upgrades(cap: &mut UpgradeCap) { restrict(cap, DEP_ONLY) } } ``` Once an `UpgradeCap` has been restricted in this way, it can only become more restrictive with time, not less.

Implement a custom upgrade policy

 Below is an example of a custom upgrade policy that requires K-of-N votes to authorize a particular upgrade, built on types in the `sui::package` module: ```rust module example::k_of_n { use sui::object::{Self, ID, UID}; use sui::package::{Self, UpgradeCap, UpgradeTicket, UpgradeReceipt}; use sui::tx_context::{Self, TxContext}; use sui::vec_set::{Self, VecSet}; /// An upgrade policy where upgrades through `cap` are controlled by /// voting, with `required_votes` needed from the addresses in /// `possible_votes`. struct KofNUpgradeCap has key, store { id: UID, cap: UpgradeCap, required_votes: u64, possible_votes: VecSet
, } /// An in-progress vote on whether `signer` should be allowed to issue an /// upgrade with digest `digest`. struct Vote has key, store { id: UID, /// The ID of the `KofNUpgradeCap` that this vote was initiated from. cap: ID, /// The address requesting permission to perform the upgrade. signer: address, /// The digest of the bytecode that the package will be upgraded to. digest: vector, /// The voters who have already agreed to this upgrade. voters: VecSet
, } /// This address cannot participate in votes for this upgrade cap. const EInvalidVoter: u64 = 0; /// Not enough votes accrued to issue a ticket. const ENotEnoughVotes: u64 = 1; /// An upgrade ticket has already been issued from this ticket. const EAlreadyIssued: u64 = 2; /// The address requesting the ticket does not match the `signer` in the /// vote. const ESignerMismatch: u64 = 3; /// Protect `cap` in a `KofNUpgradeCap`. public fun new( cap: UpgradeCap, required_votes: u64, possible_votes: VecSet
, ctx: &mut TxContext, ): KofNUpgradeCap { KofNUpgradeCap { id: object::new(ctx), cap, required_votes, possible_votes, } } /// Start a new vote for `signer` to upgrade package in `cap` so its /// content's digest matches `digest`. public fun propose( cap: &KofNUpgradeCap, signer: address, digest: vector, ctx: &mut TxContext, ): Vote { Vote { id: object::new(ctx), cap: object::id(&cap), signer, digest, voters: vec_set::empty(), } } /// Discard a vote in progress. public fun burn_vote(vote: Vote) { let Vote { id, voters: _ } = vote; object::delete(id); } /// Vote in favour of an upgrade, aborts if the sender is not one of the /// possible voters for this cap. public fun vote(cap: &KofNUpgradeCap, vote: &mut Vote, ctx: &TxContext) { let voter = tx_context::sender(ctx); assert!(vec_set::contains(&cap.possible_voters, voter), EInvalidVoter); vec_set::insert(&mut vote.voters, voter); } /// Issue an `UpgradeTicket` for the upgrade being voted on. Aborts if /// the vote has not accrued enough votes yet, or has already issued a /// ticket, or the sender of this request is not the proposed signer of /// the upgrade transaction. public fun authorize_upgrade( cap: &mut KofNUpgradeCap, vote: &mut Vote, ctx: &TxContext, ): UpgradeTicket { assert!( vec_set::size(&vote.voters) >= cap.required_votes, ENotEnoughVotes, ); let signer = tx_context::sender(ctx); assert!(vote.signer != @0x0, EAlreadyIssued); assert!(vote.signer == signer, ESignerMismatch); vote.signer = @0x0; let policy = package::upgrade_policy(&cap.cap); package::authorize_upgrade( &mut cap.cap, policy, vote.digest, ) } /// Finalize the upgrade that ran to produce the given `receipt`. public fun commit_upgrade( cap: &mut KofNUpgradeCap, receipt: UpgradeReceipt, ) { package::commit_upgrade(&mut cap.cap, receipt) } } ```

Deprecate a type, function, or package

 Extreme cases may call for the prevention of future access to functions or types in a buggy package. This will not be possible for function calls involving purely owned objects, but will be if shared objects are involved. Deprecation will use the "Struct Version Constraints" extension (see below), to **prevent access to the shared object by older versions of the package, without losing data**: Doing so forces any access to that shared object to go through the newer version of the package, so even though the older version exists on-chain, it cannot be used. From the user perspective, this poses an **additional risk**, to watch out for when assessing exposure to risk due to package upgrades in the presence of shared objects.

Gotchas

Diamond Problem

 The [diamond dependency problem](https://jlbp.dev/what-is-a-diamond-dependency-conflict) manifests with a dependency graph as follows: ``` +-- A --+ | | v v B C | | +-> D <-+ ``` Where all packages are published by different parties who are unaware of each other. Although the graph appears consistent, the version of `D` that `B` depends on is different from the version that `C` depends on (W.l.o.g. `C` depends on the newer version): ``` +-- A --+ | | v v B C | | v v D < D' ``` This was not an issue for `B` or `C` separately, but becomes an issue when `A` enters the picture, because it is not clear which version of `D` it should depend on. This will be addressed by supporting **dependency overrides**: `A` will be able to specify its own version of `D` which subsumes the version specified by `B` and `C`. In general, a package can override the dependencies of any package that it dominates in the dependency graph. When `A` is published, there will be a further **check that the version of `D` it picks is at least as new as the newest version picked by its dependent packages**. This is to ensure that `A` does not pick a version that is link incompatible with one of its dependencies.

Module Initializers

 Module Initializers are commonly used to perform operations that developers rely on happening exactly once per package (such as creating one-time witnesses). As such, they will not be re-run when a package is upgraded.

Invariant Violation

 One consequence of packages being immutable is that they cannot be deleted, even when they are superseded by a later version and as mentioned above, older versions of a package will still be callable, potentially accessing objects that are being used by newer version of that package. This set-up can introduce bugs when the new version of the package is maintaining invariants that the old version is completely unaware of: ```rust module 0xA0::counter { use sui::object::{Self, UID}; use sui::tx_context::TxContext; use sui::transfer; struct Counter has key { id: UID, value: u64, } fun init(ctx: &mut TxContext) { transfer::share_object(Counter { id: object::new(ctx), value: 0, }); } public entry fun increment(c: &mut Counter) { c.value = c.value + 1; } } module 0xA1::counter { use sui::object::{Self, UID} use sui::tx_context::TxContext; use sui::event; struct Counter has key { id: UID, value: u64, } struct Progress has copy, drop { reached: u64 } public entry fun increment(c: &mut Counter) { c.value = c.value + 1; if (c.value % 100 == 0) { event::emit(Progress { reached: c.value }); } } } ``` In the example above, the new version of `A::counter::increment` emits a `Progress` event every 100 increments, whereas the old version does not. If there are a mix of callers for this function and some still use the old version of the package, this invariant will be broken (maintaining dynamic fields that need to remain in sync with a struct's original fields is another source of bugs). While this issue could affect any object, it's **particularly problematic for shared objects** who do not have an owner to keep track of a common thread and make sure the object is not sent to packages at different versions. The "Struct Version Constraints" extension will help alleviate this risk in some cases, but for the most part **package library developers need to be aware of the potential interplay between versions of their package**.

Package Rug Pulls

 This is a pattern where a user hands over control of their assets (e.g. by putting them in a shared object or through some other means) based on their understanding of what a package allows, only to have that change through an upgrade: ```rust module 0xB0::safe { use sui::coin::Coin; use sui::object::{Self, ID, UID}; use sui::sui::SUI; use sui::table::{Self, Table}; use sui::tx_context::{Self, TxContext}; struct Safe has key { id: UID, accounts: Table>, } fun init(ctx: &mut TxContext) { transfer::share_object(Safe { id: object::new(ctx), accounts: table::new>(), }); } public entry fun deposit( safe: &mut Safe, coin: Coin, ctx: &TxContext, ) { table::add(&mut safe.accounts, tx_context::sender(ctx), coin); } public entry fun withdraw(safe: &mut Safe, ctx: &TxContext) { transfer::transfer( table::remove(&mut safe.accounts, tx_context::sender(ctx)), tx_context::sender(ctx), ); } } module OxB1::safe { /* ... as before ... */ public entry fun steal(safe: &mut Safe, account: address, ctx: &TxContext) { transfer::transfer( table::remove(&mut safe.accounts, account), tx_context::sender(ctx), ); } } ``` This is **undesirable**, but stems from flexibility that is required to allow developers to fix bugs or add functionality -- core value propositions of package upgrades. This is countered by providing a clear audit trail so users can know who controls the packages they are relying on, and decide whether to trust them or not (informed consent, see "Assess risk from package upgrades", above).

Extensions

Automated address management

 Managing on-chain package addresses is already cumbersome -- developers need to remember to update `[addresses]` entries after they publish, after wipes, when switching between devnet, testnet and mainnet etc. Package upgrades introduce another address to manage -- the ID of the latest version of the package. The proposal introduces the `published-at` manifest field as a bare minimum form of support, but in the long-term, developers will not have to worry about updating addresses themselves. Address information will be moved to the lock file, and operations like `publish` and `upgrade` will update them automatically.

Struct Version Constraints

 Given a struct `S` that was introduced at version `V` of a package `P`, if it appears in a later version `U` with a `#[min_version(U)]` annotation, it implies that transactions that use `P` at version `U` or above will write back instances of `S` with the package ID of `P` at version `U` (not `P`). From that point packages at versions below `U` will not be able to read that instance of `S`, because they will treat it as a different type. This functionality allows us to perform **one-way migrations** on specific types where previously a type would have been forward and backward compatible, which enabled patterns to "Deprecate a type, function, or package" (see above).

Upgrade Hooks

 Like the module initializer, it may be helpful to have a function that is run as part of a successful upgrade, e.g. to migrate objects (e.g. shared objects managed by a package), however this only becomes useful when this kind of call (i.e. module initializers) supports accepting parameters (which is not currently true).

Appendix: End-to-end Example

 Consider packages `A`, depending on `B`, depending on `C`, where `B` depends on `C` at version `0`, and `A` depends on `C` at version `1`. Assume for convenience that the on-chain version of package `P` at version `X` is found at `0xPX`, its upgrade cap is found at `0xCA4P` and the source is found at: ``` P = { git = "https://github.com/MystenLabs/example", subdir = "P", rev = "4a540X" } ``` e.g. Version 1 of `C` is found at `0xC1` on-chain, sub-directory `C` of revision `4a5401` of `https://github.com/MystenLabs/example` off-chain, and its `UpgradeCap` is found at `0xCA4C`. Their manifests would be set-up as follows: ```toml # MystenLabs/example/A/Move.toml @4a5400 [package] name = "A" version = "0.0.0" published-at = "0xA0" [addresses] A = "0xA0" B = "0xB0" C = "0xC0" [dependencies] B = { git = "https://github.com/MystenLabs/example", subdir = "B", rev = "4a5400" } C = { git = "https://github.com/MystenLabs/example", subdir = "C", rev = "4a5401" } # MystenLabs/example/B/Move.toml @4a5400 [package] name = "B" version = "0.0.0" published-at = "0xB0" [addresses] B = "0xB0" C = "0xC0" [dependencies] C = { git = "https://github.com/MystenLabs/example", subdir = "C", rev = "4a5400" } # MystenLabs/example/C/Move.toml @4a5400 [package] name = "C" version = "0.0.0" published-at = "0xC0" [addresses] C = "0xC0" # MystenLabs/example/C/Move.toml @4a5401 [package] name = "C" version = "0.0.0" published-at = "0xC1" [addresses] C = "0xC0" ``` The packages would be published/upgraded with the following commands, and assuming that at the time of the transaction, its self-address is set to `0x0`: ```bash C (4a5400)$ sui client publish --gas-budget 10000 # Txn: linkage = {}, package = "..." C (4a5401)$ sui client upgrade --gas-budget 10000 --cap 0xCA4C # Txn: linkage = {}, package = "..." B (4a5400)$ sui client publish --gas-budget 10000 # Txn: linkage = {0xC0}, package = "..." A (4a5400)$ sui client publish --gas-budget 10000 # Txn: linkage = {0xB0, 0xC1}, package = "..." ``` Which is represented on-chain as: ```rust 0xA0.linkage = { 0xB0: (0xB0, 0), 0xC0: (0xC1, 1) } module 0xA0::A { use 0xB0::B::b; use 0xC0::C::{c, update, S, T}; public fun foo(s: S) { bar(update(s)) } public fun bar(t: T) { /* ... */ } } 0xB0.linkage = { 0xC0: (0xC0, 0) } module 0xB0::B { use 0xC0::C; } 0xC1.linkage = {} module 0xC1::C { struct S { a: u64 } struct T { a: u64, b: bool } public fun update(s: S): T { let S { a } = s; T { a, b: false } } public fun c(): u64 { 43 } } 0xC0.linkage = {} module 0xC0::C { struct S { a: u64 } public fun c(): u64 { 42 } } ``` - There is a clear distinction between the on-chain and off-chain representation of the package, for example: - `version` in the manifest has no bearing on the on-chain package, and - the only thing linking the source of a package at version `N+1` with the package at version `N` is the call to `sui client upgrade`. - When a package is published, it references other packages at their **original** IDs, not their upgraded IDs, relying on the linkage table to translate that back to the actual Package IDs during resolution. - The linkage information is provided in the transaction as a set of Package IDs (which is automatically gathered from the `published-at` fields of dependency packages), but is stored on-chain as a mapping. This translation is done by the `Upgrade` transaction, after verifying that the upgrade is valid, to speed up loading in future.
FrankC01 commented 1 year ago

Two things:

  1. Immutable (flagging it so it can not be updated) is valuable in cases of one-time capability or time based subscription oriented contracts
  2. Can these insightful info subjects be made into discussions or GitHub pages so as not to be hidden in the issues bucket?
amnn commented 1 year ago

Hey @FrankC01, thanks for raising. Immutable packages are indeed something that we're looking to support and let me talk to the team about what's a good place to surface these kinds of topics (this topic, and Time are the two examples I'm aware of).

One benefit of issues is that other projects can link to them and we get to see that, but yes, they do get kind of buried among all the other day-to-day issues. Maybe we can introduce a tag for larger features, so they can be filtered out better?

FrankC01 commented 1 year ago

Tagging, discussions (which can generate issues from the discussions) are two good ones.

JackyWYX commented 1 year ago

Can the upgradeable permission (in description on publisher of the module) can be changed to another account? Is PackageUpgradeCap an object that can be transferred?

amnn commented 1 year ago

@JackyWYX, yes it will. Sorry it's taking a little while for me to publish the proposal for how upgrades will work -- our current testnet wave is taking all of our focus, but it will be out soon!

ericEnjoy commented 1 year ago

This feature will be very convenient for contract development, and I see it's in the Wave 3 milestone, so this feature will come out before mainnet, am I right?

amnn commented 1 year ago

Hi @ericEnjoy, it will be there for mainnet, but not before Wave 3. Our tags are a little stale on this issue, unfortunately. Thanks for spotting!

amnn commented 1 year ago

Heads up: This issue has now been updated with the latest proposal for how upgrades will work.

d-moos commented 1 year ago

Hey, happy to see that there is progress on this topic. Are package upgrades allowed to modify the friend declarations as well?

amnn commented 1 year ago

Hi @d-moos, yes, you can change your friend declarations, as well as the signatures and implementations of friend visibility functions, because friend modules all have to be in the same package (i.e. they behave like package-private visibility), so they do not get exposed in the package's public interface.

WGB5445 commented 1 year ago

Thanks for providing such interesting features Is the #[min_version] function available now and how should I use it ?

amnn commented 1 year ago

Hi @WGB5445, version constraints on structs are not part of the initial upgrades release, they are still to come!

WGB5445 commented 1 year ago

Thank you for your answer, Will there be before the mainnet goes live?

amnn commented 1 year ago

While it or a similar feature will be added, we haven't put it on the roadmap yet!

nguyenhoaibao commented 1 year ago

According to the Package Upgrade documentation, it said we can add abilities for existing structs. I tested with a struct without the store ability, that means it cannot be transferred:

struct Test has key { id: UID };

then I published that package to the testnet.

Later on I added the store ability for that struct:

struct Test has key, store { id: UID };

I upgraded the package successfully, however when I create a new instance of that new Test struct, I still cannot transfer that new instance. What can I do to make newly objects of that Test struct be able to be transferred?

Another thing to notice is if I create an entry transfer function for that Test struct in its module, like:

public entry fun transfer(object: Test, recipient: address) {
    transfer::transfer(object, recipient)
}

then I can use that entry function to transfer Test objects.

amnn commented 1 year ago

@nguyenhoaibao, thanks for reporting this, this does look like an issue we should address. I've started a discussion internally around how best to do that.

The technical context here is that when we upgrade a package, we don't change the identities of existing types in that package (i.e. if your type Test was introduced first in a package at address 0x123 then it will always be referred to as 0x123::Test on-chain), this means that when a transaction block tries to Transfer it, it's going to look at the abilities of 0x123::Test, even if 0x456::Test exists with a later version of the package (and type).

The reason why your example in Move works is that in Move, we are allowed to "re-link" packages and their dependencies, as long as they remain compatible. So if you were to call transfer::transfer in the upgraded package, or even from a package that depends on your upgraded package (i.e. outside the module, or even the package that defines Test), that would work, because it understands the expanded ability set of Test.

Alivers commented 1 year ago

After I upgraded my contract, I added a new dynamic object field to an existing parent object. But I could not get the new child object data of this dynamic object field. getObject(include explorer and cli) api returns with error: Failure serializing object in the requested format: "Could not find module". here is my original object id: 0x78673aae6463c14dab82feb636d7cf2b89a72a255be8375e22b47c71749d903b , on testnet https://explorer.sui.io/object/0x78673aae6463c14dab82feb636d7cf2b89a72a255be8375e22b47c71749d903b?network=testnet

The LendingMarket Object is created using old package entry. I added a new entry func to add dynamic field for LendingMarket in new package. The child object added to LendingMarket is a new struct defined in new module. Is there anything I can do to fix this error?

amnn commented 1 year ago

@Alivers, thanks for the report, we'll look into it!

nguyenhoaibao commented 1 year ago

@amnn I see the latest v0.33.0 release does not allow to add abilities to existing struct while upgrading the package anymore. So is there any other way to make existing struct can be transferred, if it cannot at the first publishing, like the case I described above https://github.com/MystenLabs/sui/issues/2045#issuecomment-1519596272?

By the way, after upgrading to v0.33.0, I cannot publish the contract anymore:

Multiple source verification errors found:

- Local dependency did not match its on-chain version at 0000000000000000000000000000000000000000000000000000000000000002::Sui::kiosk
- Local dependency did not match its on-chain version at 0000000000000000000000000000000000000000000000000000000000000002::Sui::transfer_policy

Do you know what it is and how to fix it?

vivekascoder commented 1 year ago

@nguyenhoaibao I'm having the same issue.

amnn commented 1 year ago

@nguyenhoaibao, @vivekascoder, the issue you are facing with source verification is happening because the network will undergo a framework upgrade. The branch that you are depending on contains the system packages that will be used after the next protocol upgrade, so if there are any changes, source verification will flag them. You have a couple of choices for how to get around this:

@nguyenhoaibao, regarding adding store (or other abilities) to types during upgrades. When investigating your report, we realised that there is an issue with how this feature is currently exposed, so we had to disable it (thanks again for the report!) We do plan on re-enabling this feature, but when we do, it will probably be along with "struct version constraints" (i.e. to change a type, you would also need to apply a version constraint to prevent old versions of the package from being able interact with it). Sorry we don't have a solution for you to do this today, but we're working on it.

@Alivers, we've figured out what is causing the issue you're facing, and we're working on fixing it!

nguyenhoaibao commented 1 year ago

@amnn have you found any solution yet? Sui is going to release mainnet today - within next 8 hours to be precise - so we also have to finalize our package to prepare for the deployment asap 😅

amnn commented 1 year ago

Hi @nguyenhoaibao, struct version constraints, and therefore the ability to add abilities to existing types was not on the roadmap before mainnet, but is high on the list of features we will tackle post-launch.

10xhunter commented 9 months ago

@sblackshear @amnn could you please add store ability to UpgradeTicket struct?

My use case: dao users can propose to issue an upgrade ticket to a third party to upgrade an external package.

amnn commented 9 months ago

Hi @10xhunter, no we cannot add store to UpgradeTicket, because doing so would allow someone to stall an upgrade half way through by issuing a ticket but then never using it.

You should still be able to achieve what you want by using a custom upgrade policy -- this is why UpgradeCap has store.

To allow someone to propose an upgrade, the package being upgraded should use an upgrade policy that allows such proposals. Take a look at the K-of-N upgrade policy example in this issue, or alternatively this PR, which is creating a productionised version of the same:

14879

The details of the custom upgrade policy for your DAO example would be different (we can discuss this more if you share some more details), but these examples show how to add custom logic around upgrades.

10xhunter commented 9 months ago

@amnn I've also considered the issue you mentioned, in that case, we can propose to issue another ticket with different digest to another party to skip the stalled upgrade.

10xhunter commented 9 months ago

My use case is that, for example, group A published a package AP, and gave the AP UpgradeCap to shared object DAO_A, which is managed by dao3.ai contracts, when group A wants to upgrade package AP, they will propose in DAO_A, and issue an UpgradeTicket to whoever the receiver is to upgrade package AP.

amnn commented 9 months ago

It sounds like what you want to do can be achieved by using a custom upgrade policy, in this case, one that authorizes a specific address, or the holder of some capability, the ability to upgrade a package whose UpgradeCap is owned by a DAO.

Have you had a look at our documentation on custom upgrade policies, and if so, does it seem like you could create an upgrade policy that fits your purpose, if not, what are the blockers to doing that?