nathanielhourt commented 5 years ago

BSIP: TBD
Title: Make Update Operation Fields Optional
Authors: Nathan Hourt <nat.hourt@gmail.com>
Status: Draft
Type: Protocol
Created: April 6, 2019
Discussion: https://github.com/bitshares/bsips/issues/159

Abstract

The BitShares protocol specifies a number of operations to update objects within the database. Many of these operations contain several fields, all of which must be specified, even if only one or a few of them are being updated. This requires the operation to redundantly specify old values of fields for which no change is desired, which is wasteful of valuable blockchain storage space. Moreover, this strategy makes stateless analysis of such updates, i.e. to discern which fields are being modified, impossible.

This BSIP proposes to replace object update operations which contain many non-optional fields with new versions which specify all updated fields optionally. This will reduce the storage size of affected operations and make stateless analysis of their effects possible. Following a grace period, the old operations may be deprecated to require usage of the newer, more efficient versions.

Motivation

The size of the BitShares blockchain history is growing rapidly. Moreover, the rate of usage is increasing, causing the growth to accelerate. While the long term solution is the development of improved storage and handling of historical data, in the meantime it behooves stakeholders to eliminate frivolous waste of storage space. Several of the database update operations currently require the superfluous specification of all fields rather than only the particular fields the user wishes to modify. This results in storage space being consumed by redundant information which provides no benefit whatsoever to the stakeholders or software.

Another compelling motivation to adopt this change is that it supports other desirable use cases. BSIP 40 is a proposal to add a new authority type to the blockchain, the "Custom Active Authority," which grants limited privileges to sub-authorities rather than requiring full active authority for all operations. One important use case for this feature is to allow the creation of a limited authority capable of updating, for example, an account's memo key, but not its votes. With the current design of account_update_operation, however, an update to the memo key must also specify the votes, and it cannot be determined whether the new votes are identical to the old ones without accessing chain state.

As currently specified, BSIP 40 cannot provide for this use case, and modifying the proposal to support analysis of the operations with access to chain state increases its implementation complexity dramatically and may have detrimental impacts on future blockchain scalability improvements. The addition of new update operations that specify field updates optionally is a comparatively small modification to the blockchain protocol, and it enables BSIP 40 to support this critical use case with no alterations to its current specification.

Rationale

BitShares operations are designed according to some loosely defined principles, one of which is that they should facilitate stateless analysis of their effects. For instance, operations which modify an account balance are expected to explicitly state the amount and asset type being moved, even when these can be inferred from chain state, so that programs can track the balance of accounts by looking at operation history alone, without requiring access to chain state. This principle can also be extended to update operations to require that such operations facilitate stateless analysis of their changes -- in particular, the ability to determine which specific fields are being changed.

An update to a field can be expressed in two ways: a delta of change, or a replacement value. Thus the question is raised, which is more important to stateless analysis -- the delta of change, or the new value? This proposal suggests to use the new value for simple field updates, but adjustment of asset balances must always be specified as a delta, as these adjustments represent movement of asset rather than arbitrary specification of a value. Updates to sets will support both a delta (items to add/remove) and a new value (complete replacement set), with the requirement that only one of these options be specified in a given operation.

The complete set of update operations presently defined by the protocol is as follows:

account_update_operation
asset_update_operation
asset_update_bitasset_operation
asset_update_feed_producers_operation
asset_update_issuer_operation
call_order_update_operation
committee_member_update_operation
committee_member_update_global_parameters_operation
proposal_update_operation
withdraw_permission_update_operation
witness_update_operation
htlc_extend_operation

Several of these operations are already complicit with the above principle. Only the following require alteration or replacement to comply:

account_update_operation
asset_update_operation
asset_update_bitasset_operation
committee_member_update_global_parameters_operation
withdraw_permission_update_operation

Note: call_order_update_operation is an edge case, as it requires specification of both delta_collateral and delta_debt; however, these are deltas and therefore support stateless analysis, and the size advantage gained by specifying only one of these values versus both of them is not deemed sufficient to justify replacing and deprecating the operation.

Specification

Two hardfork dates will be specified for the implementation of this BSIP: one is the activation date for the new operations/extensions, and the other is the cutoff date for the old operations/fields.

To reduce boilerplate around set updates, which may be expressed either as a delta or a complete replacement set, the following template will be defined:

template<typename Element>
struct set_updater {
   // A delta is a map of Elements to a bool where true means add the element, and false means remove it
   using delta = flat_map<Element, bool>;
   using new_set = flat_set<Element>;
   optional<static_variant<delta, new_set>> update;
};

A discussion of the proposed modifications are described for each affected operation below.

Account update

Of the operations requiring alteration, account_update_operation stands out since all of its update fields are already optional; however, one field in the operation, new_options, is an object with several non-optional sub-fields.

Since all fields on the operation itself are already optional, the existing operation does not need to be deprecated; instead, new operation extensions can be defined which update each sub-field in new_options individually, and new_options can subsequently be required to be a null optional after the cutoff date. Extensions are always optional, thus satisfying the optional updates design principle.

The account_update_operation::new_options field is an optional of type account_options. The fields of account_options are:

struct account_options {
   public_key_type memo_key;
   account_id_type voting_account;
   uint16_t num_witness;
   uint16_t num_committee;
   flat_set<vote_id_type> votes;
};

The following extensions will be added to account_update_operation to support updating these fields without using new_options:

optional<public_key_type> new_memo_key;
optional<account_id_type> new_voting_account;
optional<uint16_t> new_num_witness;
optional<uint16_t> new_num_committee;
set_updater<vote_id_type> votes_update;

The evaluator will be updated to verify that, if num_witness or num_committee or the votes are updated, then the desired number of witness/committee seats does not exceed the number of votes.

Asset Updates

Of the operations requiring alteration, the asset_update_operation also stands out, as it only has two update fields, one of which, new_issuer, is already optional, but is also required to be null as it has already been replaced with the dedicated asset_update_issuer_operation. The other field is a struct called asset_options. The fields in this struct are listed below:

struct asset_options {
   share_type max_supply;
   uint16_t market_fee_percent;
   share_type max_market_fee;
   uint16_t issuer_permissions;
   uint16_t flags;
   price core_exchange_rate;
   flat_set<account_id_type> whitelist_authorities;
   flat_set<account_id_type> blacklist_authorities;
   flat_set<asset_id_type> whitelist_markets;
   flat_set<asset_id_type> blacklist_markets;
   string description;
   <extensions, listed below>
};

<extensions> {
   uint16_t reward_percent;
   flat_set<account_id_type> whitelist_market_fee_sharing;
};

A new asset_options_update_operation, which does not contain the deprecated new_issuer field, will be defined with the fields specified below. After the cutoff date, the asset_update_operation will be deprecated and disabled from use.

struct asset_options_update_operation {
   /* asset to update, fee, payer... */

   optional<share_type> new_max_supply;
   optional<uint16_t> new_market_fee_percent;
   optional<share_type> new_max_market_fee;
   optional<uint16_t> new_issuer_permissions;
   optional<uint16_t> new_flags;
   optional<price> new_core_exchange_rate;
   set_updater<account_id_type> whitelist_authorities_update;
   set_updater<account_id_type> blacklist_authorities_update;
   set_updater<asset_id_type> whitelist_markets_update;
   set_updater<asset_id_type> blacklist_markets_update;
   optional<string> new_description;
   optional<uint16_t> new_reward_percent;
   set_updater<account_id_type> whitelist_market_fee_sharing_update;
};

A second asset updating operation, asset_update_bitasset_operation, must be modified as well. This operation bundles all of its fields for updating into a single struct field of type bitasset_options:

struct bitasset_options {
   uint32_t feed_lifetime_sec;
   uint8_t minimum_feeds;
   uint32_t force_settlement_delay_sec;
   uint16_t force_settlement_offset_percent;
   uint16_t maximum_force_settlement_volume;
   asset_id_type short_backing_asset;
};

A new operation, asset_update_bitasset_options_operation, will be defined with the fields listed below. After the cutoff date, asset_update_bitasset_operation will be deprecated and disabled from use.

struct asset_update_bitasset_options_operation {
   /* asset to update, fee, payer... */

   optional<uint32_t> new_feed_lifetime_sec;
   optional<uint8_t> new_minimum_feeds;
   optional<uint32_t> new_force_settlement_delay_sec;
   optional<uint16_t> new_force_settlement_offset_percent;
   optional<uint16_t> new_maximum_force_settlement_volume;
   optional<asset_id_type> new_short_backing_asset;
};

Global Parameters Update

The committee_member_update_global_parameters_operation contains all of its updates in the chain_parameters struct:

struct chain_parameters {
   std::shared_ptr<const fee_schedule> current_fees;
   uint8_t block_interval;
   uint32_t maintenance_interval;
   uint8_t maintenance_skip_slots;
   uint32_t committee_proposal_review_period;
   uint32_t maximum_transaction_size;
   uint32_t maximum_block_size;
   uint32_t maximum_time_until_expiration;
   uint32_t maximum_proposal_lifetime;
   uint8_t maximum_asset_whitelist_authorities;
   uint8_t maximum_asset_feed_publishers;
   uint16_t maximum_witness_count;
   uint16_t maximum_committee_count;
   uint16_t maximum_authority_membership;
   uint16_t reserve_percent_of_fee;
   uint16_t network_percent_of_fee;
   uint16_t lifetime_referrer_percent_of_fee;
   uint32_t cashback_vesting_period_seconds;
   share_type cashback_vesting_threshold;
   bool count_non_member_votes;
   bool allow_non_member_whitelists;
   share_type witness_pay_per_block;
   uint32_t witness_pay_vesting_seconds;
   share_type worker_budget_per_day;
   uint16_t max_predicate_opcode;
   share_type fee_liquidation_threshold;
   uint16_t accounts_per_fee_scale;
   uint8_t account_fee_scale_bitshifts;
   uint8_t max_authority_depth;

   <htlc options in extension:>
   uint32_t max_timeout_secs;
   uint32_t max_preimage_size;
};

A new operation, global_parameters_update_operation will be defined with the fields listed below. After the cutoff date, committee_member_update_global_parameters_operation will be deprecated and disabled from use.

struct global_parameters_update_operation {
   asset fee;

   flat_set<fee_parameters> fee_updates;
   optional<uint8_t> new_block_interval;
   optional<uint32_t> new_maintenance_interval;
   optional<uint8_t> new_maintenance_skip_slots;
   optional<uint32_t> new_committee_proposal_review_period;
   optional<uint32_t> new_maximum_transaction_size;
   optional<uint32_t> new_maximum_block_size;
   optional<uint32_t> new_maximum_time_until_expiration;
   optional<uint32_t> new_maximum_proposal_lifetime;
   optional<uint8_t> new_maximum_asset_whitelist_authorities;
   optional<uint8_t> new_maximum_asset_feed_publishers;
   optional<uint16_t> new_maximum_witness_count;
   optional<uint16_t> new_maximum_committee_count;
   optional<uint16_t> new_maximum_authority_membership;
   optional<uint16_t> new_reserve_percent_of_fee;
   optional<uint16_t> new_network_percent_of_fee;
   optional<uint16_t> new_lifetime_referrer_percent_of_fee;
   optional<uint32_t> new_cashback_vesting_period_seconds;
   optional<share_type> new_cashback_vesting_threshold;
   optional<bool> new_count_non_member_votes;
   optional<bool> new_allow_non_member_whitelists;
   optional<share_type> new_witness_pay_per_block;
   optional<uint32_t> new_witness_pay_vesting_seconds;
   optional<share_type> new_worker_budget_per_day;
   optional<uint16_t> new_max_predicate_opcode;
   optional<share_type> new_fee_liquidation_threshold;
   optional<uint16_t> new_accounts_per_fee_scale;
   optional<uint8_t> new_account_fee_scale_bitshifts;
   optional<uint8_t> new_max_authority_depth;
   optional<uint32_t> new_htlc_max_timeout_secs;
   optional<uint32_t> new_htlc_max_preimage_size;
};

Withdraw Permission Update

The fields of withdraw_permission_update_operation are listed below.

struct withdraw_permission_update_operation {
   asset fee;
   account_id_type withdraw_from_account;
   account_id_type authorized_account;
   withdraw_permission_id_type permission_to_update;

   asset withdrawal_limit;
   uint32_t withdrawal_period_sec;
   time_point_sec period_start_time;
   uint32_t periods_until_expiration;
};

A new operation, withdraw_permission_update_options_operation, will be defined with the fields listed below. After the cutoff date, withdraw_permission_update_operation will be deprecated and disabled from use.

struct withdraw_permission_update_options_operation {
   /* fee, payer, permission specifier... */

   optional<asset> new_withdrawal_limit;
   optional<uint32_t> new_withdrawal_period_sec;
   optional<time_point_sec> new_period_start_time;
   optional<uint32_t> new_periods_until_expiration;
};

Discussion and Summary for Shareholders

The main cost of accepting this proposal is the addition of four new operations to the protocol, and the deprecation of four old ones, as well as the addition of several extensions to one operation and the deprecation of one of its fields. In return, however, the new operation versions will be much smaller, which is advantageous in the face of rapidly accelerating blockchain history growth. It should be noted, however, that these operations are not major contributors to the chain growth rate, and thus this improvement will be minor and will not substantially decrease the rate of growth, nor the acceleration of growth.

The main advantage gained by accepting this proposal is that the new operation versions are more self-describing. It will be possible to determine exactly which fields are updated merely by examining the operations themselves, rather than needing to simultaneously examine the blockchain database. This paves the road for critical use cases within the already-accepted BSIP 40 proposal, which are otherwise unsupportable.

Copyright

Intellectual Property is BS. Those who wish to pretend otherwise are hereby directed to treat this document as public domain.

bitshares / bsips

New BSIP: Make Update Operation Fields Optional #159