Suez serialization support

Bitsharesjs currently doesn't yet support the OSO extension serialization yet

https://github.com/bitshares/bitsharesjs/blob/master/lib/serializer/src/operations.js#L621

It also doesn't yet serialize the limit order update operation neither.

export const limit_order_create = new Serializer("limit_order_create", {
  fee: asset,
  seller: protocol_id_type("account"),
  amount_to_sell: asset,
  min_to_receive: asset,
  expiration: time_point_sec,
  fill_or_kill: bool,
  extensions: extension([
    {
      name: "fee_asset_id",
      type: protocol_id_type("asset"),
    },
    {
      name: "spread_percent",
      type: uint16,
    },
    {
      name: "size_percent",
      type: uint16,
    },
    {
      name: "expiration_seconds",
      type: uint32,
    },
    {
      name: "repeat",
      type: bool,
    },
  ]),
});

export const limit_order_update = new Serializer("limit_order_update", {
  fee: asset,
  account_id_type: protocol_id_type("account"),
  limit_order_id_type: protocol_id_type("limit_order"),
  new_price: optional(price),
  delta_amount_to_sell: optional(asset),
  new_expiration: optional(time_point_sec),
  extensions: extension([
    {
      name: "fee_asset_id",
      type: protocol_id_type("asset"),
    },
    {
      name: "spread_percent",
      type: uint16,
    },
    {
      name: "size_percent",
      type: uint16,
    },
    {
      name: "expiration_seconds",
      type: uint32,
    },
    {
      name: "repeat",
      type: bool,
    },
  ]),
});

Look good?

Easy test: can you broadcast? :)

Look good?

I don't think so.

A limit_order_create_operation with the on_fill extension looks like this: [ 1,{ "fee": { "amount": 100, "asset_id": "1.3.0" }, "seller": "1.2.3833", "amount_to_sell": { "amount": 10000, "asset_id": "1.3.1515" }, "min_to_receive": { "amount": "9900000000", "asset_id": "1.3.0" }, "expiration": "2051-09-06T21:30:20", "fill_or_kill": false, "extensions": { "on_fill": [[ 0, { "fee_asset_id": "1.3.0", "spread_percent": 100, "size_percent": 9990, "expiration_seconds": 1000000000, "repeat": true } ]] } } ]

A limit_order_update_operation with a valid on_fill field looks like this: [ 77,{ "fee": { "amount": 75, "asset_id": "1.3.0" }, "seller": "1.2.3833", "order": "1.7.9089072", "delta_amount_to_sell": { "amount": 697, "asset_id": "1.3.1515" }, "new_expiration": "2068-06-10T16:24:23", "on_fill": [[ 0, { "fee_asset_id": "1.3.0", "spread_percent": 100, "size_percent": 9990, "expiration_seconds": 1000000000, "repeat": true } ]], "extensions": [] } ]

I think we need to define a new type create_take_profit_order_action with

  {
    "fee_asset_id": protocol_id_type("asset"),
    "spread_percent": uint16,
    "size_percent": uint16,
    "expiration_seconds": uint32,
    "repeat": bool,
    "extensions": set(future_extensions)
  }

and define limit_order_auto_action with static_variant([ create_take_profit_order_action ]), then we can use array(limit_order_auto_action) as the type of the on_fill fields in the operations.

In limit_order_create_operation, it may look like


extensions: extension([
{
  name: "on_fill",
  type: array(limit_order_auto_action)
}
])

In limit_order_update_operation, the on_fill field is not in extensions, and it is optional, so it may look like
```
"on_fill": optional(array(limit_order_auto_action))
```

See assert_operation for an example: https://github.com/bitshares/bitsharesjs/blob/e262273c4704de51a6612cd29833c2352cfb876c/lib/serializer/src/operations.js#L1141

Update: limit_order_update_operation::on_fill is optional< vector< limit_order_auto_action > > in core, so in JS it may be

"on_fill": optional(array(limit_order_auto_action))

Note that on_fill in limit_order_create_operation::extensions is optional too, but in the current implementation of the extension type in bitsharesjs, we don't define it explicitly.

Cool, so I've thrown together my initial implementation in the bts-buntime repo, the code should be compatible.

https://github.com/BTS-CM/bts-buntime/blob/main/src/serializer/src/operations.js

New operation fee parameter entries lines 531-548: https://github.com/BTS-CM/bts-buntime/blob/main/src/serializer/src/operations.js#L531

fee_parameters updated (lines 626-628): https://github.com/BTS-CM/bts-buntime/blob/main/src/serializer/src/operations.js#L626

new static variant action (lines 698-706): https://github.com/BTS-CM/bts-buntime/blob/main/src/serializer/src/operations.js#L698

Updated limit_order_create: https://github.com/BTS-CM/bts-buntime/blob/main/src/serializer/src/operations.js#L715

Added liquidity_pool_update, credit_deal_update and limit_order_update (lines 1612 - 1639): https://github.com/BTS-CM/bts-buntime/blob/main/src/serializer/src/operations.js#L1611

Updated operations: https://github.com/BTS-CM/bts-buntime/blob/main/src/serializer/src/operations.js#L1717

Looking good?

Created the following bitsharesjs PR: https://github.com/bitshares/bitsharesjs/pull/125

Created the following bitsharesjs PR: #125

merged for integration testing

[
    {
        "seller": "1.2.1811495",
        "amount_to_sell": {
            "amount": "9090909",
            "asset_id": "1.3.0"
        },
        "min_to_receive": {
            "amount": "10000",
            "asset_id": "1.3.5649"
        },
        "expiration": "2023-11-24T19:09:56.517Z",
        "fill_or_kill": false,
        "extensions": {
            "on_fill": [
                [
                    0,
                    {
                        "fee_asset_id": "1.3.0",
                        "spread_percent": 100,
                        "size_percent": 10000,
                        "expiration_seconds": 1000000000,
                        "repeat": true
                    }
                ]
            ]
        }
    }
]

creates the following BEET request JSON:

{
    "id": "3a10c1eb-c4ee-4c86-8d48-df997816570e",
    "type": "injectedCall",
    "payload": {
        "method": "injectedCall",
        "params": [
            "signAndBroadcast",
            "{\"ref_block_num\":0,\"ref_block_prefix\":0,\"expiration\":\"2023-11-24T20:11:28\",\"operations\":[[1,{\"fee\":{\"amount\":\"48260\",\"asset_id\":\"1.3.0\"},\"seller\":\"1.2.1811495\",\"amount_to_sell\":{\"amount\":\"9090909\",\"asset_id\":\"1.3.0\"},\"min_to_receive\":{\"amount\":\"10000\",\"asset_id\":\"1.3.5649\"},\"expiration\":\"2023-11-24T19:09:56\",\"fill_or_kill\":false,\"extensions\":{\"on_fill\":[[0,{\"fee_asset_id\":\"1.3.0\",\"spread_percent\":100,\"size_percent\":10000,\"expiration_seconds\":1000000000,\"repeat\":true}]]}}]],\"extensions\":[],\"signatures\":[]}",
            []
        ],
        "appName": "Static Bitshares Astro web app",
        "chain": "BTS",
        "browser": "web browser",
        "origin": "localhost"
    }
}

Which produces the following error:

{
  "code": 4010001,
  "message": "Execution error: missing required active authority: Missing Active Authority 1.2.1811495",
  "data": {
    "code": 4010001,
    "name": "tx_missing_active_auth",
    "message": "missing required active authority",
    "stack": [
      {
        "context": {
          "level": "error",
          "file": "transaction.cpp",
          "line": 339,
          "method": "verify_authority",
          "hostname": "",
          "thread_name": "th_a",
          "timestamp": "2023-11-24T18:13:21"
        },
        "format": "Missing Active Authority ${id}",
        "data": {
          "id": "1.2.1811495",
          "auth": {
            "weight_threshold": 1,
            "account_auths": [],
            "key_auths": [["removed_key", 1]],
            "address_auths": []
          },
          "owner": {
            "weight_threshold": 1,
            "account_auths": [],
            "key_auths": [["removed_key", 1]],
            "address_auths": []
          }
        }
      },
      {
        "context": {
          "level": "warn",
          "file": "transaction.cpp",
          "line": 347,
          "method": "verify_authority",
          "hostname": "",
          "thread_name": "th_a",
          "timestamp": "2023-11-24T18:13:21"
        },
        "format": "",
        "data": {
          "rejected_custom_auths": [],
          "ops": [
            [
              1,
              {
                "fee": { "amount": 48260, "asset_id": "1.3.0" },
                "seller": "1.2.1811495",
                "amount_to_sell": { "amount": 9090909, "asset_id": "1.3.0" },
                "min_to_receive": { "amount": 10000, "asset_id": "1.3.5649" },
                "expiration": "2023-11-24T19:09:56",
                "fill_or_kill": false,
                "extensions": {
                  "on_fill": [
                    [
                      0,
                      {
                        "fee_asset_id": "1.3.0",
                        "spread_percent": 100,
                        "size_percent": 10000,
                        "expiration_seconds": 1000000000,
                        "repeat": true,
                        "extensions": []
                      }
                    ]
                  ]
                }
              }
            ]
          ],
          "sigs": ["removed_key"]
        }
      },
      {
        "context": {
          "level": "warn",
          "file": "transaction.cpp",
          "line": 475,
          "method": "verify_authority",
          "hostname": "",
          "thread_name": "th_a",
          "timestamp": "2023-11-24T18:13:21"
        },
        "format": "",
        "data": {
          "*this": {
            "ref_block_num": 55684,
            "ref_block_prefix": 739726531,
            "expiration": "2023-11-24T20:11:28",
            "operations": [
              [
                1,
                {
                  "fee": { "amount": 48260, "asset_id": "1.3.0" },
                  "seller": "1.2.1811495",
                  "amount_to_sell": { "amount": 9090909, "asset_id": "1.3.0" },
                  "min_to_receive": { "amount": 10000, "asset_id": "1.3.5649" },
                  "expiration": "2023-11-24T19:09:56",
                  "fill_or_kill": false,
                  "extensions": {
                    "on_fill": [
                      [
                        0,
                        {
                          "fee_asset_id": "1.3.0",
                          "spread_percent": 100,
                          "size_percent": 10000,
                          "expiration_seconds": 1000000000,
                          "repeat": true,
                          "extensions": []
                        }
                      ]
                    ]
                  }
                }
              ]
            ],
            "extensions": [],
            "signatures": [
              "removed_key"
            ]
          }
        }
      },
      {
        "context": {
          "level": "warn",
          "file": "db_block.cpp",
          "line": 782,
          "method": "_apply_transaction",
          "hostname": "",
          "thread_name": "th_a",
          "timestamp": "2023-11-24T18:13:21"
        },
        "format": "",
        "data": {
          "trx": {
            "ref_block_num": 55684,
            "ref_block_prefix": 739726531,
            "expiration": "2023-11-24T20:11:28",
            "operations": [
              [
                1,
                {
                  "fee": { "amount": 48260, "asset_id": "1.3.0" },
                  "seller": "1.2.1811495",
                  "amount_to_sell": { "amount": 9090909, "asset_id": "1.3.0" },
                  "min_to_receive": { "amount": 10000, "asset_id": "1.3.5649" },
                  "expiration": "2023-11-24T19:09:56",
                  "fill_or_kill": false,
                  "extensions": {
                    "on_fill": [
                      [
                        0,
                        {
                          "fee_asset_id": "1.3.0",
                          "spread_percent": 100,
                          "size_percent": 10000,
                          "expiration_seconds": 1000000000,
                          "repeat": true,
                          "extensions": []
                        }
                      ]
                    ]
                  }
                }
              ]
            ],
            "extensions": [],
            "signatures": [
              "removed_signature"
            ]
          }
        }
      },
      {
        "context": {
          "level": "warn",
          "file": "db_block.cpp",
          "line": 280,
          "method": "push_transaction",
          "hostname": "",
          "thread_name": "th_a",
          "timestamp": "2023-11-24T18:13:21"
        },
        "format": "",
        "data": {
          "trx": {
            "ref_block_num": 55684,
            "ref_block_prefix": 739726531,
            "expiration": "2023-11-24T20:11:28",
            "operations": [
              [
                1,
                {
                  "fee": { "amount": 48260, "asset_id": "1.3.0" },
                  "seller": "1.2.1811495",
                  "amount_to_sell": { "amount": 9090909, "asset_id": "1.3.0" },
                  "min_to_receive": { "amount": 10000, "asset_id": "1.3.5649" },
                  "expiration": "2023-11-24T19:09:56",
                  "fill_or_kill": false,
                  "extensions": {
                    "on_fill": [
                      [
                        0,
                        {
                          "fee_asset_id": "1.3.0",
                          "spread_percent": 100,
                          "size_percent": 10000,
                          "expiration_seconds": 1000000000,
                          "repeat": true,
                          "extensions": []
                        }
                      ]
                    ]
                  }
                }
              ]
            ],
            "extensions": [],
            "signatures": [
              "removed"
            ]
          }
        }
      }
    ]
  },
  "digest": "removed",
  "transaction": "removed"
}

This is also the case with a testnet account.

Is this wrong?

"extensions": {
            "on_fill": [
                [
                    0,
                    {
                        "fee_asset_id": "1.3.0",
                        "spread_percent": 100,
                        "size_percent": 10000,
                        "expiration_seconds": 1000000000,
                        "repeat": true
                    }
                ]
            ]
        }

Or is it just the new bitsharesjs serialization code we've got an issue with?

If I do:

{
  on_fill: {
    fee_asset_id: "1.3.0",
    spread_percent: spreadPercent ? spreadPercent * 100 : 0,
    size_percent: sizePercent ? sizePercent * 100 : 0,
    expiration_seconds: 1000000000,
    repeat: repeat,
  },
}

Then beet shows:

{
    "fee": {
        "amount": "48260",
        "asset_id": "1.3.0"
    },
    "seller": "1.2.1811495",
    "amount_to_sell": {
        "amount": "9090910",
        "asset_id": "1.3.0"
    },
    "min_to_receive": {
        "amount": "10000",
        "asset_id": "1.3.5649"
    },
    "expiration": "2023-11-24T20:03:56",
    "fill_or_kill": false,
    "extensions": {
        "on_fill": []
    }
}

And then trying this doesn't work neither:

{
  on_fill: [
    "limit_order_create",
    {
      fee_asset_id: "1.3.0",
      spread_percent: spreadPercent ? spreadPercent * 100 : 0,
      size_percent: sizePercent ? sizePercent * 100 : 0,
      expiration_seconds: 1000000000,
      repeat: repeat,
    },
  ],
}

It causes the beet api to throw this error:

error: limit_order_create.extensions     cause: value required operation l undefined 
 stack: Error: value required operation l undefined

@grctest the error usually means there is a serialization issue. However, the code looks OK. Maybe the extra commas at the end of each class caused the problem? I noticed that there was no comma there.

To debug, please print out the serialization result (the binary form) of the transaction, and compare it with the result of get_transaction_hex_without_sig API. See https://github.com/bitshares/bitsharesjs/issues/13#issuecomment-348681279 and https://github.com/bitshares/bitsharesjs/issues/13#issuecomment-348684435 for examples.

Oh wait, it looks like the extensions field is missing in create_take_profit_order_action.

Oh wait, it looks like the extensions field is missing in create_take_profit_order_action.

Should be fixed by https://github.com/bitshares/bitsharesjs/pull/126 .

Alright, I've successfully broadcast a limit order with OSO enabled:

[
    {
        "id": "071849232af8da8f9dd318cad1b1135348615308",
        "block_num": 84932251,
        "trx_num": 0,
        "trx": {
            "ref_block_num": 63130,
            "ref_block_prefix": 3934157669,
            "expiration": "2023-11-25T02:25:57",
            "operations": [
                [
                    1,
                    {
                        "fee": {
                            "amount": 48260,
                            "asset_id": "1.3.0"
                        },
                        "seller": "1.2.1811495",
                        "amount_to_sell": {
                            "amount": 90909,
                            "asset_id": "1.3.0"
                        },
                        "min_to_receive": {
                            "amount": 100,
                            "asset_id": "1.3.5649"
                        },
                        "expiration": "2023-11-25T01:20:47",
                        "fill_or_kill": false,
                        "extensions": {
                            "on_fill": [
                                [
                                    0,
                                    {
                                        "fee_asset_id": "1.3.0",
                                        "spread_percent": 100,
                                        "size_percent": 10000,
                                        "expiration_seconds": 1000000000,
                                        "repeat": true,
                                        "extensions": []
                                    }
                                ]
                            ]
                        }
                    }
                ]
            ],
            "extensions": [],
            "signatures": ["signature"
            ],
            "operation_results": [
                [
                    1,
                    "1.7.531970513"
                ]
            ]
        }
    }
]

Still need to test out the limit order update (with/without OSO); need to create an astro page to support this.

https://blocksights.info/#/objects/1.7.531970513

For limit_order_update, the field delta_amount_to_sell since it's delta, when I update a limit order, this should be the positive/negative difference, instead of just the new amount value? Ie if before it was 2, and now it's 1, the delta_amount_to_sell should be -1 instead of 1 , right?

Should have the limit order update astro ui page completed within the next day or so, with the above info provided :)

For limit_order_update, the field delta_amount_to_sell since it's delta, when I update a limit order, this should be the positive/negative difference, instead of just the new amount value? Ie if before it was 2, and now it's 1, the delta_amount_to_sell should be -1 instead of 1 , right?

Right.

I'm encountering this issue when I try to update a limit order:

[
    {
        "fee": {
            "amount": 0,
            "asset_id": "1.3.0"
        },
        "account_id_type": "1.2.1811495",
        "limit_order_id_type": "1.7.514355466",
        "on_fill": [
            [
                0,
                {
                    "fee_asset_id": "1.3.0",
                    "spread_percent": 100,
                    "size_percent": 10000,
                    "expiration_seconds": 1000000000,
                    "repeat": true
                }
            ]
        ],
        "extensions": [],
        "delta_amount_to_sell": {
            "amount": -69405197359,
            "asset_id": "1.3.6149"
        }
    }
]

{
    "code": 4010006,
    "message": "Execution error: committee account cannot directly approve transaction: Committee account may only propose transactions",
    "data": {
        "code": 4010006,
        "name": "invalid_committee_approval",
        "message": "committee account cannot directly approve transaction",
        "stack": [
            {
                "context": {
                    "level": "error",
                    "file": "transaction.cpp",
                    "line": 319,
                    "method": "verify_authority",
                    "hostname": "",
                    "thread_name": "th_a",
                    "timestamp": "2023-12-06T14:44:16"
                },
                "format": "Committee account may only propose transactions",
                "data": {}
            },
            {
                "context": {
                    "level": "warn",
                    "file": "transaction.cpp",
                    "line": 347,
                    "method": "verify_authority",
                    "hostname": "",
                    "thread_name": "th_a",
                    "timestamp": "2023-12-06T14:44:16"
                },
                "format": "",
                "data": {
                    "rejected_custom_auths": [],
                    "ops": [
                        [
                            77,
                            {
                                "fee": {
                                    "amount": 37500,
                                    "asset_id": "1.3.0"
                                },
                                "seller": "1.2.0",
                                "order": "1.7.0",
                                "delta_amount_to_sell": {
                                    "amount": "-900000000000",
                                    "asset_id": "1.3.6303"
                                },
                                "new_expiration": "2023-12-07T13:53:26",
                                "on_fill": [
                                    [
                                        0,
                                        {
                                            "fee_asset_id": "1.3.0",
                                            "spread_percent": 135,
                                            "size_percent": 10000,
                                            "expiration_seconds": 1000000000,
                                            "repeat": true,
                                            "extensions": []
                                        }
                                    ]
                                ],
                                "extensions": []
                            }
                        ]
                    ],
                    "sigs": [
                        "signature_removed"
                    ]
                }
            },
            {
                "context": {
                    "level": "warn",
                    "file": "transaction.cpp",
                    "line": 475,
                    "method": "verify_authority",
                    "hostname": "",
                    "thread_name": "th_a",
                    "timestamp": "2023-12-06T14:44:16"
                },
                "format": "",
                "data": {
                    "*this": {
                        "ref_block_num": 2939,
                        "ref_block_prefix": 2509600036,
                        "expiration": "2023-12-06T16:43:06",
                        "operations": [
                            [
                                77,
                                {
                                    "fee": {
                                        "amount": 37500,
                                        "asset_id": "1.3.0"
                                    },
                                    "seller": "1.2.0",
                                    "order": "1.7.0",
                                    "delta_amount_to_sell": {
                                        "amount": "-900000000000",
                                        "asset_id": "1.3.6303"
                                    },
                                    "new_expiration": "2023-12-07T13:53:26",
                                    "on_fill": [
                                        [
                                            0,
                                            {
                                                "fee_asset_id": "1.3.0",
                                                "spread_percent": 135,
                                                "size_percent": 10000,
                                                "expiration_seconds": 1000000000,
                                                "repeat": true,
                                                "extensions": []
                                            }
                                        ]
                                    ],
                                    "extensions": []
                                }
                            ]
                        ],
                        "extensions": [],
                        "signatures": [
                            "signature_removed"
                        ]
                    }
                }
            },
            {
                "context": {
                    "level": "warn",
                    "file": "db_block.cpp",
                    "line": 782,
                    "method": "_apply_transaction",
                    "hostname": "",
                    "thread_name": "th_a",
                    "timestamp": "2023-12-06T14:44:16"
                },
                "format": "",
                "data": {
                    "trx": {
                        "ref_block_num": 2939,
                        "ref_block_prefix": 2509600036,
                        "expiration": "2023-12-06T16:43:06",
                        "operations": [
                            [
                                77,
                                {
                                    "fee": {
                                        "amount": 37500,
                                        "asset_id": "1.3.0"
                                    },
                                    "seller": "1.2.0",
                                    "order": "1.7.0",
                                    "delta_amount_to_sell": {
                                        "amount": "-900000000000",
                                        "asset_id": "1.3.6303"
                                    },
                                    "new_expiration": "2023-12-07T13:53:26",
                                    "on_fill": [
                                        [
                                            0,
                                            {
                                                "fee_asset_id": "1.3.0",
                                                "spread_percent": 135,
                                                "size_percent": 10000,
                                                "expiration_seconds": 1000000000,
                                                "repeat": true,
                                                "extensions": []
                                            }
                                        ]
                                    ],
                                    "extensions": []
                                }
                            ]
                        ],
                        "extensions": [],
                        "signatures": [
                            "signature_removed"
                        ]
                    }
                }
            },
            {
                "context": {
                    "level": "warn",
                    "file": "db_block.cpp",
                    "line": 280,
                    "method": "push_transaction",
                    "hostname": "",
                    "thread_name": "th_a",
                    "timestamp": "2023-12-06T14:44:16"
                },
                "format": "",
                "data": {
                    "trx": {
                        "ref_block_num": 2939,
                        "ref_block_prefix": 2509600036,
                        "expiration": "2023-12-06T16:43:06",
                        "operations": [
                            [
                                77,
                                {
                                    "fee": {
                                        "amount": 37500,
                                        "asset_id": "1.3.0"
                                    },
                                    "seller": "1.2.0",
                                    "order": "1.7.0",
                                    "delta_amount_to_sell": {
                                        "amount": "-900000000000",
                                        "asset_id": "1.3.6303"
                                    },
                                    "new_expiration": "2023-12-07T13:53:26",
                                    "on_fill": [
                                        [
                                            0,
                                            {
                                                "fee_asset_id": "1.3.0",
                                                "spread_percent": 135,
                                                "size_percent": 10000,
                                                "expiration_seconds": 1000000000,
                                                "repeat": true,
                                                "extensions": []
                                            }
                                        ]
                                    ],
                                    "extensions": []
                                }
                            ]
                        ],
                        "extensions": [],
                        "signatures": [
                            "signature_removed"
                        ]
                    }
                }
            }
        ]
    },
    "digest": "signature_removed",
    "transaction": "signature_removed"
}

It looks like you need to replace

        "account_id_type": "1.2.1811495",
        "limit_order_id_type": "1.7.514355466",

in your transaction with

        "seller": "1.2.1811495",
        "order": "1.7.514355466",

Should be fixed by https://github.com/bitshares/bitsharesjs/pull/128.

By the way, it looks like the error message was not for the transaction you sent, because the amounts and asset IDs don't match.

Applied the same changes to bts-buntime package too, with your fix it now broadcasts the limit order update operation successfully: https://blocksights.info/#/operations/1.11.1294152022

By the way, it looks like the error message was not for the transaction you sent, because the amounts and asset IDs don't match.

Yeah, I don't understand where the error about committee approval came from, sorted now at least 👍

bitshares / bitsharesjs

Suez serialization support #123