Can operation_results please show the operation id(s) of all ops in the tx?

[xeroc]

Updated by @pmconrad to recap discussion so far, 2019-04-12

As a client developer I want to query the database for all operation_history and/or account_history entries related to a given block.

Original message below

+-------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------+
| Key                     | Value                                                                                                                                            |
+-------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------+
| extensions              | []                                                                                                                                               |
| previous                | 00dc788eee4533479fde0b9189e310017beade53                                                                                                         |
| timestamp               | 2017-03-02T11:56:33                                                                                                                              |
| transaction_merkle_root | 2b114c0d45b77198880cdcce5a8be512014b622b                                                                                                         |
| transactions            | [                                                                                                                                                |
|                         |     {                                                                                                                                            |
|                         |         "ref_block_num": 30862,                                                                                                                  |
|                         |         "signatures": [                                                                                                                          |
|                         |             "1f620755f8c7384ed94a748f3f6aec083892b1a8cfab34a411c12fa01b29fa25a70e0f181fa083c76247692938b58e8f9baac0e98d0e38cc271619b7ed72dece07" |
|                         |         ],                                                                                                                                       |
|                         |         "operations": [                                                                                                                          |
|                         |             [                                                                                                                                    |
|                         |                 0,                                                                                                                               |
|                         |                 {                                                                                                                                |
|                         |                     "to": "1.2.36644",                                                                                                           |
|                         |                     "from": "1.2.100665",                                                                                                        |
|                         |                     "extensions": [],                                                                                                            |
|                         |                     "memo": {                                                                                                                    |
|                         |                         "to": "BTS7tqXBbtJwDoT8b2LY3kx547ocVfSacTirBMFEsDyhGf2FNR48p",                                                           |                                                                 
|                         |                         "message": "1afad56165a60016979d78d3162bc172d83a40025c7e57873625152367c483e0",                                           |                                                                 
|                         |                         "from": "BTS5AjtESJLTuBmL5hDSLeuigpkWDTe4ZUSjugwAvHSzMJWEUbDAf",                                                         |                                                                 
|                         |                         "nonce": "381044681844390"                                                                                               |                                                                 
|                         |                     },                                                                                                                           |                                                                 
|                         |                     "fee": {                                                                                                                     |                                                                 
|                         |                         "asset_id": "1.3.0",                                                                                                     |                                                                 
|                         |                         "amount": 279652                                                                                                         |                                                                 
|                         |                     },                                                                                                                           |                                                                 
|                         |                     "amount": {                                                                                                                  |                                                                 
|                         |                         "asset_id": "1.3.861",                                                                                                   |                                                                 
|                         |                         "amount": 7294494                                                                                                        |                                                                 
|                         |                     }                                                                                                                            |                                                                 
|                         |                 }                                                                                                                                |                                                                 
|                         |             ]                                                                                                                                    |                                                                 
|                         |         ],                                                                                                                                       |                                                                 
|                         |         "operation_results": [                                                                                                                   |                                                                 
|                         |             [                                                                                                                                    |                                                                 
|                         |                 0,                                                                                                                               |                                                                 
|                         |                 {}                                                                                                                               |                                                                 
|                         |             ]                                                                                                                                    |                                                                 
|                         |         ],                                                                                                                                       |                                                                 
|                         |         "expiration": "2017-03-02T11:56:48",                                                                                                     |                                                                 
|                         |         "extensions": [],                                                                                                                        |                                                                 
|                         |         "ref_block_prefix": 1194542574                                                                                                           |                                                                 
|                         |     }                                                                                                                                            |                                                                 
|                         | ]                                                                                                                                                |                                                                 
| witness                 | 1.6.17                                                                                                                                           |                                                                 
| witness_signature       | 201c2f657322e05c71a32dd975419365978dfe96c1d29df41e16ff5a079b7a545d48f0d81d23b1a3c7b0b90853d0d7b8fea4ba126707fcd5c762a16f33c2cd58d7               |                                                                 
+-------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------+         ```

as you can see .. the `operation_ids` are empty .. that shouldn't be the case but the operation should get an operation_id when it is included in a block

[oxarbitrage]

I was checking this issue since a while and trying to understand how the operation_results work. Here are some of my findings.

Please check the following comment in code:

https://github.com/bitshares/bitshares-core/blob/master/libraries/chain/include/graphene/chain/protocol/transaction.hpp#L184-L196

It states that operation_results will be filled only if new objects are created.

So, an account creation operation will return the new account object, this is an object of 1.2.X as for example:

http://cryptofresh.com/tx/2dd4394e7dcf066dd5a64b3db51b11aa84ec8f22

Use "show raw transaction" to see details as the operation_results.

A limit order operation(cryptofresh call this "wants" operations) will return a limit order object of 1.7.X :

http://cryptofresh.com/tx/0b4ad64a76eb5ac3db8c27c53dcfba8d0884bb8a

For reference, a list of bitshare object can be found at: http://docs.bitshares.org/development/blockchain/objects.html

Note that in the first sample(account creation) the operation type is "5".

For reference the operation types are found at: https://github.com/bitshares/bitshares-core/blob/master/libraries/chain/include/graphene/chain/protocol/operations.hpp#L50-L94

With transfer_operation = 0 and counting down, then 5 = account_create_operation:

https://github.com/bitshares/bitshares-core/blob/master/libraries/chain/include/graphene/chain/protocol/operations.hpp#L55

In a transfer operation(sample from @xeroc is a transfer) there is no new object created so the operation results are empty.

Xeroc operation can be found in cryptofresh here:

http://cryptofresh.com/tx/8bbd533363da0a459077d2d45e4c7c67570bb1a1

Transfer are not the only operation types that don't return an object and an empty operation results.

Exploring the blockchain i was able to find for example the creation of a proposal(operation type 22) :

http://cryptofresh.com/tx/47c2fd88fbef9fbc0415622b520d46c2c8d988f6

Returning as expected a proposal object (1.10.x)

While updates to proposal will return empty operation_results, such as:

http://cryptofresh.com/tx/e0b0ff5d1cfe2d7a1ff7756a471ac1e9e7a22f2c

Please note in the above example that the index of the results_operations correspond with the index of each transaction in the block. First result is for first operation.

A call order update will return empty operation results:

http://cryptofresh.com/tx/083e6eaafa89a80f3d023f9d827b20d2f2c58b89

An account update operation will return empty results as the account object is already created:

http://cryptofresh.com/tx/f30ca7927167604fc1e07b5425550d2ca0df83c7

One operation type that goes against this rule(only show results if new objects are created) is the limit order operation cancel, such as:

http://cryptofresh.com/tx/8dcb994576d092c018a97c4e27de1e53f4848624

Where amount and asset_id are included into the response. If someone haves any idea about why this values are included will be great to share here.

Hope it helps to bring some light in the subject.

[xeroc]

I get that now. Thanks for the explanation. Given that each and every operation - after being added to the blockchain - obtain their own operation id 2.9.x, would it be possible that the API offers those numbers when doing a get_block?

[elmato]

@xeroc after each operation is applied a new operation_history_object (1.11.x) is created that has all the necessary information where the operation was applied. (block/tx index in block/operation index in tx)

uint32_t database::push_applied_operation( const operation& op )
{
   _applied_ops.emplace_back(op);
   operation_history_object& oh = *(_applied_ops.back());
   oh.block_num    = _current_block_num;
   oh.trx_in_block = _current_trx_in_block;
   oh.op_in_trx    = _current_op_in_trx;
   oh.virtual_op   = _current_virtual_op++;
   return _applied_ops.size() - 1;
}

But the get_block function just return the json serialization of a signed_block And signed_block only has operations inside and not operation_history_object

So, the only way i can imagine of doing what you request is to rewrite the get_block function and manually lookup every operation_history_object for every operation inside the block and add it to the response.

[oxarbitrage]

I am on the process of doing just that. Thanks @elmato. I will have a pull request for you guys to check on monday.

El may. 6, 2017 7:53 PM, "Matias Romeo" notifications@github.com escribió:

@xeroc https://github.com/xeroc after each operation is applied a new operation_history_object (1.11.x) is created that has all the necessary information where the operation was applied. (block/tx index in block/operation index in tx)

uint32_t database::push_applied_operation( const operation& op ) { _applied_ops.emplace_back(op); operation_history_object& oh = *(_applied_ops.back()); oh.block_num = _current_block_num; oh.trx_in_block = _current_trx_in_block; oh.op_in_trx = _current_op_in_trx; oh.virtual_op = _current_virtual_op++; return _applied_ops.size() - 1; }

But the get_block function just return the json serialization of a signed_block And signed_block only has operations inside and not operation_history_object

So, the only way i can imagine of doing what you request is to rewrite the get_block function and manually lookup every operation_history_object for every operation inside the block and add it to the response.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/bitshares/bitshares-core/issues/243#issuecomment-299670796, or mute the thread https://github.com/notifications/unsubscribe-auth/AUrjaSd0JI83JwtPauW25B0PcWVHP30Xks5r3PnvgaJpZM4MQ-HZ .

[abitmore]

I think perhaps it's better to do it via a new API. By the way #265 is related. ID's are not guaranteed to be consistent.

[oxarbitrage]

https://github.com/bitshares/bitshares-core/pull/275

can be replaced by 1.11.X objects instead of 2.9.X.

[abitmore]

Just a note, changes made in #296 have been reverted. This is not done yet.

[abitmore]

We can add an API to return all operations from data stored in account_history plugin, with a block number as parameter. Since the data contains trx_in_block and op_in_trx, clients would be able to merge result of this API into result of get_block API.

Note: due to partial-operations, some history entries could have already been removed, so the result can be incomplete.

Alternatively, we can extend the get_block API to include the operation IDs and the virtual operations.

Pros and cons: The 2nd approach is more convenient for clients, but will likely cause higher load for API nodes.

Thoughts?

[jmjatlanta]

Approach 1: Clients for an API node with partial-operations turned on would still get the block information, and operation IDs for probably the transactions they are interested in. I am sure there are use cases for complete block information, but I would guess this approach would cover the majority of cases.

Approach 2: Could this be done as a plug-in? Perhaps it could be contributed later.

[abitmore]

In my previous comment, approach 1 and approach 2 have the same logic, the difference is approach 1 requires more client-side code, but approach 2 is pure server-side code, both would produce incomplete results due to partial-operation thus may be confusing for clients.

Using a plugin to store the IDs data on disk seems doable, this (approach 3) can produce completed results, thus would be better UX. It requires much bigger disk though.

[jmjatlanta]

Thank you for the clarifications. Here is my $0.02:

I say approach 3 is out (for now). Approach 2 would probably cost less (cycles/bandwidth) if most use cases require the IDs. If the vast majority do not need it, I would say approach 1 would be better. It would cause more code on the client side to combine the data, but an overloaded API server hurts all who are connected to it.

[abitmore]

Ultimately, virtual operations by block number can be queried from ES.

I forgot why I added this issue to 3.3.0 project/milestone.

[abitmore]

I'll add an API related to this, hope it can be included in 3.3 release.

[abitmore]

Created get_block_operation_history API in #1899, which will return a list of operations (including virtual operations) in specified block. Limitations of that API are noted in docs:

* @note the data is fetched from account_history plugin, thus the result is possible to
*       be incomplete due to the partial-operations option configured in the API node.
*       For complete data, it is recommended to query from ElasticSearch where data is
*       maintained by @a elastic_search plugin.

[abitmore]

Since an ES-based solution is more suitable, I closed PR #1899 and created https://github.com/bitshares/bitshares-explorer-api/issues/65. Closing this one.

[abitmore]

Done via #1899.