search

steemit / sbds

Steem Blockchain Data Service
https://developers.steem.io/#services-sbds
Other
37 stars 23 forks source link

sbds must also index virtual ops #88

Closed sneak closed 6 years ago

sneak commented 6 years ago

This is a prereq for #87.

sbds is a service with two parts: a blockchain follower that fetches blocks from steemd and breaks those blocks into transactions, breaks those transactions into operations, and then injects those operations into the right SQL table for that operation type. The second part is a JSON-RPC service that allows methods that do reads from that database and return json data about the info in one or more of those tables. The only writer is the blockchain follower, the JSON-RPC api is only reads.

Right now, sbds has a blockchain follower that keeps a block height and periodically queries steemd to see if the last irreversible block of the blockchain (as determined by the results of get_dynamic_global_properties from steemd) is higher than its last db-inserted block and if so, fetches the blocks between the last inserted block and the last irreversible block and inserts them.

The way you get virtual ops from steemd is with the get_ops_in_block(blocknum, true) json-rpc call.

Depending on how the block follower now works, there are two potential avenues:

1) add a second cursor tracking which blocks have had their virtual ops INSERTed too, and call get_ops_in_block for each, effectively having a second blockchain follower that inserts only virtual ops

2) update the primary follower to call get_ops_in_block on each block as the non-virtual ops are inserted

I think 2 is a much better design, but may involve more edge cases.

sneak commented 6 years ago

Ping @sneakyness

sneak commented 6 years ago

Per @roadscape

To add virtual ops to SBDS, we need to use get_ops_in_block in addition to get_block.

SBDS main routines are here, relevant are task_add_missing_blocks and task_stream_blocks: https://github.com/steemit/sbds/blob/0350085171f06280809918bcb25e055cd6bce8b8/sbds/storages/db/scripts/populate.py

The steemd call is get_ops_in_block(block_num, only_virtual). https://github.com/steemit/steem/blob/06e67bd4aea73391123eca99e1a22a8612b0c47e/libraries/app/database_api.cpp#L220

goldibex commented 6 years ago

in master at least, the only places where the account_history plugin is invoked are in database_api:

sneak commented 6 years ago

sbds cannot replicate get_state in full as it stands, but the data that certain get_state calls return can be implemented in new methods.

sneak commented 6 years ago

@goldibex this discussion about read calls is better suited in #87 - leave #88 (this) for the writer changes