In order to create the Blockchain Streamer, we need to create a query that extracts a large chunk of the blockchain file.
Currently our Metadata table has the following columns
Height - This is the non-decreasing ID of the block, starting from zero.
Position - This represents the position of the byte slice in the blockchain file. The blockchain file contains one continuous byte slice representing the concatenation of all blocks in byte form. So it is necessary to track where a particular block begins if we want to extract it.
Size - This represents the length of the block in byte form. Blocks can vary in size because they can have varying amounts of Data in their body. You can extract a block at height h by getting its position p and reading in its size s in bytes from position p.
Example: I want to extract block 478. I look in the metadata table at the row that has 478, then I look at its position and height. Suppose its position is 3016, and its size is 289. I will open the blockchain file, move my file pointer to index 3016, and read in 289 bytes from that position. You can look at this function to see how this is done: https://github.com/SIGBlockchain/project_aurum/blob/master/internal/blockchain/blockchain.go#L96
Batching blocks is a bit different. We need to make a query that given height h and a number n, extracts blocks from h to h + (n - 1). So if I give this query h = 478 and n = 5, that means I want to extract blocks 478, 479, 480, 481, and 482.
This is a little different from the function I linked above. Given a height h and a count n, go into the table and return rows with columns height, size, and position for all blocks x within the range:
h >= x < ( h + n )
[x] Create a new SQL query in sqlstatements.go that provides these rows.
In order to create the Blockchain Streamer, we need to create a query that extracts a large chunk of the blockchain file.
Currently our Metadata table has the following columns
Example: I want to extract block 478. I look in the metadata table at the row that has 478, then I look at its position and height. Suppose its position is 3016, and its size is 289. I will open the blockchain file, move my file pointer to index 3016, and read in 289 bytes from that position. You can look at this function to see how this is done: https://github.com/SIGBlockchain/project_aurum/blob/master/internal/blockchain/blockchain.go#L96
Batching blocks is a bit different. We need to make a query that given height h and a number n, extracts blocks from h to h + (n - 1). So if I give this query h = 478 and n = 5, that means I want to extract blocks 478, 479, 480, 481, and 482.
This is a little different from the function I linked above. Given a height h and a count n, go into the table and return rows with columns height, size, and position for all blocks x within the range:
h >= x < ( h + n )
sqlstatements.gothat provides these rows.