Create separate microservice for circulating supply calculation & saving it to DB

[telezhnaya]

I understood that it's better to clarify how it should look like.

1. Are we sure that we want the service only for calculating circulating supply? Maybe it's better to take into account that it could calculate other values as well?

2. General idea: I see it as a CLI utility that

   • [optionally] takes block_height as an input (or taking latest)
   • calculates the last block_height for the previous day (read: rounds it) to remove the dependency on the exact time of invoking (could be switched off)
   • calculates circulating supply value for this block_height
   • insert the value to the DB (could be switched off)
   • give the value to the output

   It could be invoked by cron to calculate it once a day. I will also provide it with a list of historical block_heights to fill the table at the very beginning.

3. It should be written in Rust.
4. The DB migration and the creation of the table are better to put into near-indexer-for-explorer since we have all the migrations there. It's better not to shuffle it up between several projects.
5. The DB will contain one more table circulating_supply with columns block_height (primary key and foreign key to blocks table), total_supply, circulating_supply, locked_supply_on_lockups, locked_foundation_supply. There will be one line for one day.

6. I need to create a separate project in Near organization (uuuh, awesome feeling). Let's discuss the name. It fully depends on p.1, so I am waiting for your opinion.

7. After all this stuff, I can remove my ugly solution here https://github.com/near/near-explorer/pull/642 and just query this new table

@frol @khorolets

[frol]

Are we sure that we want the service only for calculating circulating supply? Maybe it's better to take into account that it could calculate other values as well?

Here is what I am thinking:

There are various approaches we can use:

1. Implement these side-computations in the Indexer for Explorer itself in a separate thread
2. Implement all the side-computations in a separate service
3. Implement all the side-computations in individual services ("microservices"-like idea)

(1) feels like we implement a monolith and to implement a tiny change to the analytics we will need to roll out the whole indexer.

(2) and (3) have the overhead of being independent projects, gits, deployments, etc, but at the same time, they can have their out release iteration. So it is good and bad at the same time.

(1) has a benefit that we already have the setup (migrations, database access, models), and can even have direct access to the archival blockchain data (not all the information is stored into Postgres, and sometimes you may even want to do view-calls to the contracts)

Today I lean towards the pragmatic approach of re-using the setup we already have, so go with the first approach and just make sure we have a clean separation on the modules level inside the Indexer for Explorer implementation.

NOTE: I want us to keep in mind that we run several Indexer for Explorer instances for redundancy, so we should make sure our analytics don't produce duplicates.

I need to create a separate project in Near organization (uuuh, awesome feeling). Let's discuss the name. It fully depends on p.1, so I am waiting for your opinion.

It is not that day, yet :smile: