Chain-Events Community Semantics Origin Refactor

[timolegros]

The Problem

Currently ALL events and entities are directly linked to a chain from the Chains model (being renamed to Communities). This is restrictive since it means an event cannot be consumed by more than 1 community. If we want a system that is truly a general indexer we need to be able to determine the origin of an event or entity without referencing any CW-specific data.

The Solution

Thus we should introduce a new data point called origin.

Directory structure refactor

• Rename src/chains to Bases or something agnostic to chain
• Create a directory EVM and move all EVM chain-event listener modules to that directory
  • aave, commonwealth, compound, erc20, erc721, moloch
  • Resulting high-level directories under src/bases/:
    • EVM
    • Cosmos
    • Substrate

Chain-Events Library Refactor

• Replace all instances of chain in processor.ts, storageFetcher.ts, subscriberFunc.ts, subscriber.ts, and /filters with origin
  • Instances of chain in these files are almost exclusively used for enhanced logging prefixes like [Substrate:Stafi] some log info here the appearance of the log will remain the same for event-feeds that don’t contain a contract address but since origin contains contract addresses the new log prefixes for say Aave will be [Aave:Ethereum:Aave:0x1234] some log info here. While it might look strange to have Aave precede Ethereum in the prefix, this is intentional. Aave in this case is meant to describe which network or base chain-events is using (info that would be unavailable to us while debugging unless we go to the contract address on Ethereum and see that it is an Aave contract).
  • New log prefixes: [SupportedNetwork:chain-name:protocol:module]
• Replace chain with origin in Listener.ts

Chain-Events Services Refactor

ChainSubscriber

• Refactor queries to get the origin value and create event-feeds
  • For Substrate chains, the origin is the name in ChainNodes table for the ChainNode associated with each Substrate chain/community from the Chains table
    • has-chain-events-listener = true
    • type = 'chain'
    • base = 'substrate'
  • For EVM contracts, the origin is a combination of name in ChainNodes table with the address from the Contracts table
    • has-chain-events-listener = true
    • base = 'ethereum'
  • For Cosmos, the origin is the name in the ChainNodes table for the ChainNode associated with each Cosmos chain/community from the Chains table
    • has-chain-events-listener = true
    • type = 'chain'
    • base = 'cosmos'
• Organize the queried event-feeds first by ChainNode (name/url) and then by SupportedNetwork
  • This enables us to eventually refactor listeners to be ChainNode centric i.e. only 1 listener per endpoint with that listener handling a variety of networks i.e. 1 listener to Ethereum processing Aave + compound + erc20

CE Database

• Replace chain column with origin in ChainEntities and ChainEvents tables (requires cross-database migration).

Acceptance Criteria

• No usage of chain anywhere in the chain-events package (only origin).

[timolegros]

Blocked by #3228 -> contains quite a few changes to the structure of the subscriber therefore I will wait until it is merged before I make additional changes to the subscriber for this issue.

[timolegros]

After reviewing the changes that would be required CE app-side to query the new chain-events format it seems to me that simply redefining chain to be ChainNode.name and then optionally including a contract address in the event may be a better approach. This is because a separate contract_address column that can be indexed in the database is better for querying than having the name + contract_address combined in a single column. Thus on the backend the chain + address will be stored separately but we can of course still combine both into an origin for logging/display purposes if needed.

Since chain is updated to community by Ian CW side there should be no conflicts with CE. Most of the logic above holds it simply won't be renamed to origin and there will be an extra contract_address column. @jnaviask thoughts?

[timolegros]

3963 is related to this issue. If the chains with has_chain_events_listener = true change then #3963 may become a blocker.

[timolegros]

3991 should be merged before continuing here to avoid unnecessary work.

[timolegros]

4020 should also be completed and merged before continuing. The problem currently is that ChainEntities on the chain-events side are no longer related to any specific community while ChainEntityMeta inherently are since they are titles set by community members/authors on existing proposals. Since this is the case we have the option of either refactoring ChainEntityMeta to make them work within a community-agnostic CE or completely delete the table and the title setting functionality that it supports.

Given that #3991 will be merged soon and #3542 was already merged we now have no more use for the ChainEntityMeta table and we can completely remove it without any loss of functionality (considering that only Substrate supported title setting and the feature has been broken since the Listeners have been broken as well).

[timolegros]

To be clear, #3991 #4020 and #3806 need to merged/completed before this ticket will be unblocked.

[timolegros]

Still blocked by #3991 and #4020. Once those 2 are merged I can work on resolving merge conflicts and splitting the existing PR into 3 separate PRs to avoid app downtime during deployment (aka ensure backward compatibility between CE/CW and then PR to remove the backward compatibility).

[timolegros]

Also, this is not a 3-point ticket. This ticket has been ongoing with lots of testing + preliminary changes required. This is a 5-point ticket.

[timolegros]

If chain-events is removed, this issue can be closed.

[timolegros]

CE will be removed per #4696