Open CowMuon opened 1 year ago
*Throwing research into comments here as I go
Database root table + branches
Chains
are really communities by other nomenclature in our DBChainNodes
is the most relevant table that actually relates to a chain
At the core Chains could be renamed to communities, fallout of effected foreign keys is pretty large:
SELECT
tc.table_name,
kcu.column_name,
ccu.table_name AS foreign_table_name,
ccu.column_name AS foreign_column_name
FROM
information_schema.table_constraints AS tc
JOIN information_schema.key_column_usage AS kcu
ON tc.constraint_name = kcu.constraint_name
JOIN information_schema.constraint_column_usage AS ccu
ON ccu.constraint_name = tc.constraint_name
WHERE tc.constraint_type = 'FOREIGN KEY' AND ccu.table_name = 'Chains';
The following tables would need a definite refactor per column names for chain id. And on further research there are many more tables referencing chain w/o a foreign key relationship(will documents these in next comments)
All tables referencing Chain
community
or community_id
but not both as part of the refactor/cleanup Current Scope of DB refactor look like it would include
App Scope
After talking with Ian we agreed it makes sense to split this ticket into 2 PRs.
origin
for events/entities which defines where the event/entity came from without referencing a community (Chains
). This will affect proposal links, event generation/storage, and ChainEntityMeta
.As mentioned in-call, I'll follow up with another comment describing some of the changes that are required chain-events side (mostly relating to PR 2).
Currently ALL events and entities are directly linked to a chain from the Chains
model (being renamed to Communities
). This is restrictive since it means an event cannot be consumed by more than 1 community. If we want a system that is truly a general indexer we need to be able to determine the origin of an event or entity without referencing any CW-specific data.
Thus we should introduce a new data point called origin
.
src/chains
to Bases
or something agnostic to chainEVM
and move all EVM chain-event listener modules to that directory
src/bases/
:
chain
in processor.ts
, storageFetcher.ts
, subscriberFunc.ts
, subscriber.ts
, and /filters
with origin
[Substrate:Stafi] some log info here
the appearance of the log will remain the same for event-feeds that don’t contain a contract address but since origin
contains contract addresses the new log prefixes for say Aave will be
[Aave:Ethereum:Aave:0x1234] some log info here
. While it might look strange to have Aave precede Ethereum in the prefix, this is intentional. Aave
in this case is meant to describe which network
or base
chain-events is using (info that would be unavailable to us while debugging unless we go to the contract address on Ethereum and see that it is an Aave contract).[SupportedNetwork:chain-name:protocol:module]
chain
with origin
in Listener.ts
origin
value and create event-feeds
name
in ChainNodes table for the ChainNode associated with each Substrate chain/community from the Chains table
has-chain-events-listener = true
type = 'chain'
base = 'substrate'
name
in ChainNodes table with the address
from the Contracts
table
has-chain-events-listener = true
base = 'ethereum'
name
in the ChainNodes table for the ChainNode associated with each Cosmos chain/community from the Chains table
has-chain-events-listener = true
type = 'chain'
base = 'cosmos'
chain
column with origin
in ChainEntities
and ChainEvents
tables (requires cross-database migration).@timolegros really great breakdown -- please break it out into a separate, linked ticket for chain event specific discussion.
@ianrowan couple comments:
@jnaviask
chain
column will need to be update in CEMetachain
names very temporarily. In this case CE related work could be unblocked by the migration and changes to app could be made fresh off of the migration as an immediate separate task.As far as automation, I'll keep looking into this. I've done some initial research here, and havent found an all encompassing solution(strategies I've seen so far just update models and really direct usage of them, but not nested etc) The de-risk on this side would be to continue to support aliases || actual column names until we can remove those guardrails
How exactly would aliasing work? Would it be at the database or the server level? If we can figure out a clean scope of work here, then we should create a ticket and get moving.
@jnaviask aliasing would happen at the app level in the models the way I'm thinking about it. Basically the exported attributes and the table definitions keys remain the same(as used across the app), but the options allow us to specify what column a given key points to.
So for example, somewhere using Addresses.chain_id
could still use that. but the model(in the app) would resolve that to community_id
And from there we could chip away at removing aliases/replacing with correct naming.
The Chains
table itself would be the only table name to change, so I think at a minimum we would refactor usage of Chains but alias all fkeys in other models where they are used
I really like this solution @ianrowan. Please put up a ticket for it, linked to this ticket (which will be converted to bucket), and then we can start executing.
@jnaviask created issue #3779
@ForestMars @jnaviask I'd like to transfer this ticket to someone on the PENG team as the only remaining variable semantic changes are on the client i.e. app.chain
and someone from PENG should lead that change given they will have a better understanding of when/how to do this. That part of the refactor was delayed because PENG had anticipated potentially removing the app
state objecting but months have passed and we have not progressed much in that direction.
I previously noted that this should be transferred to PENG on May 2: https://github.com/hicommonwealth/commonwealth/issues/5335#issuecomment-2089316120.
We need to clean up the semantics around how we handle (define) chains & communities. We've been discussing this for a bit, it's about time to get going on fixing it.