Postgres sql database design

Colm3na / polkastats-backend-v3

polkastats.io backend repository

Apache License 2.0

18 stars 15 forks source link

Postgres sql database design #4

Open ghost opened 4 years ago

mariopino commented 4 years ago

I think PolkaScan is currently the most complete database schema for substrate blockchains.

One easy pending task:

Deploy polkascan-pre and get the generated MySQL, we can post in the repo for reference. Then we can study and see if we can use it, porting to Postgres should be not too complicated.

https://github.com/polkascan/polkascan-pre

Any volunteer? :-)

derfredy commented 4 years ago

We should have a look and leverage this team work. They are doing good job.

https://github.com/usetech-llc/polkadot_psql_indexer?files=1

ghost commented 4 years ago

I added sql dump of polkascan on new branch. Analizaré el enlace de Fredy.

mariopino commented 4 years ago

We should have a look and leverage this team work. They are doing good job.

https://github.com/usetech-llc/polkadot_psql_indexer?files=1

Really interesting! developed in C# and seems that they are already using the PolkaScan schema in Postgres.

derfredy commented 4 years ago

I added sql dump of polkascan on new branch. Analizaré el enlace de Fredy.

The file has not only the schema but a lot of data. I'd leave it like this and start working with polkadot_psql_indexer from usetech team data base. We are now sure that they leveraged polkascan schema as well. So part of the work is already done.

This is a critical task needed for other team members to keep going. (Javi and Aitor)

derfredy commented 4 years ago

Following a tlf call, I'll try to be accurate with the nexts steps I would suggest.

Check if the sql dump we already have from polkascan match the one the other team is using. Most likely yes.
If that's the case, we could do 2 things. Clean the actual dump from data leaving a clean schema or try a new sql dump from the other team.
Once we have the sql schema, see if it breaks actual crawlers. (i.e. we call tables block and events, maybe they call them differently)

With those steps finished we unlock issues: #7 #5 #8 #6 #9

derfredy commented 4 years ago

I am sorry @melozo but this task is no blocking anything. We can keep going without it. I have realized today after a conversation with @mariopino Sorry bro!

derfredy commented 4 years ago

@melozo , do you need my help or @mariopino to analyse this issue? Please, let us know or tell me how to proceed. I see you moved from blocked to analysis.

ghost commented 4 years ago

The next weekend i will work on this issue, if you have time, perfect. Thanks!