truflation / marketplace

MIT License
2 stars 3 forks source link

Bug: MySQL tables without primary keys #34

Open srust99 opened 2 months ago

srust99 commented 2 months ago

Hello,

Our systems have indicated that your MySQL cluster, db-mysql-sfo3-53387, has tables without primary keys. We have identified that MySQL tables without primary keys can lead to service replication issues that jeopardize performance and availability. If primary keys are not present for database tables exceeding 5,000 rows, data-loss can occur.

We urge you to take the following actions to ensure that your database db-mysql-sfo3-53387 continues to operate correctly:

  1. Review the following product documentation which details the issue and provides clear mitigation strategies for all impacted users.
  2. Implement primary keys for all database tables as described in the documentation.

See the full details at the following link: https://www.digitalocean.com/docs/databases/mysql/how-to/create-primary-keys/

We will continue to notify you of this issue until all MySQL tables have primary keys.

DigitalOcean

Blocked by:

Problems

zolotokrylin commented 2 months ago

@truflation/team-python-pipelines guys, please investigate and fix. Thank you!

zolotokrylin commented 2 months ago

@joequant @ambahk ?

joequant commented 2 months ago

This isnt an issue in production because we are not using hosted mysql. @ambahk is moving us to hosted mysql and the errors are coming from his test instances. Adding primary keys to our tables is a todo before we move our system to hosted mysql.

Jarryd-pretorius commented 2 months ago

Hey @joequant can you add me as an admin to the digital ocean please

joequant commented 2 months ago

Hey @joequant can you add me as an admin to the digital ocean please

Done. Just upgraded you to admin and you should get an confirm email to click

Jarryd-pretorius commented 2 months ago

Hey @georgeciubotaru Can you please look at this ticket first. I have added you to digital ocean let me know if you need any help or access

georgeciubotaru commented 2 months ago

Log: Based on the instructions from the link we should manually alter the current tables, and set up Primary key creations for the new ones.

Based on the query we have around 1060 tables to define primary keys

georgeciubotaru commented 2 months ago

@Jarryd-pretorius I'm trying to find where is the starting point for these tables, can you help me to accelerate that?

https://github.com/truflation/truflation/tree/6f4a7b995cae3b8c6dc8240a21c137550c0d382c/src/truflation

cc @joequant, @ambahk

Jarryd-pretorius commented 2 months ago

@joequant can you please give @georgeciubotaru the info he needs to solve this problem regarding the db

joequant commented 2 months ago

@Jarryd-pretorius I'm trying to find where is the starting point for these tables, can you help me to accelerate that?

https://github.com/truflation/truflation/tree/6f4a7b995cae3b8c6dc8240a21c137550c0d382c/src/truflation

cc @joequant, @ambahk

Can you telegram me on joequant. The easiest thing that I can I do is to give you access to the test database. @georgeciubotaru

joequant commented 2 months ago

@Jarryd-pretorius I'm trying to find where is the starting point for these tables, can you help me to accelerate that? https://github.com/truflation/truflation/tree/6f4a7b995cae3b8c6dc8240a21c137550c0d382c/src/truflation cc @joequant, @ambahk

Can you telegram me on joequant. The easiest thing that I can I do is to give you access to the test database. @georgeciubotaru

actually I'm at joequanthk

georgeciubotaru commented 2 months ago

Log: clarified during telegram chat.

Going to split this task into 2 sub-tasks:

Context: Reference source

@Jarryd-pretorius FYI

Jarryd-pretorius commented 2 months ago

Awesome thanks @georgeciubotaru for the update

Jarryd-pretorius commented 2 months ago

Hey @georgeciubotaru how is the progress on this?

georgeciubotaru commented 2 months ago

Hey @Jarryd-pretorius, I am currently trying to test the changes

markholdex commented 2 months ago

@georgeciubotaru any ETA?

georgeciubotaru commented 2 months ago

@markholdex Still haven't a successful result for testing, but the ETA is EOD.

georgeciubotaru commented 2 months ago

Log: Requested review for the PR: https://github.com/truflation/truflation/pull/49

if no major changes requested the ETA is EOD

markholdex commented 2 months ago

@Jarryd-pretorius can you please help follow up Joseph here? I know he might be afraid of something breaking on production but with a bit of attention to the review, it can be mitigated. Let me know if anything changes about this Goal in favor of:

p.s. I know George is working on it while waiting review but still asking.

Jarryd-pretorius commented 2 months ago

Yeah I feel the same way I pinged @joequant yesterday about getting this reviewed I will follow this up again right now on TG @markholdex

markholdex commented 1 month ago

log: moving back into Design Ready to not confuse everybody until the Goal is not blocked anymore.

Jarryd-pretorius commented 1 month ago

@georgeciubotaru where are we at with this can we merge this in now?

georgeciubotaru commented 1 month ago

@Jarryd-pretorius This task is on hold as wasn't fully tested on how it will interact with the production data.

We definitely need to resolve our process issues and we can incorporate that fix

Jarryd-pretorius commented 1 month ago

We definitely need to resolve our process issues and we can incorporate that fix

Ok right what needs to be done process wise to handle this?

markholdex commented 3 weeks ago

log: discussed with George. He will pick up this Problem himself. Needs to be super careful with it as it may impact production negatively.

zolotokrylin commented 2 weeks ago

@georgeciubotaru please fix this one after you finish with: