julien-duponchelle / python-mysql-replication

Pure Python Implementation of MySQL replication protocol build on top of PyMYSQL
2.31k stars 678 forks source link

upgrade and improve perfermance #603

Closed YaoDongJi closed 4 months ago

YaoDongJi commented 7 months ago

Version

Please specify the versions you are using. Exact version numbers are preferred.

Symptoms

We are currently using 0.31 as our production version, but we have encountered a performance bottleneck with a processing speed of 60m/s. Is the latest version compatible with version 0.31? Can we improve performance by upgrading the version?

sean-k1 commented 7 months ago

@YaoDongJi If you use Mysql 5.7version Use 0.45.1 version

YaoDongJi commented 7 months ago

您好,我已收到邮件 。                                                     收件人:姚东济

julien-duponchelle commented 7 months ago

You can try using PyPy instead of the standard python interpreter this is normally faster.

Le lun. 29 janv. 2024 à 06:22, YaoDongJi @.***> a écrit :

您好,我已收到邮件 。 收件人:姚东济

— Reply to this email directly, view it on GitHub https://github.com/julien-duponchelle/python-mysql-replication/issues/603#issuecomment-1913983745, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACUKXIXEDNFDJL3ZQHRROLYQ4W2HAVCNFSM6AAAAABCNECBVCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMJTHE4DGNZUGU . You are receiving this because you are subscribed to this thread.Message ID: <julien-duponchelle/python-mysql-replication/issues/603/1913983745@ github.com>

YaoDongJi commented 7 months ago

@YaoDongJi If you use Mysql 5.7version Use 0.45.1 version

thanks, we will try this version

YaoDongJi commented 7 months ago

You can try using PyPy instead of the standard python interpreter this is normally faster. Le lun. 29 janv. 2024 à 06:22, YaoDongJi @.***> a écrit : 您好,我已收到邮件 。 收件人:姚东济 — Reply to this email directly, view it on GitHub <#603 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACUKXIXEDNFDJL3ZQHRROLYQ4W2HAVCNFSM6AAAAABCNECBVCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMJTHE4DGNZUGU . You are receiving this because you are subscribed to this thread.Message ID: <julien-duponchelle/python-mysql-replication/issues/603/1913983745@ github.com>

We are currently using pypy3.7. Currently, we have observed a maximum of 10000 tps per second, corresponding to approximately 10MB of data. We would like to ask for some observation indicators that can be used as a reference to help us confirm whether there is still room for performance improvement.

sean-k1 commented 7 months ago

@YaoDongJi we don't maintain this for mysql5.7 anymore, but I can put this feature in 0.45.2 if you want ?

YaoDongJi commented 7 months ago

n 0.45.2

Yes, performance improvement is really needed. Thank you very much for your help. We will test the performance of this version after merging this feature.

YaoDongJi commented 7 months ago

Could you please help us merge this feature into version 0.30? I checked and found that we are using version 0.30, and I compared the performance of 0.45.1 and version 0.30, and found that 0.30 will perform better than 0.45.1, thanks.

sean-k1 commented 7 months ago

@YaoDongJi Need 0.30.1 or 0.31.1?

YaoDongJi commented 7 months ago

@YaoDongJi Need 0.30.1 or 0.31.1?

0.30.1 would be better.

sean-k1 commented 7 months ago

@YaoDongJi released 0.30.1 Let me know if you have any problems

YaoDongJi commented 7 months ago

@YaoDongJi released 0.30.1 Let me know if you have any problems

We will test this soon,thanks