search

julien-duponchelle / python-mysql-replication

Pure Python Implementation of MySQL replication protocol build on top of PyMYSQL
2.33k stars 679 forks source link

upgrade and improve perfermance #603

Closed YaoDongJi closed 7 months ago

YaoDongJi commented 10 months ago

Version

Please specify the versions you are using. Exact version numbers are preferred.

Symptoms

We are currently using 0.31 as our production version, but we have encountered a performance bottleneck with a processing speed of 60m/s. Is the latest version compatible with version 0.31? Can we improve performance by upgrading the version?

sean-k1 commented 10 months ago

@YaoDongJi If you use Mysql 5.7version Use 0.45.1 version

YaoDongJi commented 10 months ago

您好，我已收到邮件 。                                                     收件人：姚东济

julien-duponchelle commented 10 months ago

You can try using PyPy instead of the standard python interpreter this is normally faster.

Le lun. 29 janv. 2024 à 06:22, YaoDongJi @.***> a écrit :

您好，我已收到邮件 。 收件人：姚东济

— Reply to this email directly, view it on GitHub https://github.com/julien-duponchelle/python-mysql-replication/issues/603#issuecomment-1913983745, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACUKXIXEDNFDJL3ZQHRROLYQ4W2HAVCNFSM6AAAAABCNECBVCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMJTHE4DGNZUGU . You are receiving this because you are subscribed to this thread.Message ID: <julien-duponchelle/python-mysql-replication/issues/603/1913983745@ github.com>

YaoDongJi commented 10 months ago

@YaoDongJi If you use Mysql 5.7version Use 0.45.1 version

thanks, we will try this version

YaoDongJi commented 10 months ago

You can try using PyPy instead of the standard python interpreter this is normally faster. Le lun. 29 janv. 2024 à 06:22, YaoDongJi @.***> a écrit : … 您好，我已收到邮件 。 收件人：姚东济 — Reply to this email directly, view it on GitHub <#603 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACUKXIXEDNFDJL3ZQHRROLYQ4W2HAVCNFSM6AAAAABCNECBVCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMJTHE4DGNZUGU . You are receiving this because you are subscribed to this thread.Message ID: <julien-duponchelle/python-mysql-replication/issues/603/1913983745@ github.com>

We are currently using pypy3.7. Currently, we have observed a maximum of 10000 tps per second, corresponding to approximately 10MB of data. We would like to ask for some observation indicators that can be used as a reference to help us confirm whether there is still room for performance improvement.

sean-k1 commented 10 months ago

@YaoDongJi we don't maintain this for mysql5.7 anymore, but I can put this feature in 0.45.2 if you want ?

YaoDongJi commented 10 months ago

n 0.45.2

Yes, performance improvement is really needed. Thank you very much for your help. We will test the performance of this version after merging this feature.

YaoDongJi commented 10 months ago

Could you please help us merge this feature into version 0.30? I checked and found that we are using version 0.30, and I compared the performance of 0.45.1 and version 0.30, and found that 0.30 will perform better than 0.45.1, thanks.

sean-k1 commented 9 months ago

@YaoDongJi Need 0.30.1 or 0.31.1?

YaoDongJi commented 9 months ago

@YaoDongJi Need 0.30.1 or 0.31.1?

0.30.1 would be better.

sean-k1 commented 9 months ago

@YaoDongJi released 0.30.1 Let me know if you have any problems

YaoDongJi commented 9 months ago

@YaoDongJi released 0.30.1 Let me know if you have any problems

We will test this soon,thanks