search

the4thdoctor / pg_chameleon

MySQL to PostgreSQL replica system
http://www.pgchameleon.org
BSD 2-Clause "Simplified" License
377 stars 82 forks source link

Log error misleading #88

Open akea013 opened 5 years ago

akea013 commented 5 years ago

The logs are constantly telling me the source is consistent, the show_status command that they are inconsistent:

In pg_lib.py, check_source_consistent method currently reads: (pseudo code): if source_consistent: if source_consistent[0]: (update t_replica_tables ...) log(consistent) else: log(inconsistent) else: log(consistent)

Line 3306 of pg_lib.py reads:

------->------->-------self.logger.debug("The source: %s is consistent" %(self.source, ) ) I think it should read: ------->------->-------self.logger.debug("The source: %s is not consistent" %(self.source, ) )

the4thdoctor commented 5 years ago

hi, the esternal if checks whether the source have a consistent record which describes the consistent status.

I agree the message is wrong. can you please run the following SQL on your database to check what's returned?

https://pastebin.com/kVEtKsdT

You'll need to replace the %s with your source id. You'll find it in the show_status output.

Thanks

akea013 commented 5 years ago

LOG: duration: 2.572 ms b_consistent

f (1 row)

and when I run chameleon show_status .... I get Source id Source name Type Status Consistent Read lag Last read Replay lag Last replay

      2  mysql                       mysql   running   No

N/A N/A

== Schema mappings == Origin schema Destination schema

projectsdb projectdb

== Replica status ==

Tables not replicated 0 Tables replicated 51 All tables 51 Last maintenance 2018-10-10 02:21:35 Next maintenance N/A Replayed rows Replayed DDL Skipped rows

The replica daemon has been running for three days - the source is mostly static. Changes I make to the source are reflected quickly on the target, but still I get this report of an inconsistent state. (I have been trying to get a consistent state report for a couple of weeks

Thanks for any help -- Anita

On Wed, Oct 10, 2018 at 9:20 PM, Federico Campoli notifications@github.com wrote:

hi, the esternal if checks whether the source have a consistent record which describes the consistent status.

I agree the message is wrong. can you please run the following SQL on your database to check what's returned?

https://pastebin.com/kVEtKsdT

You'll need to replace the %s with your source id. You'll find it in the show_status output.

Thanks

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/the4thdoctor/pg_chameleon/issues/88#issuecomment-428481565, or mute the thread https://github.com/notifications/unsubscribe-auth/AhBvcR61gOgbcdpDnfiG8HiRPqSS29vjks5uja27gaJpZM4XUXc0 .

the4thdoctor commented 5 years ago

looks like no event has been received. assuming you are not using the limit_tables option in the source, can you create and drop a dummy table? this will produce a couple of events sufficient to trigger the consistent status. I need to think a different way to determine the consistent status though.

the4thdoctor commented 5 years ago

never mind the previous comment. I'm still sleeping. I need to reproduce this behaviour. can you please tell me about your environment? what I need is mysql version pgsql version operating system for mysql, postgres, the replica machine (assuming is a third one) python version and pgchameleon's version

thanks

akea013 commented 5 years ago

mysql: OS Centos 7

Linux version 3.10.0-693.2.2.el7.x86_64 (builder@kbuilder.dev.centos.org) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-16) (GCC) ) #1 SMP Tue Sep 12 22:26:13 UTC 2017

mysql version

Server version: 10.2.17-MariaDB-log MariaDB Server Name : MariaDB-server Arch : x86_64 Version : 10.2.18 Release : 1.el7.centos Repo : mariadb License : GPLv2 Arch : x86_64 Epoch : 1 Version : 5.5.60 Release : 1.el7_5 Size : 11 M Repo : updates/7/x86_64

postgres:

OS: Another Centos 7 Linux version 3.10.0-693.2.2.el7.x86_64 (builder@kbuilder.dev.centos.org) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-16) (GCC) ) #1 SMP

postgresql package: Name : postgresql96-server Arch : x86_64 Version : 9.6.10 Release : 1PGDG.rhel7 Installed Packages Name : postgresql96-server Arch : x86_64 Version : 9.6.10 Release : 1PGDG.rhel7 Size : 19 M Repo : installed From repo : pgdg96

Replica machine is the postgres machine.

Python version: Python 3.4.9 [GCC 4.8.5 20150623 (Red Hat 4.8.5-28)] on linux

pg_chameleon version: pg_chameleon-2.0.10

Latest show_status output: Source id Source name Type Status Consistent Read lag Last read Replay lag Last replay

      2  mysql          mysql   running   No            01:01:07

2018-10-11 22:22:59 00:00:00 2018-10-11 22:22:59

== Schema mappings == Origin schema Destination schema

projectsdb projectdb

== Replica status ==

Tables not replicated 0 Tables replicated 51 All tables 51 Last maintenance 2018-10-10 02:21:35 Next maintenance N/A Replayed rows 32 Replayed DDL 0 Skipped rows 0

Thanks for any help. -- Anita

On Thu, Oct 11, 2018 at 6:26 PM, Federico Campoli notifications@github.com wrote:

never mind the previous comment. I'm still sleeping. I need to reproduce this behaviour. can you please tell me about your environment? what I need is mysql version pgsql version operating system for mysql, postgres, the replica machine (assuming is a third one) python version and pgchameleon's version

thanks

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/the4thdoctor/pg_chameleon/issues/88#issuecomment-428825101, or mute the thread https://github.com/notifications/unsubscribe-auth/AhBvcbPC9CTb8IQxMqy9C5rn6hyviszdks5ujtZrgaJpZM4XUXc0 .

the4thdoctor commented 5 years ago

thanks for the info. I'm still moving home so probably on 2/3 weeks I'll be able to look at this and the issue #87 .