instructure / jsoncdc

Takes a logical decoding stream from postgresql and outputs it in a pluggable and safe way
51 stars 4 forks source link

Get extra commit info #2

Closed addisonj closed 6 years ago

addisonj commented 6 years ago

This merge brings us up to latest master of jsoncdc.

This already includes the changes we have for 9.6 support with messages

Additionally, this adds one extra commit which adds some fields we want for messages, which includes the epoch to allow us to handle xid rollover more sanely, as well as xmin and catalog_xmin which gives us a better idea of progress in the stream.

This allows us to use the xmin to try and find any missing rows in the event of a db failover.

CLAassistant commented 6 years ago

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


Addison Higham seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

posix4e commented 6 years ago

This is great work y'all. Should we merge any of this stuff into posix4e/jsoncdc? Should I mark my repo as deprecated?

Mythra commented 6 years ago

Hey @posix4e,

I think the only difference right now is slightly up to date build config, this change in flight, and our private FPM Debian build (hopefully not private for too much longer).

That being said I have been planning to pull request our changes in for awhile now. I'm happy to pull request them to your repo once this PR is merged, or if you'd like to deprecate that would work too. Whichever you'd like more I guess!

posix4e commented 6 years ago

You have a much cooler name. I’d love to move it to your repo. Is Rpgffi have everything yOu need?

On Tue, Feb 13, 2018 at 6:12 PM Eric notifications@github.com wrote:

Hey @posix4e https://github.com/posix4e,

I think the only difference right now is slightly up to date build config, this change in flight, and our private FPM Debian build (hopefully not private for too much longer).

That being said I have been planning to pull request our changes in for awhile now. I'm happy to pull request them to your repo once this PR is merged, or if you'd like to deprecate that would work too. Whichever you'd like more I guess!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/instructure/jsoncdc/pull/2#issuecomment-365472956, or mute the thread https://github.com/notifications/unsubscribe-auth/AAxN21pdM7w-6TbQ6ToXjZLgahqSrWm0ks5tUkEXgaJpZM4R7wsK .

Mythra commented 6 years ago

Currently RPG FFI has everything we need, but we are just starting down the road of logical replication. So there may be something unforseen.

posix4e commented 6 years ago

We have your back! On Tue, Feb 13, 2018 at 7:17 PM Eric notifications@github.com wrote:

Currently RPG FFI has everything we need, but we are just starting down the road of logical replication. So there may be something unforseen.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/instructure/jsoncdc/pull/2#issuecomment-365483336, or mute the thread https://github.com/notifications/unsubscribe-auth/AAxN2z0iF-H80F0zFxgdpMWa8uyl_Z5vks5tUlA6gaJpZM4R7wsK .

addisonj commented 6 years ago

@SecurityInsanity K, add those changes.

Also, i ended up running the checkstyle, which seems to have reformated some code. Lemme know if you don't want those changes.... but otherwise, seems like we might as they were part of existing automation

Mythra commented 6 years ago

Looks like checkstyle is broken again @addisonj, after running the tests in docker I got some changed files:

ecoan@kappa:~/jsoncdc$ git status
On branch addisonj-v10-commit
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

    modified:   build.rs
    modified:   src/lib.rs

Untracked files:
  (use "git add <file>..." to include in what will be committed)

    build.rs.bk
    src/lib.rs.bk

no changes added to commit (use "git add" and/or "git commit -a")

However tests are working now so if we get checkstyle fixed we can go ahead, and merge this in.

Mythra commented 6 years ago

Files look good now, and since you work here I won't worry about the CLA :wink: . I'll go ahead and work on getting a new version pushed out for us, and filing a PR upstream.