MeltanoLabs / Meta

The why, what, and how of MeltanoLabs
MIT License
5 stars 1 forks source link

Meltano supported tap-postgres #35

Open pnadolny13 opened 1 year ago

pnadolny13 commented 1 year ago

Development Scope:

Out of Scope Features Mentioned:

pnadolny13 commented 1 year ago

@tayloramurphy I created this to define the scope of development for these new plugins, track status, and decide when its ready to recommend. Can you help fill in the description with the features (specific to this source) we expect the initial release to support? It will help us and users understand what done is and when we're there.

cc @visch @aaronsteers

visch commented 1 year ago

This one needs some love that's for sure

These 4 are pretty straight forward and <a few hours of work.

  1. tests pass in CI - https://github.com/MeltanoLabs/tap-postgres/pull/26 Quinn has this one pretty much done
  2. branch protection - https://github.com/MeltanoLabs/tap-postgres/issues/27
  3. cookiecutter TODOs completed and removed - https://github.com/MeltanoLabs/tap-postgres/issues/28
  4. clean README- https://github.com/MeltanoLabs/tap-postgres/issues/29
  5. CD together, with deploys to PyPi - https://github.com/MeltanoLabs/tap-postgres/issues/31

Feature coverage missing things. I can't say what should/shouldn't be there before we make the default switch but here's generally the "Important" missing features. The biggest thing transferwise has over this tap is that they have had their tap around longer so it's just seen more stuff in the wild

  1. Log based pulls - https://github.com/MeltanoLabs/tap-postgres/issues/3
  2. SSL / Username / Host broken out configurations - https://github.com/MeltanoLabs/tap-postgres/issues/30
aaronsteers commented 1 year ago

@visch - Thanks for logging these. We have an emerging standard practice for CD deployments to PyPi, so I think it's safe to leave that out or do manually a few times for now.

I do agree this one is important since it may front-load some breaking changes:

This can come along a little later when higher priorities are all addressed:

Agreed SSL is a nice-to-have and important but perhaps not yet "urgent". After we build into tap-postgres (and/or target-postgres), the long-term plan is to work this into the SDK as well:

pnadolny13 commented 1 year ago

@aaronsteers @visch this is great! I updated the issue checklist with what I think is the consensus in/out of scope items. Definitely feel free to change them if more stuff comes up thats blocking us recommending this variant.