search

MeltanoLabs / tap-gitlab

Singer.io Tap for extracting data from Gitlab's API
GNU Affero General Public License v3.0
8 stars 25 forks source link

SDK Port - new branch or new repo? #66

Open aaronsteers opened 2 years ago

aaronsteers commented 2 years ago

The easiest way to support the new port is probably just to have a new branch name like v2-main be the default branch for the Port - at least until we've gotten near-100% compatibility (if that's possible) and/or implemented meltano lock to help users freeze their pip references at something other than the repo's default branch ref.

I'm also open to the option of just creating a different repo like tap-gitlab-sdk. The advantage of that approach would be to still have a sensible default branch on both repos, and avoid related confusion. The disadvantage of this approach is that it'll be harder to re-merge these repos if/when we finally deprecate the legacy version of the tap.

kgpayne commented 1 year ago

@aaronsteers what is the latest on this? I see the SDK port is on main and the default branch is set to legacy-stable. Are we tracking parity somewhere, with a view to switching over?

aaronsteers commented 1 year ago

@kgpayne - We're unfortunately in a situation where we can never switch over without breaking existing Meltano users. Meltano users who are pinned to the repo directly (and not a released version or branch tag) would be bumped over to the "v2" if we ever change the default branch.

kgpayne commented 1 year ago

@aaronsteers In that case, I'd be in favour of i) adding a deprecation warning to the legacy-stable implementation (so we don't have to maintain it forever), and ii) moving the SDK variant into a new repo as suggested. We can continue to deploy the new one as tap-gitlab in PyPi from there, so shouldn't impact users. If there happen to be users pinning the main branch of this repo, we can leave the code as-is with another deprecation warning pointing at the new PyPi location. That way those users won't break, but also won't update beyond the added deprecation warning 🙂

aaronsteers commented 1 year ago

@kgpayne - It's not simple to do this. It's also not impossible.

New repo name could be tap-gitlab-v2. I don't love adding elaborate suffixes/prefixes like -sdk or -msdk or meltanolabs-, but no solution is perfect.

The new repo would need a new registry on the Hub. For the new plugin name and variant name on the hub, we could go with tap-gitlab as name and perhaps meltanolabs-2 as the variant name. This breaks existing precedent where variant is expected to be equivalent with maintainer - but we can mitigate that pretty easily for now by just treating meltanolabs-2 as it's own maintainer on the Hub. Not elegant, but shouldn't cause any major issues. Also, the maintaner==variant expectation is not known by Meltano Core - only the Hub has that equivalence expectation as of now in how we render maintainers.

Anyway, I'd like variant to evolve from simply be equivalent to {maintainer} to be something like {maintainer}[-{optional-subvariant}], and we'd basically just be acting as though that meaning was already accepted, until we internally build support for it in the Hub.