"In Development" Flag

[pnadolny13]

Theres already this issue to document quality expectations of connectors being migrated to MeltanoLabs https://github.com/MeltanoLabs/Meta/issues/1 but we also need to document a process for connectors that were developed in MeltanoLabs (right now theres a lot of these because of tap-toberfest).

There should be a way to tell users that new tap/target is not production ready yet even though its hosted in MeltanoLabs. From an outside perspective users probably assume that connectors in this repo have been given our stamp of approval but many of the new connectors developed in MeltanoLabs are still experiemental and under development which could be misleading.

There needs to be a way for us to flag that a new connector is beta/experimental/in development/etc. and that users should help raise bugs to get it to a production state.

Right now theres a few connectors that use the convention of adding a note to the description but we should have a consistent way to do this and document it somewhere so users are aware of it.

https://github.com/MeltanoLabs/target-yaml https://github.com/MeltanoLabs/tap-google-analytics

Action Items:

• set all existing connectors to experimental
• create a blurb in https://github.com/MeltanoLabs/Meta describing what experimental means
• complete https://github.com/MeltanoLabs/Meta/issues/1 to define what it takes for a connector to be out experimental state

[pnadolny13]

@tayloramurphy @aaronsteers What do you think? Do you have an opinion about what label we should give these - beta/experimental/under development. I'd vote for beta. Or maybe even a finer grain? like:

In Development: Just created, still developing, single developer that might be pushing directly to main/master, low chance that it works off the shelf. Beta: Functioning, PRs for changes, probably buggy but works. None aka GA: runs in prod, hardened

[aaronsteers]

Not necessarily to put a vote in for the names yet, but I do I think "Beta" and "Under Development" should be different grains.

• Unknown
• Under Development
• Beta
• Stable

And perhaps separately or as requirements for "Stable":

• CI Tests
• PyPi / published status
• ...

[pnadolny13]

@aaronsteers I agree with that. What were you thinking for the Unknown bucket? I'm not sure we need that since everything in MeltanoLabs should have gone through some sort of review as it was added.

I'm good with the naming - Under Development/Beta/Stable

[aaronsteers]

@pnadolny13 - I think the benefit of Unknown bucket is just that it makes a built-in burndown list. I think functionally, it should probably carry the same warnings as "Under Development" and could be phased out once everything has been reviewed and/or after a certain amount of time has passed.

[tayloramurphy]

@pnadolny13 I think we can make custom badges on shields.io. I don't think they would show up in the list, but we could have them as part of the readme.

[pnadolny13]

@pnadolny13 - I think the benefit of Unknown bucket is just that it makes a built-in burndown list. I think functionally, it should probably carry the same warnings as "Under Development" and could be phased out once everything has been reviewed and/or after a certain amount of time has passed.

@aaronsteers ah ok that makes sense. So we can set everything to Unknown to start then work up the hierarchy from there to make sure theyre all appropriately labeled.

[pnadolny13]

@pnadolny13 I think we can make custom badges on shields.io. I don't think they would show up in the list, but we could have them as part of the readme.

@tayloramurphy yeah good idea. We can add those to the top of the READMEs. Looks like we can have something like this:

[aaronsteers]

@pnadolny13 - LOVE it! ❤️

[pnadolny13]

These are the statuses as I see it right now:

tap-github - Beta tap-dbt - Beta tap-stackexchange (default) - Beta tap-athena (default) - UnderDevelopment tap-google-analytics - Beta tap-gitlab (default) - Beta target-athena (default) - Beta tap-slack - Unknown target-sqlite (default) - Beta target-yaml (default) - Beta tap-csv (default) - Beta tap-peloton (default) - Beta target-csv - Beta tap-sqlalchemy - Unknown

Let me know if you disagree with any of those.

[tayloramurphy]

@pnadolny13 I think tap-dbt is pretty Stable. @edgarrmondragon thoughts?

tap-peloton is probably beta. it uses the SDK and relies on an working library underneath.

Also, I'd recommend adding spaces in between the words if we could. so Development Status: In Progress or something like that.

[edgarrmondragon]

@pnadolny13 @tayloramurphy

I think tap-dbt is pretty Stable. @edgarrmondragon thoughts?

I think it's stable though it may be accumulating bugs 😬. I recall a user (Stephen Bailey, maybe?) saying the other day they identified a type mismatch in some streams.

[pnadolny13]

@pnadolny13 I think tap-dbt is pretty Stable. @edgarrmondragon thoughts?

tap-peloton is probably beta. it uses the SDK and relies on an working library underneath.

Also, I'd recommend adding spaces in between the words if we could. so Development Status: In Progress or something like that.

@tayloramurphy sounds good I updated my previous comments to include spaces and your suggestions for tap-dbt and tap-peloton.

[edgarrmondragon]

@pnadolny13 @tayloramurphy would it be helpful to have those statuses as metadata in the Hub? It could still be made into a badge through a JSON endpoint in the Hub and that way they'd be in-sync.

[pnadolny13]

@edgarrmondragon interesting idea! Its sort of starting to remind me of https://gitlab.com/meltano/meltano/-/issues/2829 again. We have also have maintenance status (Unknown, Active, Unresponsive) which is kind of similar but still different.

@tayloramurphy I'm also hesitant to give anything in MeltanoLabs a "Stable" badge until we finish this issue and document what that means.

[edgarrmondragon]

@pnadolny13 either that or if we want to allow the dev keep all that metadata in the repo, I think somewhere we talked about having a file like meltano.hub.yaml that gets scraped along with the other repo data.

[aaronsteers]

@pnadolny13 - Regarding your comment:

We have also have maintenance status (Unknown, Active, Unresponsive) which is kind of similar but still different.

I think perhaps we should consider merging "Maintenance Status" and "Development Status":

What if we expand "Maintenance Status" to something like this:

cc @tayloramurphy, @edgarrmondragon

[tayloramurphy]

@aaronsteers I like it 👍

[pnadolny13]

@edgarrmondragon I hadnt heard about the meltano.hub.yaml, do you have an issue you can link me to? I also like the idea of having a hub endpoint for these repos to reference so the status is only stored in one place. Although we'd need to implement that new endpoint in the hub first.

@aaronsteers I like that break down! I can create an issue to update the hub definitions with those statuses and render them as badges instead of text ("Maintenance Status: Active") on the front end.

Unknown - Active - Unresponsive -

[edgarrmondragon]

@pnadolny13

@edgarrmondragon I hadnt heard about the meltano.hub.yaml, do you have an issue you can link me to? I also like the idea of having a hub endpoint for these repos to reference so the status is only stored in one place. Although we'd need to implement that new endpoint in the hub first.

I couldn't the find exact mention of such file but there's a .singer with similar intentions discussed in https://gitlab.com/meltano/sdk/-/issues/255.

[aaronsteers]

@pnadolny13 - re:

I can create an issue to update the hub definitions with those statuses and render them as badges instead of text ("Maintenance Status: Active") on the front end.

I like this path forward. I think it's worth looking at other text renderings and evaluating them as shields also. Generally speaking, I think it's a nice touch to have consistent bade-like rendering on fields/metrics where it makes sense. (Probably it's possible to go "too far" with badges but I don't fee like we're there yet. 😅

[pnadolny13]

@aaronsteers I created an issue for these two issue for maintenance status and other general badges. https://gitlab.com/meltano/hub/-/issues/162 https://gitlab.com/meltano/hub/-/issues/163

[aaronsteers]

Nice! Thanks, @pnadolny13 !