search

airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
15.88k stars 4.07k forks source link

Unique name for Airbyte resources (sources, destinations, connections) #14448

Closed alafanechere closed 1 month ago

alafanechere commented 2 years ago

Tell us about the problem you're trying to solve

When creating a resource on the Airbyte console the resource default name is the connector name used. If user create a second resource with the same connector they will have two resources with the same name. This is confusing for users to find the exact resource they are looking for. On the octavia-cli side this is causing a problem because we enforce a unique resource name on the octavia project side. This will require users to rename their resource on the console if they want to import it.

Describe the solution you’d like

Additional context

The problem on the octavia-cli side is described here.

octavia-squidington-iii commented 2 years ago

cc @airbytehq/frontend

timroes commented 2 years ago

I'd have some concern of using the name as the unique identifier for a connection. The name (as it is at least used today) is meant to be a human readable representation, and I think trying to enforce uniqueness on a human readable component won't be the best solution. We already have connection IDs that are unique. Shouldn't we use them instead for addressing? If not, is there a specific reason for this?

evantahler commented 2 years ago

+1 for using the (already unique) connection IDs for all API requests, and the storage mechanism for the octavia cli. We talked about this topic a little bit here https://github.com/airbytehq/airbyte/issues/13203 as well

davinchia commented 2 years ago

all octavia issues route to me so I'm doing a drive by and saying +1 for using IDs instead of names.

sebastianreloaded commented 1 year ago

Using a unique name for sources and destinations would make sense to me from an UX perspective. E.g. when creating new connections you select destination and source based on the name. If there are multiple sources with the same name you can't distinguish them except they happen to belong to different connectors.

Maybe make them unique for a combined (actor_definition_id, name, actor_type) .. This might also improve the airflow AirbyteTriggerSyncOperator, which is a hassle if you have mutliple environments.