airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
15.4k stars 3.97k forks source link

Show users IP addresses during connector setup #17680

Closed timroes closed 1 year ago

timroes commented 1 year ago

We'd like to show users the IP address our cloud connects from during the setup of the connector so they can allow list them already in their tool before testing the connection.

The list of IP addresses should also be linked up to LaunchDarkly, so we can quicker adjust it in case they change, before we adjust it in code.

octavia-squidington-iv commented 1 year ago

cc @airbytehq/frontend

timroes commented 1 year ago

@andyjih or @misteryeo Do you know if there is a reasonable way to detect from the connector spec somehow if it makes sense to show that information for that connector? I.e. if that connector needs to communicate to another system of the user, since I'd say it makes no sense showing this on the API connectors.

misteryeo commented 1 year ago

I don't believe so but I think @alafanechere @girarda might be able to confirm?

girarda commented 1 year ago

@timroes, you can use the sourceType, which is specified in the source definition, if all you care about is whether the source is an API or a database

pmossman commented 1 year ago

Refining notes:

timroes commented 1 year ago

@pmossman yes we already have access to the sourceType via the api/v1/source_definitions/list and api/v1/source_definitions/list_latest APIs.

mfsiega-airbyte commented 1 year ago

Refining notes:

nataliekwong commented 1 year ago

As an additional data point for how it would help the Support team, today we get about 17 inbound chats a month (5% of the intercom chats) asking about the IP whitelist requirements. Data source: https://airbyte.metabaseapp.com/question/2021-intercom-conversations-about-ip-whitelist

josephkmh commented 1 year ago

Thanks Natalie, that is quite a high number and maybe warrants prioritizing this issue. I wonder if the inbound chats are questions about which IPs to whitelist or how to whitelist IPs. If it's the former, then it would help to show the IPs during setup. If it's the latter, we probably need a different solution.

I just did a quick check in Intercom, and only 1 of the first 5 conversations that included an IP (34.106.109.131) was actually a root cause about IP whitelisting. A lot of other conversations were just including the IPs as a potential troubleshooting step, but the user had already whitelisted them.

nataliekwong commented 1 year ago

Thank you, I hadn't looked as closely at the content! If it's really just 1% of the Intercom chats it won't have much impact on deflecting those chats.

The only other data point I can think of it more upstream - the errors from the source/destination checks themselves. We haven't done any reporting on those yet as we just started getting them recorded in Segment calls, so I don't have data on how often a check error shows up related to a lack of IP whitelisting (or which errors from each destination necessarily relate to IP whitelisting).

Can leave this in the nice to have bucket unless we can pull data that says otherwise.

josephkmh commented 1 year ago

@lmossman removing this from the compose backlog as it seems like a more extensibility related issue!

lmossman commented 1 year ago

We are currently running an experiment with the Postgres connector to show a message about allowlisting Airbyte IP addresses. See relevant slack thread here