airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
15.64k stars 4.02k forks source link

Source Mixpanel: Support Service Accounts #14607

Closed misteryeo closed 2 years ago

misteryeo commented 2 years ago

Project Secrets for Mixpanel are being deprecated in favor of Service Accounts. As part of bringing Mixpanel to GA, we should investigate how to support this new method.

lazebnyi commented 2 years ago

@ycherniaiev @igrankova could you clarify did this issue pass vetting?

igrankova commented 2 years ago

@lazebnyi I don't see any vetting task for Mixpanel

lazebnyi commented 2 years ago

@igrankova If it possible could you check credentials for service account?

igrankova commented 2 years ago

@lazebnyi we have 2 service accounts: image mixpanel

If you need creds, there's an option to create a service account and see the creds. I don't see we have the creds on these two. I've created a new Service Account: image You can see the creds in the Lastpass: mixpanel.com

midavadim commented 2 years ago

@Andy @lazebnyi

https://developer.mixpanel.com/reference/authentication

Deprecating methods are:

but our mixpanel source uses:

The question: should we still add an additional authentication method: Service Accounts ?

YowanR commented 2 years ago

Thanks @midavadim! Do most of our supported streams fall under only the Mixpanel Ingestion API? ? Based on the information in the MLP, it seems that we support quite a few streams from the Query API. This leads me to believe that we will be needing Auth via Service Account going forward. Since we're pushing Mixpanel to GA, I would be inclined to front-loading this work (but ideally in September as we are currently focused on alpha to beta certification). What do you think?

midavadim commented 2 years ago

@YowanR I suppose that in order to implement additional streams "Service Account" authentication is not required.

But if we need additional auth methods I'll implement it.

The only reason why I ask this question is because the original reason described in the issue is incorrect.

YowanR commented 2 years ago

Thanks for explaining. Since Mixpanel prefers that we use Service Account going forward, I think we should still do this work but it's lower priority for now given @midavadim's explanation above. I suggest that we move this back to our backlog and tackle it in September when we take Mixpanel to GA. Let's focus on alpha to beta connectors for this sprint instead. cc @lazebnyi @YuliiaNahoha -- let's make sure this gets re-prioritize after our 3 week sprint, please.

midavadim commented 2 years ago

Update - our current auth method is deprecating indeed.

So started implementation of service account authentication. But new auth method requires update of all streams - because now we should extract list of granted projects and then run current streams for each project

YowanR commented 2 years ago

Are we tackling this in this sprint? @midavadim @lazebnyi