airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
16.14k stars 4.13k forks source link

New Source: Pendo #2977

Closed sherifnada closed 1 year ago

sherifnada commented 3 years ago

Tell us about the new integration you’d like to have

https://developers.pendo.io/docs/

Describe the context around this new integration

Describe the alternative you are considering or using

┆Issue is synchronized with this Asana task by Unito

cheyura commented 3 years ago

Webhook-based? (no/partially/yes) Partially https://support.pendo.io/hc/en-us/articles/360032285012-Webhooks

Available authentication modes (API key/Oauth/other)

Pendo provides the following APIs and integration kits for customers:

Authentication of requests is handled by an integration key

To create an Integration Key:

Creating an account Free up to 1000 users monthly

Sandbox account? No

How to populate the account with data? App is needed and then UI interaction

Available streams for sync Browser Interactions Functions Debugging Events Track Events Guide Events Options

Integration supports incremental sync?

Other information/blockers Most obviously, an app is needed. Once installed in your application, visitor activity is collected automatically by the Pendo Agent and sent via Image Beacons to Pendo for data processing. Pendo accepts these events immediately but ingests and processes visitor data asynchronously. So visitor events sent by the Agent are not immediately available via the API but usually within 5 minutes of receipt.

dannob commented 3 years ago

Just adding a +1 to this request.

sherifnada commented 3 years ago

@dannob which data would you like to pull from pendo? this helps us know which endpoints to support

lionelmoi commented 3 years ago

I would like to be able to pull anything under the aggregation category https://developers.pendo.io/docs/?bash#aggregation

Being able to define an aggregation and pull data out of Pendo this way would be a huge time saver.

tym-xqo commented 3 years ago

We are currently using Fivetran's Pendo connector, which mostly pulls event-level data and is described at https://fivetran.com/docs/applications/pendo. A similar connector in Airbyte would make it much easier for us to consider switching.

pmamarques commented 3 years ago

We are currently using Fivetran's Pendo connector, which mostly pulls event-level data and is described at https://fivetran.com/docs/applications/pendo. A similar connector in Airbyte would make it much easier for us to consider switching.

Same here.

cheyura commented 3 years ago

No Sandbox available and real Application (where Pendo Agent is installed) is needed to retrieve data through API

lionelmoi commented 3 years ago

https://www.pendo.io/pendo-free/ may be worth a look.

lionelmoi commented 3 years ago

Hey @cheyura how can we unblock this connector?

cheyura commented 3 years ago

Hey @cheyura how can we unblock this connector?

@lionelmoi @sherifnada Since Pendo has no sandbox we are unable to test this connector (we need a real app where we install Pendo). All the connectors that need a real application are currently blocked

sherifnada commented 3 years ago

@lionelmoi would you be open to sharing your pendo credentials with us to help us build the connector?

lionelmoi commented 3 years ago

I'll come back to you on this.

adayNU commented 3 years ago

I just opened a request with Pendo for a test API key

kerbam commented 2 years ago

Since the sources for Airbyte appear to be able to use Singer taps maybe their Pendo tap might be helpful along the way to getting this built out on the Airbyte side: https://github.com/singer-io/tap-pendo

SteadyGiant commented 2 years ago

Any update on this?

YowanR commented 1 year ago

I believe we're still blocked on this one. Waiting for Pendo to reply. @igrankova Can you confirm?

igrankova commented 1 year ago

@YowanR yes, we don't have any answer from Pendo.

fortynachos commented 1 year ago

Hey folks, Pendo employee here 👋. I lead our Engineering Services team. I can likely get you hooked up with an integration key, stay tuned while I check some things on my end.

fortynachos commented 1 year ago

For folks that are interested in working on this, can y'all email mikefo@pendo.io? I can invite you to a test Pendo instance and provide a read-only integration key directly via email.

YowanR commented 1 year ago

Thanks @fortynachos! @igrankova can you follow up, please? @YiyangLi is this something you'd like to work on?

igrankova commented 1 year ago

@YowanR We accepted the invitation to the sandbox, LastPass: Pendo Sandbox Account API Key saved in the LastPass The account is already seeded! Thanks a lot for help, @fortynachos

YowanR commented 1 year ago

@YiyangLi Any updates on this one? 🙏

YiyangLi commented 1 year ago

@YowanR Yes, I just created a PR https://github.com/airbytehq/airbyte/pull/24081 I run the integration tests locally, it works fine. I only added 4 streams.

I used the same credential you shared with me, please setup the CI so that the integration tests are run successfully. Thanks.

marcosmarxm commented 1 year ago

Closed by #24081