New Source: LinkedIn Pages

nwayne94 commented 2 years ago

Tell us about the new connector you’d like to have

Which source and which destination? LinkedIn Pages --> S3 and/or Redshift

Describe the context around this new connector

Why do you need this integration? How does your team intend to use the data? The current LinkedIn Ads connection appears to give us most of what we need, but does not appear to output a data stream for Page metrics. Our marketing team would like to utilize this data in conjunction with the rest of the data we pull into our data lake through Airbyte connections to gain a full picture of insights, and so that we can present high-level dashboards to our and our clients' executive teams.
How often do you want to run syncs? ~ once every 24 hours
If this is an API source connector, which entities/endpoints do you need supported? https://developer.linkedin.com/product-catalog/marketing/page-management https://docs.microsoft.com/en-us/linkedin/marketing/integrations/community-management/organizations

Describe the alternative you are considering or using

Our marketing team will continue to utilize the tools provided by LinkedIn, with no real ability to join it with the rest of the data sources we have connected to our data lake through Airbyte.

Are you willing to submit a PR?

Yes, once the new year starts and we wrap up some of our current projects

jscottpolevault commented 2 years ago

My team is currently working on building a LinkedIn Pages Source Connector, but since this will be our first attempt at building a connector, I don't have an ETA. But we really want this to be done this month. I'll try to assign this issue to myself and keep everyone updated. Let me know if someone else is already working on it as well!

jscottpolevault commented 2 years ago

@octavia-squidington-iii - We're almost done with the first version of this LinkedIn Pages source connector, but having a bit of trouble with the authentication piece. Could I request some guidance? I believe the LinkedIn Ads connector has the remaining pieces we need since it uses the same API, but our connector is written in Javascript instead of Python. What are my options for getting some help finishing it up?

harshithmullapudi commented 2 years ago

Hey @jscottpolevault excited to see the connector. Also, my guess is you are working over this using the connector using faros-ai there is a list of connectors they have built here which you can check.

Also if you face any issue you can push the code to some github repo and share over our discourse with the issue so that team can respond over it.

jscottpolevault commented 2 years ago

@harshithmullapudi @octavia-squidington-iii Alright! I have a LinkedIn Company Pages source connector functional in a very basic form now! And I used the python http cdk instead of faros-ai.

Once again, I'm very new at this, so the source.py still has hardcoded endpoints for initial testing, but it's working as expected with a BigQuery destination. I'll work on adding dynamic user inputs into the stream classes next, but after that, I think it will be much quicker to pass this off to airbyte to hopefully tie up any loose ends? If not, please let me know how far I need to try and take it before publishing my branch and sharing however I need to. Thanks!

jscottpolevault commented 2 years ago

Update -- I just submitted a PR for my LinkedIn Pages source connector. It's very basic right now, but works for 7 endpoints/streams. I think the rest of the conversation will occur over there, but just wanted to update this thread that it's ready for review and hopefully will be cleaned up to merge soon @nwayne94 🤞

marcosmarxm commented 1 year ago

Solved by https://github.com/airbytehq/airbyte/pull/13098

airbytehq / airbyte