New Source: Bing Ads

[sherifnada]

Tell us about the new integration you’d like to have

I would like to support the Bing Ads API as a source: https://docs.microsoft.com/en-us/advertising/guides/?view=bingads-13

There is a Singer tap here which works for this connector that we could build on top of.

┆Issue is synchronized with this Asana task by Unito

[yevhenii-ldv]

Integration Vetting

Webhook-based? (no/partially/yes)

no

Available authentication modes (API key/Oauth/other)

In this regard, it is rather difficult here. Bing Ads uses Oauth, but it can be configured in such a way that due to the refresh_token to constantly update access token and work with it. This is described in more detail in this article.

However, using Bing Ads as a connector will require a lot of input to work with the API. For example, Singer Tap requires the following data to receive data:

{
  "developer_token": "<developer_token>",
  "customer_id": "<customer_id>",
  "oauth_client_secret": "<oauth_client_secret>",
  "oauth_client_id": "<oauth_client_id>",
  "refresh_token": "<refresh_token>",
  "account_ids": "<account_id_list>",
  "start_date": "2020-11-12T00:00:00Z"
}

Creating an account

self-service free account available? I tried to create an account, but received the message "Account with this email already exists", and I just logged into the account using integration-test@airbyte.io as a login and password from Microsoft (I found it in Lastpass).

How to populate the account with data?

I found that the existing account already has a Campaign to work with. However, I am not sure that it is possible to add Ads and AdGroups for free, perhaps it will require small material costs.

Also, We can use the Sandbox. For CI testing Bing Ads connector with Singer Tap we can use only production.

Available streams for sync

• Campaign
• AdGroup
• Ad
• AdvertiserAccount
• Reports (e. g. KeywordPerformanceReport, AdPerformanceReport, AdGroupPerformanceReport, GeographicPerformanceReport, AgeGenderAudienceReport and other)

Integration supports incremental sync?

Yes

Other information/blockers

It is difficult to initially configure the connector (to get all the config for its correct operation). I have not yet managed to fully configure it and test the correct operation of Singer Tap.

Clients:

• official client
• singer-tap

[yevhenii-ldv]

Task breakdown

• [ ] Populate source with data. (estimation 8h, because filling in the data of the Reports can take a long time).
• [ ] Create source with all streams with full/incremental sync(using Singer Tap) (estimation 6h)
• [ ] Ensure that connector follows the best practices, rate limiting, retry call, proper error messages, has tests (estimation 4h).

[sherifnada]

Like yevhenii mentioned, this integration requires updating the refresh token in Airbyte. So it is blocked from being fully implemented.

We can always just tell the user they need to get a new refresh token and input it into the Airbyte UI/API.

[bruce-a-gitor]

One question we have about your upcoming Bing Ads release is whether it will have the ability to bring in the Product Dimension Performance Report? This is definitely a requirement for us and has been a long time shortcoming of Singer.

This Singer issue is the one I am referring to: https://github.com/singer-io/tap-bing-ads/issues/23

[sherifnada]

@bruce-a-gitor thanks for the feedback! We were planning to build on top of the Singer tap but if it's insufficient for this case we can adjust our goals accordingly. We'll update this ticket with plans as we make them.

[davinchia]

Note: when using the sandbox, I had to create a new account under airbyte-integration-test@outlook-int.com (in Lastpass) per this link. I don't see any populated data. integration-test@airbyte.io was not working for me.

Where did you log into the account from @yevhenii-ldv?

[davinchia]

This is the most complete guide to Bing Authentication I found.

[sherifnada]

Deprioritized for the time bing due to bandwidth contstraints. Will update here as we revisit.

[cheyura]

Sandbox supports MSAs created using an outlook-int.com email account only. You may not use an outlook.com email account. Also, you may not use an email account from another email service (for example, @contoso.com) even if the account is linked to an outlook.com or outlook-int.com email account. That why I ve created Yuri.Cherniaiev@outlook-init.com for testing ad campaign

[sherifnada]

@cheyura can you create MSA for sandbox integration-test@outlook-int.com and add the credentials to lastpass as "Microsoft Ads Integration Test account" ?

You should be able to sign into the existing account using the LP credential titled "Microsoft Integration Test Credentials"

[cheyura]

@cheyura can you create MSA for sandbox integration-test@outlook-int.com and add the credentials to lastpass as "Microsoft Ads Integration Test account" ?

You should be able to sign into the existing account using the LP credential titled "Microsoft Integration Test Credentials"

@sherifnada Done and added

[midavadim]

Core streams (same as stitch data): Accounts Ads Ad Groups

Report Streams (subset of Fivetran’s reports): Ad Performance Report (horly, daily like in Fivetran) Ad Group Performance report (horly, daily like in Fivetran) Campaign performance report (horly, daily like in Fivetran) Keyword performance report (horly, daily like in Fivetran)

Python bing ads client - bingads: https://docs.microsoft.com/en-us/advertising/guides/get-started-python?view=bingads-13 pip install bingads

Python tutorial: https://docs.microsoft.com/en-us/advertising/guides/walkthrough-desktop-application-python?view=bingads-13

SOAP based API https://pypi.org/project/suds-jurko/ Lightweight SOAP client (Jurko's fork).

References: Microsoft Docs: https://docs.microsoft.com/en-us/advertising/guides/?view=bingads-13 Stitch Data docs: https://www.stitchdata.com/docs/integrations/saas/microsoft-advertising Fivetran docs: Report table schema: https://fivetran.com/docs/applications/microsoft-advertising#reporttables ERD for the connector: https://docs.google.com/presentation/d/1xNeRCtU5dNVy01kTjE-ydX_BLoRDgKZOHGci1vUKDFg/edit#slide=id.g311502b468_5_443

[LuliRib]

Hi, I sync the basic streams from Bing Ads to in Airbyte and I get a lot of tables for each stream.

For example, for the ad_groups, I get 12 tables, as showm in the image.

Can´t find anything about it in the documentation

[isisgoliveira]

Hello! I was able to authenticate to the API with the Bing Ads connector and select the desired report/stream. However, this ingestion is bringing cost/spend values ​​that do not exactly match those of the platform, they are just very close. I suspect that it may be a timezone issue.

Anyone who has had this problem or knows how the connector makes the request properly could help me?