singer-io / tap-linkedin-ads

GNU Affero General Public License v3.0
10 stars 22 forks source link

Upgrade creatives api endpoint #57

Closed sgandhi1311 closed 1 year ago

sgandhi1311 commented 1 year ago

Description of change

The endpoint v2/adCreativesV2 is sunsetting and getting replaced with rest/creatives. Ref - Marketing API Versioning This PR adds the correct endpoint for the creatives stream. Ref - Creatives API

Few things to keep in mind -

  1. It requires the query parameter - campaigns to be passed in encoded format.
  2. Specific headers are required -
    {"X-Restli-Protocol-Version": "2.0.0",
    "X-RestLi-Method": "FINDER"}

    Jira - TDL-21780

Manual QA steps

Risks

Rollback steps

sgandhi1311 commented 1 year ago

If we're changing the key that is used to bookmark a stream, we either need to handle that ourselves or make it a major version change (even if the third party service provider has imposed this change) as this could cause a full re-replication for the stream in question.

We probably need to overwrite child_obj.get_bookmark() for this stream to ensure that we can return either the old key or the new key, but write_bookmark() needs to only write the new key. See https://jira.talendforge.org/browse/TDL-21676 for where we discovered a bug related to this for Stripe.

If this isn't actually an issue right now based off what you've observed, we should consider writing a unittest that proves it.

This time we are doing the major bump version. The reason - format for the id field (primary key for creatives stream) has changed from integer to string (the complete urn). Ref - Migration details. So I think even if we are using the updated bookmark key (last_modified_time -> last_modified_at) that shouldn't be the issue.