My earlier commit (dfbe8b5714bf76b053f971d5352e95d26f471d08 in PR #2037 ) was too broad in changing all incremental streams to use >= in the filter query param. The /profiles endpoint used by both the Profiles and GlobalExclusions streams do not support filtering with >=, so they have to use >.
For streams that do use >, I subtracted one second (the smallest time granularity for Klaviyo timestamps) before the first request to avoid missing records that were updated in the same second but weren't available in the API at the exact same time.
Workflow steps:
(How does one use this feature, and how has it changed)
Documentation links affected:
(list any documentation links that you created, or existing ones that you've identified as needing updates, along with a brief description)
Notes for reviewers:
Although /profiles does not support >= filtering & the API returns a 400 response when trying to filter with >=, existing tasks have failed silently. I'm not sure exactly why tasks didn't fail / an error message wasn't logged, but I made issue #2114 to track this.
Tested on a local stack. Confirmed:
Previously working incremental streams still work incrementally.
Profiles and GlobalExclusions streams now work, are incremental, and no longer silently fail.
Description:
My earlier commit (dfbe8b5714bf76b053f971d5352e95d26f471d08 in PR #2037 ) was too broad in changing all incremental streams to use
>=in thefilterquery param. The/profilesendpoint used by both theProfilesandGlobalExclusionsstreams do not support filtering with>=, so they have to use>.For streams that do use
>, I subtracted one second (the smallest time granularity for Klaviyo timestamps) before the first request to avoid missing records that were updated in the same second but weren't available in the API at the exact same time.Workflow steps:
(How does one use this feature, and how has it changed)
Documentation links affected:
(list any documentation links that you created, or existing ones that you've identified as needing updates, along with a brief description)
Notes for reviewers:
Although
/profilesdoes not support>=filtering & the API returns a400response when trying to filter with>=, existing tasks have failed silently. I'm not sure exactly why tasks didn't fail / an error message wasn't logged, but I made issue #2114 to track this.Tested on a local stack. Confirmed:
ProfilesandGlobalExclusionsstreams now work, are incremental, and no longer silently fail.This change is