Open acarter24 opened 1 year ago
@acarter24 thanks for raising this! I imagine the fix could be a pretty quick change similar to what we have here to first check if end_time
is present in the dictionary we're inspecting before attempting to access it. Would you be interested in contributing a PR?
I'll have a go, sure :)
Looks like replication_key
breaks for this stream though? it inherits from UserInsightsStream
which sets replication_key='end_date'
, which results in another KeyError later in the process when serialising json for state.
EDIT: UserInsightsOnlineFollowersStream
is a lifetime
table but doesn't seem to suffer from the same issue, will have to investigate.
Huh okay. It's been long enough since I wrote this that I'm blanking on some implementation details. Without having looked two closely, seems like there are two paths here:
replication_key
user_insights_audience
stream returns:parse_response
fails with a KeyError looking forend_date
in this data, but the period value islifetime
so not sure if thisend_date
key will be present.Stack trace below