Open macobo opened 2 years ago
Another instance of this: https://posthog.slack.com/archives/C0460J93NBU/p1666785896597509
bug 1: we'll just increment the id on deletion bug 2: we can clear the cache bug 3: @macobo or @yakkomajuri any ideas for how we can do this, given for postgres we have cascading deletes and in CH we have eventual deletes.
Idea for bug3: instead of using version we could use <update-timestamp>-<version>
, but then we could run into races during normal operations (though that's not common due to us sending distinctIDs to the same kafka partition). But maybe there's a way to have something concatenated with the version. An overkill working solution is keeping a separate table in posgress or is_deleted counts per distinct_id and <is_deleted_cnt>-<version>
is what we'd collapse by.
So for now I propose that we add a warning to the "delete person" button saying that if you plan to re-use any of the IDs then split IDs instead (which works as we have a new person and a version incremented), otherwise you'll have a bad time.
Bug 2 is related to https://github.com/PostHog/posthog/issues/10463
Proposal for bug3: We add a column postgres_id
to CH person_distinct_id2
table and use postgres_id
and then version
in CH instead of just version
. In postgres when we update the distinctID - personID mapping we always increment the version & ID stays the same, so that works, if we delete a person and distinctIDs, then upon re-use we create a new entry and get a new bigger postgres ID and we can start the version from 0 again.
bug3: discussed this during pipeline sync on Friday: @macobo proposed soft-deletes on postgres (date of deletion into postgres or null for not deleted) as an alternative.
The benefit of the CH route is that currently bad data would not influence anything, with soft-deletes route we might get bad entries from re-use (of persons deleted before the fix) later.
Async-deletes for person events: We delete by person_id
column and if the distinct_id is re-used we'll create a new person, so the deletes will be fine and delete exactly the right events. Furthermore importantly there's no infinite deletions runs.
Bug description
Slack thread: https://posthog.slack.com/archives/C0374DA782U/p1662025370798299
Summary:
person_distinct_id2
table on clickhouse and delete the row in postgres.person_distinct_id2
and postgres distinct_id row with version=0. This means that new inserted distinct_id might never be visible in that table.Environment
Additional context
cc @yakkomajuri and @tiina303 for team ingestion/data consistency context.
Example of a person deletion causing issues: https://posthog.slack.com/archives/C02LR7352SG/p1661424741783579
I think there's only two possible fixes:
Thank you for your bug report – we love squashing them!