weaveworks / service

☁️ Images for Weave Cloud (R) (TM) (C) ☁️
https://cloud.weave.works
2 stars 2 forks source link

Update scope data cleanup information #2732

Closed ozamosi closed 2 years ago

ozamosi commented 2 years ago

Queries to find the new org IDs: Deleted:

select string_agg(id, ' ' order by id::integer) from organizations where deleted_at >= '2021-03-15' and deleted_at < '2021-10-01' and (first_seen_scope_connected_at is not null);

Expired:

  select string_agg(id, ' ' order by id::integer) from organizations where created_at >= '2021-03-15' and deleted_at is null and (trial_expires_at < '2021-10-01' and refuse_data_upload and refuse_data_access) and not ('no-billing' = any(feature_flags)) and (first_seen_scope_connected_at is not null);

Ordinary (not enterprise deal):

  select string_agg(id, ' ' order by id::integer) from organizations where created_at >= '2021-03-15' and created_at < '2021-10-01' and deleted_at is null and not (refuse_data_upload and refuse_data_access) and not ('no-billing' = any(feature_flags)) and (first_seen_scope_connected_at is not null);

I dumped these to the end of prod-orgs-to-delete and ran (try displaying it with -w and you'll see it's mostly the same still)

cat prod-orgs-to-delete | sed 's/ /\n/g' | sort | uniq | awk 'ORS=NR%13?FS:RS' > prod-orgs && mv prod-orgs prod-orgs-to-delete

Then I ran the bigjob.yaml job to re-generate the records, and generated the file through:

  kubectl --context=dev logs -n scope robin-scope-scanner | sort -n | gzip