Posthog Migration to Govt Server

Mission-Prerna / nl-apis

All the BE APIs to configure, manage & run the NL app

1 stars 16 forks source link

Posthog Migration to Govt Server #280

Closed nikhil241 closed 2 months ago

nikhil241 commented 3 months ago

karntrehan commented 3 months ago

Picked today. Services + Data to be migrated to prod server (103.X.X.X) Backup started. Prod Downtime: ~4 hours. To be done in the second half today.

karntrehan commented 3 months ago

Deployment not completed yesterday due to @karntrehan's absence.

Deployment plan: Currently nl.telemetry.X.X maps to our server and is configured through remote config to all users.

We shall create a new domain -> nipun.telemetry.X.X mapping to the govt server (103.X.X.X) and test first on staging, followed by phased rollout of the domain to all users.

cc @nikhil241

VargishChauhan commented 3 months ago

Nikhil and Ajay are working on it. Need to take backup once more due to bugs in the first one

AakashSatpute119 commented 3 months ago

@TheNPDev to share new build for testing.

TheNPDev commented 3 months ago

@AakashSatpute119 Build: https://drive.google.com/file/d/1qbZYlzXpLj1Y6KiIif8hkjbLQTBpAfs9/view?usp=sharing

charanpreet-s commented 3 months ago

@dhruv-1001 to add posthog creds to creds sheet

dhruv-1001 commented 2 months ago

@AakashSatpute119 to connect with @nikhil241 to check out why it's not getting events.

AakashSatpute119 commented 2 months ago

Able to see events on new PostHog server. @nikhil241

charanpreet-s commented 2 months ago

@nikhil241 is migrating data. Post that android URLs will be changed on firebase.

dhruv-1001 commented 2 months ago

@charanpreet-s to point to new telemetry URL. Data gap in nipun.telemetry.x.x [9th & 10th Sept].

charanpreet-s commented 2 months ago

Moving to 10% of the users

karntrehan commented 2 months ago

Experiment config did not work. Only 3 events in an hour received. @charanpreet-s to fix 10% rollout. Post that call to be taken.

Data will needed to be migrated. To be done next week.

VargishChauhan commented 2 months ago

New Events not getting captured on govt servers. Need to do an RCT for the same

karntrehan commented 2 months ago

Clickhouse and zookeeper have gone out of sync. @nikhil241 to check by redeploying older backup. Once system is up and running, we can test internally. All users to keep pointing to nl.X.X.X.

On the weekend, nl.X.X.X to be put down. Entire data to be migrated to nipun.X.X.X. Tested quickly and on monday all users to point to nipun.X.X.X from firebase.

cc @nikhil241 @charanpreet-s @AakashSatpute119

dhruv-1001 commented 2 months ago

@AakashSatpute119 to test again. @nikhil241 to take backup during weekend.

AakashSatpute119 commented 2 months ago

@AakashSatpute119 to test again. @nikhil241 to take backup during weekend.

Tested with this Build getting all events on a new server.

VargishChauhan commented 2 months ago

Migration to govt server complete.