cofacts / rumors-db

Scripts for managing rumors db
MIT License
5 stars 11 forks source link

Reload individual file and adjust mapping #53

Closed MrOrz closed 3 years ago

MrOrz commented 3 years ago

This PR enables us to reload one DB index and leave all other indexes intact. After this PR is pushed to production, we can perform fix on cofacts/rumors-site#399 and cofacts/rumors-site#386

MrOrz commented 3 years ago

Execution steps

Setup SSH tunnel

ssh -L 62222:<elasticsearch-ip-on-server>:62222 <server url>

And ensure .env file exists, with ELASTICSEARCH_URL=localhost:62222.

Execution & Results

$ npm run reload -- users

> rumors-db@1.0.2 reload /Users/johnsonliang/workspace/rumors-db
> babel-node db/reloadSchema.js "users"

Source:  users_v1_0_2
Target:  users_v1_1_0
Reindexed from users_v1_0_2 to users_v1_1_0 in 8 seconds.
Setup users_v1_1_0 -> users alias and remove users_v1_0_2.

$ npm run reload -- articles

> rumors-db@1.0.2 reload /Users/johnsonliang/workspace/rumors-db
> babel-node db/reloadSchema.js "articles"

Source:  articles_v1_0_2
Target:  articles_v1_1_0
Reindexed from articles_v1_0_2 to articles_v1_1_0 in 63 seconds.
Setup articles_v1_1_0 -> articles alias and remove articles_v1_0_2.

$ npm run reload -- replies

> rumors-db@1.0.2 reload /Users/johnsonliang/workspace/rumors-db
> babel-node db/reloadSchema.js "replies"

Source:  replies_v1_0_2
Target:  replies_v1_1_0
Reindexed from replies_v1_0_2 to replies_v1_1_0 in 78 seconds.
Setup replies_v1_1_0 -> replies alias and remove replies_v1_0_2.

$ npm run reload -- urls

> rumors-db@1.0.2 reload /Users/johnsonliang/workspace/rumors-db
> babel-node db/reloadSchema.js "urls"

Source:  urls_v1_0_2
Target:  urls_v1_1_0
Reindexed from urls_v1_0_2 to urls_v1_1_0 in 283 seconds.
Setup urls_v1_1_0 -> urls alias and remove urls_v1_0_2.

** Note: timeouts should be increased, as urls are pretty close to timeout.