This PR updates src/schema to latest and fix unit test fixtures that would insert extra fields by accident.

Migration needed

Extra fields that needs removal

export ES_HOST=localhost:62223

# articles: 3~15 minutes
curl "$ES_HOST/articles/doc/_update_by_query?conflicts=proceed" -H 'content-type:application/json' -d '{"script":{"source":"ctx._source.remove(\"appUserId\"); ctx._source.remove(\"tags\"); ctx._source.remove(\"normalArticleReplycount\"); for (int i = 0; i < ctx._source.articleCategories.length; ++i) { ctx._source.articleCategories[i].remove(\"appUserId\"); } for (int i = 0; i < ctx._source.references.length; ++i) { ctx._source.references[i].remove(\"appUserId\"); } for (int i = 0; i < ctx._source.articleReplies.length; ++i) { ctx._source.articleReplies[i].remove(\"appUserId\"); }"}}'

# articlereplyfeedbacks: 2 minutes
curl "$ES_HOST/articlereplyfeedbacks/doc/_update_by_query?conflicts=proceed" -H 'content-type:application/json' -d '{"script":{"source":"ctx._source.remove(\"appUserId\");"},"query":{"exists": {"field":"appUserId"}}}'

# replyrequests: 42s
curl "$ES_HOST/replyrequests/doc/_update_by_query?conflicts=proceed" -H 'content-type:application/json' -d '{"script":{"source":"ctx._source.remove(\"appUserId\");"},"query":{"exists": {"field":"appUserId"}}}'

# user special case - weird nameless situation
curl -XPOST "$ES_HOST/users/doc/CAdXE3EBrhVJn3LNMoiA/_update" -H 'content-type:application/json' -d '{"doc":{"name":""}}'

# user special case - blocked by accident
curl -XPOST "$ES_HOST/users/doc/_bdjyJIBDfH0hhhZtJMk/_update" -H 'content-type:application/json' -d '{"script":{"source":"ctx._source.remove(\"blockedReason\");"}}'

Also need to manually reload (reindex) all db indexes. May need to take down API servers so that no insertion happens during reindex process (prevent new docs inserted into old index during re-index).

Stats on localhost DB & staging:

articlecategoryfeedbacks, categories, cooccurrences: quite fast
airesponses: 30~45 secs
articlereplyfeedbacks: 42 secs
replyrequests: 45s
ydocs: 4s ~ 1 min
replies: 1.5~7 mins
users: 32 secs
articles: 2min
analytics: 25 mins
urls:

Migration script executed; all indices reloaded.

docker@debian:~/rumors-db$ npm run reload -- articlecategoryfeedbacks

> rumors-db@1.0.2 reload
> babel-node --extensions .ts,.js db/reloadSchema.js "articlecategoryfeedbacks"

Source:  articlecategoryfeedbacks_v1_1_1
Target:  articlecategoryfeedbacks_v1_1_2
Reindexed from articlecategoryfeedbacks_v1_1_1 to articlecategoryfeedbacks_v1_1_2 in 1 seconds.
Setup articlecategoryfeedbacks_v1_1_2 -> articlecategoryfeedbacks alias and remove articlecategoryfeedbacks_v1_1_1.
docker@debian:~/rumors-db$ npm run reload -- categories

> rumors-db@1.0.2 reload
> babel-node --extensions .ts,.js db/reloadSchema.js "categories"

Source:  categories_v1_0_2
Target:  categories_v1_1_1
Reindexed from categories_v1_0_2 to categories_v1_1_1 in 0 seconds.
Setup categories_v1_1_1 -> categories alias and remove categories_v1_0_2.
docker@debian:~/rumors-db$ npm run reload -- cooccurrences

> rumors-db@1.0.2 reload
> babel-node --extensions .ts,.js db/reloadSchema.js "cooccurrences"

Source:  cooccurrences_v1_0_0
Target:  cooccurrences_v1_0_1
Reindexed from cooccurrences_v1_0_0 to cooccurrences_v1_0_1 in 0 seconds.
Setup cooccurrences_v1_0_1 -> cooccurrences alias and remove cooccurrences_v1_0_0.
docker@debian:~/rumors-db$ npm run reload -- airesponses

> rumors-db@1.0.2 reload
> babel-node --extensions .ts,.js db/reloadSchema.js "airesponses"

Source:  airesponses_v1_0_0
Target:  airesponses_v1_0_1
Reindexed from airesponses_v1_0_0 to airesponses_v1_0_1 in 23 seconds.
Setup airesponses_v1_0_1 -> airesponses alias and remove airesponses_v1_0_0.
docker@debian:~/rumors-db$ npm run reload -- articlereplyfeedbacks

> rumors-db@1.0.2 reload
> babel-node --extensions .ts,.js db/reloadSchema.js "articlereplyfeedbacks"

Source:  articlereplyfeedbacks_v1_2_0
Target:  articlereplyfeedbacks_v1_2_1
Reindexed from articlereplyfeedbacks_v1_2_0 to articlereplyfeedbacks_v1_2_1 in 23 seconds.
Setup articlereplyfeedbacks_v1_2_1 -> articlereplyfeedbacks alias and remove articlereplyfeedbacks_v1_2_0.
docker@debian:~/rumors-db$ npm run reload -- replyrequests

> rumors-db@1.0.2 reload
> babel-node --extensions .ts,.js db/reloadSchema.js "replyrequests"

Source:  replyrequests_v1_1_1
Target:  replyrequests_v1_1_2
Reindexed from replyrequests_v1_1_1 to replyrequests_v1_1_2 in 24 seconds.
Setup replyrequests_v1_1_2 -> replyrequests alias and remove replyrequests_v1_1_1.
docker@debian:~/rumors-db$ npm run reload -- ydocs

> rumors-db@1.0.2 reload
> babel-node --extensions .ts,.js db/reloadSchema.js "ydocs"

Source:  ydocs_v1_0_1
Target:  ydocs_v1_0_2
Reindexed from ydocs_v1_0_1 to ydocs_v1_0_2 in 2 seconds.
Setup ydocs_v1_0_2 -> ydocs alias and remove ydocs_v1_0_1.
docker@debian:~/rumors-db$ npm run reload -- replies

> rumors-db@1.0.2 reload
> babel-node --extensions .ts,.js db/reloadSchema.js "replies"

Source:  replies_v1_1_0
Target:  replies_v1_1_1
Reindexed from replies_v1_1_0 to replies_v1_1_1 in 53 seconds.
Setup replies_v1_1_1 -> replies alias and remove replies_v1_1_0.
docker@debian:~/rumors-db$ npm run reload -- users

> rumors-db@1.0.2 reload
> babel-node --extensions .ts,.js db/reloadSchema.js "users"

Source:  users_v1_2_1
Target:  users_v1_2_2
Reindexed from users_v1_2_1 to users_v1_2_2 in 17 seconds.
Setup users_v1_2_2 -> users alias and remove users_v1_2_1.
docker@debian:~/rumors-db$ npm run reload -- articles

> rumors-db@1.0.2 reload
> babel-node --extensions .ts,.js db/reloadSchema.js "articles"

Source:  articles_v1_4_0
Target:  articles_v1_4_1
Reindexed from articles_v1_4_0 to articles_v1_4_1 in 54 seconds.
Setup articles_v1_4_1 -> articles alias and remove articles_v1_4_0.
docker@debian:~/rumors-db$ 
docker@debian:~/rumors-db$ npm run reload -- analytics

> rumors-db@1.0.2 reload
> babel-node --extensions .ts,.js db/reloadSchema.js "analytics"

Source:  analytics_v1_2_0
Target:  analytics_v1_2_1

Reindexed from analytics_v1_2_0 to analytics_v1_2_1 in 724 seconds.
Setup analytics_v1_2_1 -> analytics alias and remove analytics_v1_2_0.
docker@debian:~/rumors-db$ 
docker@debian:~/rumors-db$ 
docker@debian:~/rumors-db$ npm run reload -- urls

> rumors-db@1.0.2 reload
> babel-node --extensions .ts,.js db/reloadSchema.js "urls"

Source:  urls_v1_1_0
Target:  urls_v1_1_1
Reindexed from urls_v1_1_0 to urls_v1_1_1 in 251 seconds.
Setup urls_v1_1_1 -> urls alias and remove urls_v1_1_0.

cofacts / rumors-api

Disable dynamic ES mapping and fix test fixtures #342

Migration needed