Update distribution "how to" with updated combined logic

[stepps00]

Since the whosonfirst-data repo has split into per-country repos, update the dist cookbook document to reflect the new changes. The -combined flag requires passing all the new repo names, so the new command to get all the admin data is something like:

./bin/wof-dist-build -build-bundle -custom-repo -preserve-checkout -local-checkout -index-alt-files -timings -verbose -workdir /Users/stephen.epps/development/wof -combined -combined-name whosonfirst-data-admin-combined whosonfirst-data-admin-gb whosonfirst-data-admin-pa whosonfirst-data-admin-gd whosonfirst-data-admin-pe whosonfirst-data-admin-ge whosonfirst-data-admin-pf whosonfirst-data-admin-gf whosonfirst-data-admin-pg whosonfirst-data-admin-gg whosonfirst-data-admin-ph whosonfirst-data-admin-gh whosonfirst-data-admin-pk whosonfirst-data-admin-gi whosonfirst-data-admin-pl whosonfirst-data-admin-gl whosonfirst-data-admin-pm whosonfirst-data-admin-gm whosonfirst-data-admin-pn whosonfirst-data-admin-gn whosonfirst-data-admin-pr whosonfirst-data-admin-gp whosonfirst-data-admin-ps whosonfirst-data-admin-gq whosonfirst-data-admin-pt whosonfirst-data-admin-gr whosonfirst-data-admin-pw whosonfirst-data-admin-gs whosonfirst-data-admin-py whosonfirst-data-admin-gt whosonfirst-data-admin-qa whosonfirst-data-admin-gu whosonfirst-data-admin-re whosonfirst-data-admin-ad whosonfirst-data-admin-gw whosonfirst-data-admin-ro whosonfirst-data-admin-ae whosonfirst-data-admin-gy whosonfirst-data-admin-rs whosonfirst-data-admin-af whosonfirst-data-admin-hk whosonfirst-data-admin-ru whosonfirst-data-admin-ag whosonfirst-data-admin-hm whosonfirst-data-admin-rw whosonfirst-data-admin-ai whosonfirst-data-admin-hn whosonfirst-data-admin-sa whosonfirst-data-admin-al whosonfirst-data-admin-hr whosonfirst-data-admin-sb whosonfirst-data-admin-ht whosonfirst-data-admin-sc whosonfirst-data-admin-am whosonfirst-data-admin-hu whosonfirst-data-admin-sd whosonfirst-data-admin-an whosonfirst-data-admin-id whosonfirst-data-admin-se whosonfirst-data-admin-ao whosonfirst-data-admin-ie whosonfirst-data-admin-sg whosonfirst-data-admin-aq whosonfirst-data-admin-il whosonfirst-data-admin-sh whosonfirst-data-admin-ar whosonfirst-data-admin-im whosonfirst-data-admin-si whosonfirst-data-admin-as whosonfirst-data-admin-in whosonfirst-data-admin-sj whosonfirst-data-admin-at whosonfirst-data-admin-io whosonfirst-data-admin-sk whosonfirst-data-admin-au whosonfirst-data-admin-iq whosonfirst-data-admin-sl whosonfirst-data-admin-aw whosonfirst-data-admin-ir whosonfirst-data-admin-sm whosonfirst-data-admin-ax whosonfirst-data-admin-is whosonfirst-data-admin-sn whosonfirst-data-admin-az whosonfirst-data-admin-it whosonfirst-data-admin-so whosonfirst-data-admin-ba whosonfirst-data-admin-je whosonfirst-data-admin-sr whosonfirst-data-admin-bb whosonfirst-data-admin-jm whosonfirst-data-admin-ss whosonfirst-data-admin-bd whosonfirst-data-admin-jo whosonfirst-data-admin-st whosonfirst-data-admin-be whosonfirst-data-admin-jp whosonfirst-data-admin-sv whosonfirst-data-admin-bf whosonfirst-data-admin-ke whosonfirst-data-admin-sx whosonfirst-data-admin-bg whosonfirst-data-admin-kg whosonfirst-data-admin-sy whosonfirst-data-admin-bh whosonfirst-data-admin-kh whosonfirst-data-admin-sz whosonfirst-data-admin-bi whosonfirst-data-admin-ki whosonfirst-data-admin-tc whosonfirst-data-admin-bj whosonfirst-data-admin-km whosonfirst-data-admin-td whosonfirst-data-admin-bl whosonfirst-data-admin-kn whosonfirst-data-admin-tf whosonfirst-data-admin-bm whosonfirst-data-admin-ko whosonfirst-data-admin-tg whosonfirst-data-admin-bn whosonfirst-data-admin-kp whosonfirst-data-admin-th whosonfirst-data-admin-bo whosonfirst-data-admin-kr whosonfirst-data-admin-tj whosonfirst-data-admin-bq whosonfirst-data-admin-kw whosonfirst-data-admin-tk whosonfirst-data-admin-br whosonfirst-data-admin-ky whosonfirst-data-admin-tl whosonfirst-data-admin-bs whosonfirst-data-admin-kz whosonfirst-data-admin-tm whosonfirst-data-admin-bt whosonfirst-data-admin-la whosonfirst-data-admin-tn whosonfirst-data-admin-bw whosonfirst-data-admin-lb whosonfirst-data-admin-to whosonfirst-data-admin-by whosonfirst-data-admin-lc whosonfirst-data-admin-tr whosonfirst-data-admin-bz whosonfirst-data-admin-li whosonfirst-data-admin-tt whosonfirst-data-admin-ca whosonfirst-data-admin-lk whosonfirst-data-admin-tu whosonfirst-data-admin-cc whosonfirst-data-admin-lr whosonfirst-data-admin-tv whosonfirst-data-admin-cd whosonfirst-data-admin-ls whosonfirst-data-admin-tw whosonfirst-data-admin-cf whosonfirst-data-admin-lt whosonfirst-data-admin-tz whosonfirst-data-admin-cg whosonfirst-data-admin-lu whosonfirst-data-admin-ua whosonfirst-data-admin-ch whosonfirst-data-admin-lv whosonfirst-data-admin-ug whosonfirst-data-admin-ci whosonfirst-data-admin-ly whosonfirst-data-admin-uk whosonfirst-data-admin-ck whosonfirst-data-admin-ma whosonfirst-data-admin-um whosonfirst-data-admin-cl whosonfirst-data-admin-mc whosonfirst-data-admin-un whosonfirst-data-admin-cm whosonfirst-data-admin-md whosonfirst-data-admin-us whosonfirst-data-admin-cn whosonfirst-data-admin-me whosonfirst-data-admin-uy whosonfirst-data-admin-co whosonfirst-data-admin-mf whosonfirst-data-admin-uz whosonfirst-data-admin-cr whosonfirst-data-admin-mg whosonfirst-data-admin-va whosonfirst-data-admin-cu whosonfirst-data-admin-mh whosonfirst-data-admin-vc whosonfirst-data-admin-cv whosonfirst-data-admin-mk whosonfirst-data-admin-ve whosonfirst-data-admin-cw whosonfirst-data-admin-ml whosonfirst-data-admin-vg whosonfirst-data-admin-cx whosonfirst-data-admin-mm whosonfirst-data-admin-vi whosonfirst-data-admin-cy whosonfirst-data-admin-mn whosonfirst-data-admin-vn whosonfirst-data-admin-cz whosonfirst-data-admin-mo whosonfirst-data-admin-vu whosonfirst-data-admin-de whosonfirst-data-admin-mp whosonfirst-data-admin-wf whosonfirst-data-admin-mq whosonfirst-data-admin-ws whosonfirst-data-admin-mr whosonfirst-data-admin-xk whosonfirst-data-admin-ms whosonfirst-data-admin-xn whosonfirst-data-admin-dj whosonfirst-data-admin-mt whosonfirst-data-admin-xs whosonfirst-data-admin-dk whosonfirst-data-admin-mu whosonfirst-data-admin-xx whosonfirst-data-admin-dm whosonfirst-data-admin-mv whosonfirst-data-admin-xy whosonfirst-data-admin-dn whosonfirst-data-admin-mw whosonfirst-data-admin-xz whosonfirst-data-admin-do whosonfirst-data-admin-mx whosonfirst-data-admin-ye whosonfirst-data-admin-dz whosonfirst-data-admin-my whosonfirst-data-admin-yt whosonfirst-data-admin-ec whosonfirst-data-admin-mz whosonfirst-data-admin-za whosonfirst-data-admin-ee whosonfirst-data-admin-na whosonfirst-data-admin-zm whosonfirst-data-admin-eg whosonfirst-data-admin-nc whosonfirst-data-admin-zw whosonfirst-data-admin-eh whosonfirst-data-admin-ne whosonfirst-data-admin-er whosonfirst-data-admin-nf whosonfirst-data-admin-es whosonfirst-data-admin-ng whosonfirst-data-admin-et whosonfirst-data-admin-ni whosonfirst-data-admin-fi whosonfirst-data-admin-nl whosonfirst-data-admin-fj whosonfirst-data-admin-no whosonfirst-data-admin-fk whosonfirst-data-admin-np whosonfirst-data-admin-fm whosonfirst-data-admin-nr whosonfirst-data-admin-fo whosonfirst-data-admin-nu whosonfirst-data-admin-fr whosonfirst-data-admin-nz whosonfirst-data-admin-ga whosonfirst-data-admin-om 

Also add the caveat that Go 1.12 is required. 1.10 seems to not like -mod vendor.

[thisisaaronland]

@stepps00 You are correct about the dependency on Go 1.12 or higher. As a general rule I hate requiring stuff like this but because Go modules, in general, make life easier it seems like an acceptible trade-off.

[stepps00]

Running the combined dist code against the current state of the per-country repos successfully created a ~8gb compressed (~35gb uncompressed) combined file with 5.3M admin records.

• Update any cookbook doc related to dist building with new instructions, close this issue
• Copy terminal output that created the combined dist to a Gist
• Update any README/documentation in go/docker repo(s)

Related:

• https://github.com/whosonfirst/docker-whosonfirst-data-distributions-combined
• https://github.com/whosonfirst/docker-whosonfirst-data-distributions

[stepps00]

Notes from the "combined" distribution here: https://gist.github.com/stepps00/47a42d80b4e2cd5fee7fcf10989c284e

I'd like to create two cookbook docs, one on creating a distribution for a single repository (essentially just update this one) and one with the new "combined" logic, as outlined in the gist.

I'll re-run the combined dist code today to create a new dist file and to confirm the steps in this Gist are correct, then update the cookbook docs.

[stepps00]

~Note that the "combined" data distributions also include constituency repos. Update any command line lists of repos to include the constituency repos.~

[nvkelso]

Arguably constituency is included in the "admin" combo dist in error? If it's always been there then we can consider keeping it as a documented oddity?

[stepps00]

The constituency comment was a mistake on my end - they should not be included in the combined admin distribution.. (they have their own distribution).

[stepps00]

I've updated docs in #34, mostly using the steps outlined in https://gist.github.com/stepps00/47a42d80b4e2cd5fee7fcf10989c284e

Since the existing (outdated) how-to doc is linked in other whosonfirst repos/docs, rather than renaming/deleting the existing doc, I deprecated it and linked to the new documentation.