biothings / myvariant.info

MyVariant.info: A BioThings API for human variant annotations
http://myvariant.info
Other
86 stars 32 forks source link

superhot build docs lost `index.<build_id>.host` and `index.<build_id>.environment` fields #171

Closed erikyao closed 11 months ago

erikyao commented 11 months ago

Problem introduced with biothings.api 0.12.x release.

The base ColdHotIndexer defined in biothings.api has the following structure that each of its worker indexer relies on a build doc to initialize, and these two build docs are actually pointing to the same doc in MongoDB.

ColdHotIndexer-hackerdraw-1

All worker indexer implemented in biothings.api uses the following safe procedure to update its associated build doc:

ColdHotIndexer-hackerdraw-3

However, developers subclassing the base worker indexer may be unaware of the two write operations in ColdHotIndexer and come up with following implementation (since it's quite straightforward to keep the build doc as a member of the worker indexer)

ColdHotIndexer-hackerdraw-2

Such worker indexer implementation, when put into a ColdHotIndexer, will overwrite the content first written to MongoDB.

In MyVariant.info, BaseVariantIndexer has exactly such a problem, and that's why those two fields, index.<build_id>.host and index.<build_id>.environment, are lost in build docs.

erikyao commented 11 months ago

Fixed with commit 11c3f787a64a4e11b24966caa3c4a1e6a0d3014b