search

biothings / myvariant.info

MyVariant.info: A BioThings API for human variant annotations
http://myvariant.info
Other
87 stars 32 forks source link

post-merge errors of superhot builds #150

Open erikyao opened 1 year ago

erikyao commented 1 year ago 
Oct 12 14:43:26 su12 python[1038438]: INFO:build:Finalizing target backend
Oct 12 14:43:26 su12 python[1038438]: INFO:build:Running post-merge process
Oct 12 14:43:26 su12 python[1038438]: DEBUG:build:no_uploader_running: src full names {'cgi', 'clinvar.clinvar_hg19', 'geno2mp', 'civic', 'emv.emv_hg19'}
Oct 12 14:43:26 su12 python[1038438]: INFO:build:Validating...
Oct 12 14:43:26 su12 python[1038438]: WARNING:build:Nothing was inserted in target collection...
Oct 12 14:43:26 su12 python[1038438]: WARNING:build:Total count of documents 39009683 is greater than what was inserted/updated... 0]
Oct 12 14:43:26 su12 python[1038438]: DEBUG:hub:Remove PID file '/nvme/myvariant/run/ThreadPoolExecutor-0_0_CjHRnGuY.pickle'
Oct 12 14:43:26 su12 python[1038438]: INFO:build:Fetch _ids from 'superhot_hg19_20221012_gc8jjeux' with batch_size=2800000, and create post-merger job with batch_size=50000
Oct 12 14:43:26 su12 python[1038438]: ERROR:asyncio:Task was destroyed but it is pending!
Oct 12 14:43:26 su12 python[1038438]: task: <Task pending name='Task-4239' coro=<WebSocketProtocol13.write_message.<locals>.wrapper() running at /opt/home/myvariant/venv/lib/python3.10/site-packages/tornado/websocket.py:1090>>
Oct 12 14:43:26 su12 python[1038438]: /usr/lib/python3.10/asyncio/base_events.py:671: RuntimeWarning: coroutine 'WebSocketProtocol13.write_message.<locals>.wrapper' was never awaited
Oct 12 14:43:26 su12 python[1038438]:   self._ready.clear()
Oct 12 14:43:26 su12 python[1038438]: RuntimeWarning: Enable tracemalloc to get the object allocation traceback
Oct 12 14:43:26 su12 python[1038438]: ERROR:asyncio:Task was destroyed but it is pending!
Oct 12 14:43:26 su12 python[1038438]: task: <Task pending name='Task-4240' coro=<WebSocketProtocol13.write_message.<locals>.wrapper() running at /opt/home/myvariant/venv/lib/python3.10/site-packages/tornado/websocket.py:1090>>
Oct 12 14:43:26 su12 python[1038438]: ERROR:asyncio:Task was destroyed but it is pending!
Oct 12 14:43:26 su12 python[1038438]: task: <Task pending name='Task-4241' coro=<WebSocketProtocol13.write_message.<locals>.wrapper() running at /opt/home/myvariant/venv/lib/python3.10/site-packages/tornado/websocket.py:1090>>
Oct 12 14:43:26 su12 python[1038438]: WARNING:build:Can't find information for target collection 'superhot_hg19_20221012_gc8jjeux'
Oct 12 14:43:26 su12 python[1038438]: DEBUG:build:No cache file found (or invalid) for 'superhot_hg19_20221012_gc8jjeux', use doc_feeder
Oct 12 14:43:26 su12 python[1038438]: INFO:build:Building cache file '/nvme/myvariant/cache/superhot_hg19_20221012_gc8jjeux.xz._tmp_aovrV8rM'
Oct 12 14:44:19 su12 python[1038438]: INFO:build:Creating post-merge job #1/781 to process chrom 50000/39009683 (0.1%)
Oct 12 14:44:19 su12 python[1038438]: /opt/home/myvariant/myvariant.info/src/hub/databuild/builder.py:72: RuntimeWarning: coroutine 'JobManager.defer_to_process' was never awaited
Oct 12 14:44:19 su12 python[1038438]:   job = yield from job_manager.defer_to_process(pinfo,
Oct 12 14:44:19 su12 python[1038438]: RuntimeWarning: Enable tracemalloc to get the object allocation traceback
Oct 12 14:44:19 su12 python[1038438]: ERROR:build:Failed post-merging source: cannot 'yield from' a coroutine object in a non-coroutine generator
Oct 12 14:44:19 su12 python[1038438]: Traceback (most recent call last):
Oct 12 14:44:19 su12 python[1038438]:   File "/opt/home/myvariant/venv/src/biothings/biothings/hub/databuild/builder.py", line 841, in postmerged
Oct 12 14:44:19 su12 python[1038438]:     self.logger.info("Post-merge completed [%s]" % f.result())
Oct 12 14:44:19 su12 python[1038438]:   File "/opt/home/myvariant/venv/src/biothings/biothings/utils/manager.py", line 887, in run
Oct 12 14:44:19 su12 python[1038438]:     res = await res
Oct 12 14:44:19 su12 python[1038438]:   File "/opt/home/myvariant/myvariant.info/src/hub/databuild/builder.py", line 72, in set_chrom
Oct 12 14:44:19 su12 python[1038438]:     job = yield from job_manager.defer_to_process(pinfo,
Oct 12 14:44:19 su12 python[1038438]: TypeError: cannot 'yield from' a coroutine object in a non-coroutine generator