search

nielsboecker / HoloRepository-Core

A system for transforming medical imaging studies into holograms, storing them in the cloud and providing them to other systems.
GNU Affero General Public License v3.0
6 stars 8 forks source link

Dense vnet abdominal image throws errors on idle #78

Closed boonwj closed 5 years ago

boonwj commented 5 years ago 
holopipelines-core_1                             | [2019-08-26 11:25:14 +0000] [1] [CRITICAL] WORKER TIMEOUT (pid:9)
holopipelines-core_1                             | [2019-08-26 11:25:14 +0000] [9] [INFO] Worker exiting (pid: 9)
holopipelines-core_1                             | [2019-08-26 11:25:14 +0000] [17] [INFO] Booting worker with pid: 17
holopipelines-core_1                             | /usr/local/lib/python3.7/site-packages/dotenv/main.py:52: UserWarning: File doesn't exist 
holopipelines-core_1                             |   warnings.warn("File doesn't exist {}".format(self.dotenv_path))  # type: ignore
holopipelines-core_1                             | WARNING:root:Started process pool with 4 worker processes
holopipelines-models__dense_vnet_abdominal_ct_1  | [2019-08-26 11:25:14 +0000] [1] [CRITICAL] WORKER TIMEOUT (pid:8)
holopipelines-models__dense_vnet_abdominal_ct_1  | [2019-08-26 11:25:14 +0000] [1] [CRITICAL] WORKER TIMEOUT (pid:9)
holopipelines-models__dense_vnet_abdominal_ct_1  | [2019-08-26 11:25:14 +0000] [1] [CRITICAL] WORKER TIMEOUT (pid:10)
holopipelines-models__dense_vnet_abdominal_ct_1  | [2019-08-26 11:25:14 +0000] [1] [CRITICAL] WORKER TIMEOUT (pid:12)
holopipelines-models__dense_vnet_abdominal_ct_1  | [2019-08-26 11:25:14 +0000] [8] [INFO] Worker exiting (pid: 8)
holopipelines-models__dense_vnet_abdominal_ct_1  | [2019-08-26 11:25:14 +0000] [9] [INFO] Worker exiting (pid: 9)
holopipelines-models__dense_vnet_abdominal_ct_1  | [2019-08-26 11:25:14 +0000] [12] [INFO] Worker exiting (pid: 12)
holopipelines-models__dense_vnet_abdominal_ct_1  | [2019-08-26 11:25:14 +0000] [10] [INFO] Worker exiting (pid: 10)
holopipelines-models__dense_vnet_abdominal_ct_1  | [2019-08-26 11:25:14 +0000] [16] [INFO] Booting worker with pid: 16
holopipelines-models__dense_vnet_abdominal_ct_1  | [2019-08-26 11:25:14 +0000] [17] [INFO] Booting worker with pid: 17
holopipelines-models__dense_vnet_abdominal_ct_1  | [2019-08-26 11:25:14 +0000] [18] [INFO] Booting worker with pid: 18
holopipelines-models__dense_vnet_abdominal_ct_1  | [2019-08-26 11:25:14 +0000] [20] [INFO] Booting worker with pid: 20
holopipelines-core_1                             | [2019-08-26 11:53:36 +0000] [1] [CRITICAL] WORKER TIMEOUT (pid:17)
holopipelines-core_1                             | [2019-08-26 11:53:36 +0000] [17] [INFO] Worker exiting (pid: 17)
holopipelines-core_1                             | [2019-08-26 11:53:36 +0000] [25] [INFO] Booting worker with pid: 25
holopipelines-core_1                             | /usr/local/lib/python3.7/site-packages/dotenv/main.py:52: UserWarning: File doesn't exist 
holopipelines-core_1                             |   warnings.warn("File doesn't exist {}".format(self.dotenv_path))  # type: ignore
holopipelines-core_1                             | WARNING:root:Started process pool with 4 worker processes
holopipelines-models__dense_vnet_abdominal_ct_1  | [2019-08-26 11:53:36 +0000] [1] [CRITICAL] WORKER TIMEOUT (pid:16)
holopipelines-models__dense_vnet_abdominal_ct_1  | [2019-08-26 11:53:36 +0000] [1] [CRITICAL] WORKER TIMEOUT (pid:17)
holopipelines-models__dense_vnet_abdominal_ct_1  | [2019-08-26 11:53:36 +0000] [1] [CRITICAL] WORKER TIMEOUT (pid:18)
holopipelines-models__dense_vnet_abdominal_ct_1  | [2019-08-26 11:53:36 +0000] [1] [CRITICAL] WORKER TIMEOUT (pid:20)
holopipelines-models__dense_vnet_abdominal_ct_1  | [2019-08-26 11:53:36 +0000] [16] [INFO] Worker exiting (pid: 16)
holopipelines-models__dense_vnet_abdominal_ct_1  | [2019-08-26 11:53:36 +0000] [18] [INFO] Worker exiting (pid: 18)
holopipelines-models__dense_vnet_abdominal_ct_1  | [2019-08-26 11:53:36 +0000] [20] [INFO] Worker exiting (pid: 20)
holopipelines-models__dense_vnet_abdominal_ct_1  | [2019-08-26 11:53:36 +0000] [17] [INFO] Worker exiting (pid: 17)
holopipelines-models__dense_vnet_abdominal_ct_1  | [2019-08-26 11:53:36 +0000] [24] [INFO] Booting worker with pid: 24
holopipelines-models__dense_vnet_abdominal_ct_1  | [2019-08-26 11:53:36 +0000] [25] [INFO] Booting worker with pid: 25
holopipelines-models__dense_vnet_abdominal_ct_1  | [2019-08-26 11:53:36 +0000] [27] [INFO] Booting worker with pid: 27
holopipelines-models__dense_vnet_abdominal_ct_1  | [2019-08-26 11:53:36 +0000] [29] [INFO] Booting worker with pid: 29

Noticed this happening when I left the images up during testing. Any idea what could be the case?

I haven't checked if this actually causes issues with the workflow.

boonwj commented 5 years ago

Since this is still in PR. Will direct the question back there #67