Hello,
we currently have a text analysis process running in docker. Part of it implies some text processing, which is run in a billiard Pool.
However, it randomly raises the following exception:
Traceback (most recent call last):
File "/project/tasks.py", line 37, in semantic_analysis
worker.run(corpus_id)
File "/project/text_worker.py", line 61, in run
raise e
File "/project/text_worker.py", line 35, in run
corp.start()
File "/project/sem/corpus.py", line 382, in start
self.clean_content()
File "/project/sem/corpus.py", line 178, in clean_content
self.tokenized_content = self.text.parallel_tokenize(content, allowed_pos=self.allowed_pos_union)
File "/project/sem/text.py", line 116, in parallel_tokenize
pool = Pool(4)
File "/usr/local/lib/python3.6/site-packages/billiard/context.py", line 165, in Pool
context=self.get_context())
File "/usr/local/lib/python3.6/site-packages/billiard/pool.py", line 1007, in __init__
self._create_worker_process(i)
File "/usr/local/lib/python3.6/site-packages/billiard/pool.py", line 1116, in _create_worker_process
w.start()
File "/usr/local/lib/python3.6/site-packages/billiard/process.py", line 124, in start
self._popen = self._Popen(self)
File "/usr/local/lib/python3.6/site-packages/billiard/context.py", line 333, in _Popen
return Popen(process_obj)
File "/usr/local/lib/python3.6/site-packages/billiard/popen_fork.py", line 22, in __init__
sys.stderr.flush()
ValueError: I/O operation on closed file.
It looks like billiard is unable to flesh the stderr at the Popen object creation time. Any idea where this might come from ?
My packages versions:
Hello, we currently have a text analysis process running in docker. Part of it implies some text processing, which is run in a billiard Pool. However, it randomly raises the following exception:
It looks like billiard is unable to flesh the stderr at the Popen object creation time. Any idea where this might come from ? My packages versions:
Thanks for the help !