bootphon / phonemizer

Simple text to phones converter for multiple languages
https://bootphon.github.io/phonemizer/
GNU General Public License v3.0
1.19k stars 166 forks source link

Multiprocessing unable to raise warnings #50

Closed cfrancesco closed 4 years ago

cfrancesco commented 4 years ago

Hi, when running the phonemization with multiple jobs as in

phonemize(some_text_list,
                             language='de',
                             backend='espeak',
                             strip=False,
                             preserve_punctuation=True,
                             with_stress=False,
                             njobs=16,
                             language_switch='keep-flags')

this error is raised:

---------------------------------------------------------------------------
_RemoteTraceback                          Traceback (most recent call last)
_RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/Users/fcardina/anaconda3/envs/ttsTF/lib/python3.6/site-packages/joblib/externals/loky/process_executor.py", line 418, in _process_worker
    r = call_item()
  File "/Users/fcardina/anaconda3/envs/ttsTF/lib/python3.6/site-packages/joblib/externals/loky/process_executor.py", line 272, in __call__
    return self.fn(*self.args, **self.kwargs)
  File "/Users/fcardina/anaconda3/envs/ttsTF/lib/python3.6/site-packages/joblib/_parallel_backends.py", line 600, in __call__
    return self.func(*args, **kwargs)
  File "/Users/fcardina/anaconda3/envs/ttsTF/lib/python3.6/site-packages/joblib/parallel.py", line 256, in __call__
    for func, args, kwargs in self.items]
  File "/Users/fcardina/anaconda3/envs/ttsTF/lib/python3.6/site-packages/joblib/parallel.py", line 256, in <listcomp>
    for func, args, kwargs in self.items]
  File "/Users/fcardina/anaconda3/envs/ttsTF/lib/python3.6/site-packages/phonemizer/backend/espeak.py", line 167, in _phonemize_aux
    self._warn_on_lang_switch()
  File "/Users/fcardina/anaconda3/envs/ttsTF/lib/python3.6/site-packages/phonemizer/backend/espeak.py", line 205, in _warn_on_lang_switch
    self.logger.warning(
AttributeError: 'NoneType' object has no attribute 'warning'
"""

The above exception was the direct cause of the following exception:

AttributeError                            Traceback (most recent call last)
<ipython-input-50-a02c66d832c7> in <module>
      8                              with_stress=False,
      9                              njobs=16,
---> 10                              language_switch='keep-flags')

~/anaconda3/envs/ttsTF/lib/python3.6/site-packages/phonemizer/phonemize.py in phonemize(text, language, backend, separator, strip, preserve_punctuation, punctuation_marks, with_stress, language_switch, njobs, logger)
    172     # phonemize the input text
    173     return phonemizer.phonemize(
--> 174         text, separator=separator, strip=strip, njobs=njobs)

~/anaconda3/envs/ttsTF/lib/python3.6/site-packages/phonemizer/backend/base.py in phonemize(self, text, separator, strip, njobs)
    116             text = joblib.Parallel(n_jobs=njobs)(
    117                 joblib.delayed(self._phonemize_aux)(t, separator, strip)
--> 118                 for t in chunks(text, njobs))
    119 
    120             # flatten them in a single list

~/anaconda3/envs/ttsTF/lib/python3.6/site-packages/joblib/parallel.py in __call__(self, iterable)
   1014 
   1015             with self._backend.retrieval_context():
-> 1016                 self.retrieve()
   1017             # Make sure that we get a last message telling us we are done
   1018             elapsed_time = time.time() - self._start_time

~/anaconda3/envs/ttsTF/lib/python3.6/site-packages/joblib/parallel.py in retrieve(self)
    906             try:
    907                 if getattr(self._backend, 'supports_timeout', False):
--> 908                     self._output.extend(job.get(timeout=self.timeout))
    909                 else:
    910                     self._output.extend(job.get())

~/anaconda3/envs/ttsTF/lib/python3.6/site-packages/joblib/_parallel_backends.py in wrap_future_result(future, timeout)
    552         AsyncResults.get from multiprocessing."""
    553         try:
--> 554             return future.result(timeout=timeout)
    555         except LokyTimeoutError:
    556             raise TimeoutError()

~/anaconda3/envs/ttsTF/lib/python3.6/concurrent/futures/_base.py in result(self, timeout)
    430                 raise CancelledError()
    431             elif self._state == FINISHED:
--> 432                 return self.__get_result()
    433             else:
    434                 raise TimeoutError()

~/anaconda3/envs/ttsTF/lib/python3.6/concurrent/futures/_base.py in __get_result(self)
    382     def __get_result(self):
    383         if self._exception:
--> 384             raise self._exception
    385         else:
    386             return self._result

AttributeError: 'NoneType' object has no attribute 'warning'

Setting the number of jobs to 1 does not raise the same error.

mmmaat commented 4 years ago

ok I'm replicating your bug, working on a fix!

mmmaat commented 4 years ago

Should be fixed, thanks for reporting.