BrokenProcessPool Error on Windows 11 Due to Abrupt Child Process Termination

[pythonlover22]

Hello,

I have installed DeOldify on my Windows machine and followed the tutorial for ImageColorizer.ipynb without any issues. However, I am encountering an error while working through theColorizeTrainingStable.ipynb tutorial. Specifically, the error occurs when I attempt to run the code block for “Create black and white training images.”

I downloaded the dataset from Kaggle’s ImageNet Object Localization Challenge as per the instructions and made no changes to the folder structure. The paths in the Setup cell are also correctly configured. The issue arises when I run the following block of code:

if not path_lr.exists():
    il = ImageList.from_folder(path_hr)
    parallel(create_training_images, il.items)

Upon executing this cell, I receive the following error:

---------------------------------------------------------------------------
BrokenProcessPool                         Traceback (most recent call last)
Cell In[19], line 3
      1 if not path_lr.exists():
      2     il = ImageList.from_folder(path_hr)
----> 3     parallel(create_training_images, il.items)

File D:\python_projects\conda_projects\deoldify\downloads\DeOldify-master\fastai\core.py:352, in parallel(func, arr, max_workers, leave)
    350 else:
    351     with ProcessPoolExecutor(max_workers=max_workers) as ex:
--> 352         futures = [ex.submit(func,o,i) for i,o in enumerate(arr)]
    353         results = []
    354         for f in progress_bar(concurrent.futures.as_completed(futures), total=len(arr), leave=leave): 

File D:\python_projects\conda_projects\deoldify\downloads\DeOldify-master\fastai\core.py:352, in <listcomp>(.0)
    350 else:
    351     with ProcessPoolExecutor(max_workers=max_workers) as ex:
--> 352         futures = [ex.submit(func,o,i) for i,o in enumerate(arr)]
    353         results = []
    354         for f in progress_bar(concurrent.futures.as_completed(futures), total=len(arr), leave=leave): 

File D:\python_projects\conda_projects\envs\deoldify\lib\concurrent\futures\process.py:720, in ProcessPoolExecutor.submit(self, fn, *args, **kwargs)
    718 with self._shutdown_lock:
    719     if self._broken:
--> 720         raise BrokenProcessPool(self._broken)
    721     if self._shutdown_thread:
    722         raise RuntimeError('cannot schedule new futures after shutdown')

BrokenProcessPool: A child process terminated abruptly, the process pool is not usable anymore

I am running this on a Windows 11 workstation with a Quadro RTX 5000 (GPU 0). I followed the installation instructions carefully, successfully created the environment, and installed all required dependencies.

To provide further context, I have included the details of my cloned environment below:

# packages in environment at D:\python_projects\conda_projects\envs\deoldify:
#
# Name                    Version                   Build  Channel
annotated-types           0.7.0              pyhd8ed1ab_0    conda-forge
anyio                     4.4.0              pyhd8ed1ab_0    conda-forge
aom                       3.9.1                he0c23c2_0    conda-forge
argon2-cffi               23.1.0             pyhd8ed1ab_0    conda-forge
argon2-cffi-bindings      21.2.0          py310ha8f682b_5    conda-forge
arrow                     1.3.0              pyhd8ed1ab_0    conda-forge
asttokens                 2.4.1              pyhd8ed1ab_0    conda-forge
async-lru                 2.0.4              pyhd8ed1ab_0    conda-forge
attrs                     24.2.0             pyh71513ae_0    conda-forge
babel                     2.14.0             pyhd8ed1ab_0    conda-forge
beautifulsoup4            4.12.3             pyha770c72_0    conda-forge
blas                      2.121                       mkl    conda-forge
blas-devel                3.9.0              21_win64_mkl    conda-forge
bleach                    6.1.0              pyhd8ed1ab_0    conda-forge
bottleneck                1.4.0           py310hb0944cc_2    conda-forge
brotli                    1.1.0                h2466b09_2    conda-forge
brotli-bin                1.1.0                h2466b09_2    conda-forge
brotli-python             1.1.0           py310h9e98ed7_2    conda-forge
bzip2                     1.0.8                h2466b09_7    conda-forge
ca-certificates           2024.8.30            h56e8100_0    conda-forge
cached-property           1.5.2                hd8ed1ab_1    conda-forge
cached_property           1.5.2              pyha770c72_1    conda-forge
cairo                     1.18.0               h1fef639_0    conda-forge
catalogue                 2.0.10          py310h5588dad_0    conda-forge
certifi                   2024.8.30          pyhd8ed1ab_0    conda-forge
cffi                      1.17.1          py310ha8f682b_0    conda-forge
charset-normalizer        3.3.2              pyhd8ed1ab_0    conda-forge
click                     8.1.7           win_pyh7428d3b_0    conda-forge
cloudpathlib              0.19.0             pyhd8ed1ab_0    conda-forge
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
comm                      0.2.2              pyhd8ed1ab_0    conda-forge
confection                0.1.4           py310h3bcd3f7_0    conda-forge
contourpy                 1.3.0           py310hc19bc0b_1    conda-forge
cudatoolkit               11.5.2              h7b70026_13    conda-forge
cycler                    0.12.1             pyhd8ed1ab_0    conda-forge
cymem                     2.0.8           py310h00ffb61_1    conda-forge
cython-blis               0.7.10          py310h3e78b6c_2    conda-forge
dataclasses               0.8                pyhc8e2a94_3    conda-forge
dav1d                     1.2.1                hcfcfb64_0    conda-forge
debugpy                   1.8.5           py310h9e98ed7_1    conda-forge
decorator                 5.1.1              pyhd8ed1ab_0    conda-forge
defusedxml                0.7.1              pyhd8ed1ab_0    conda-forge
docker-pycreds            0.4.0                    pypi_0    pypi
entrypoints               0.4                pyhd8ed1ab_0    conda-forge
exceptiongroup            1.2.2              pyhd8ed1ab_0    conda-forge
executing                 2.1.0              pyhd8ed1ab_0    conda-forge
expat                     2.6.3                he0c23c2_0    conda-forge
fastai                    1.0.60                        1    fastai
fastprogress              1.0.3                      py_0    fastai
ffmpeg                    7.0.1           gpl_h7cec250_100    conda-forge
ffmpeg-python             0.2.0                    pypi_0    pypi
font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    conda-forge
font-ttf-inconsolata      3.000                h77eed37_0    conda-forge
font-ttf-source-code-pro  2.038                h77eed37_0    conda-forge
font-ttf-ubuntu           0.83                 h77eed37_2    conda-forge
fontconfig                2.14.2               hbde0cde_0    conda-forge
fonts-conda-ecosystem     1                             0    conda-forge
fonts-conda-forge         1                             0    conda-forge
fonttools                 4.53.1          py310ha8f682b_1    conda-forge
fqdn                      1.5.1              pyhd8ed1ab_0    conda-forge
freetype                  2.12.1               hdaf720e_2    conda-forge
future                    1.0.0                    pypi_0    pypi
gitdb                     4.0.11                   pypi_0    pypi
gitpython                 3.1.43                   pypi_0    pypi
graphite2                 1.3.13            h63175ca_1003    conda-forge
h11                       0.14.0             pyhd8ed1ab_0    conda-forge
h2                        4.1.0              pyhd8ed1ab_0    conda-forge
harfbuzz                  8.5.0                h81778c3_0    conda-forge
hpack                     4.0.0              pyh9f0ad1d_0    conda-forge
httpcore                  1.0.5              pyhd8ed1ab_0    conda-forge
httpx                     0.27.2             pyhd8ed1ab_0    conda-forge
hyperframe                6.0.1              pyhd8ed1ab_0    conda-forge
icu                       73.2                 h63175ca_0    conda-forge
idna                      3.10               pyhd8ed1ab_0    conda-forge
importlib-metadata        8.5.0              pyha770c72_0    conda-forge
importlib_metadata        8.5.0                hd8ed1ab_0    conda-forge
importlib_resources       6.4.5              pyhd8ed1ab_0    conda-forge
intel-openmp              2024.2.1          h57928b3_1083    conda-forge
ipykernel                 6.29.5             pyh4bbf305_0    conda-forge
ipython                   8.27.0             pyh7428d3b_0    conda-forge
ipywidgets                8.1.5              pyhd8ed1ab_0    conda-forge
isoduration               20.11.0            pyhd8ed1ab_0    conda-forge
jedi                      0.19.1             pyhd8ed1ab_0    conda-forge
jinja2                    3.1.4              pyhd8ed1ab_0    conda-forge
jpeg                      9e                   hcfcfb64_3    conda-forge
json5                     0.9.25             pyhd8ed1ab_0    conda-forge
jsonpointer               3.0.0           py310h5588dad_1    conda-forge
jsonschema                4.23.0             pyhd8ed1ab_0    conda-forge
jsonschema-specifications 2023.12.1          pyhd8ed1ab_0    conda-forge
jsonschema-with-format-nongpl 4.23.0               hd8ed1ab_0    conda-forge
jupyter-lsp               2.2.5              pyhd8ed1ab_0    conda-forge
jupyter_client            8.6.2              pyhd8ed1ab_0    conda-forge
jupyter_core              5.7.2           py310h5588dad_0    conda-forge
jupyter_events            0.10.0             pyhd8ed1ab_0    conda-forge
jupyter_server            2.14.2             pyhd8ed1ab_0    conda-forge
jupyter_server_terminals  0.5.3              pyhd8ed1ab_0    conda-forge
jupyterlab                4.2.5              pyhd8ed1ab_0    conda-forge
jupyterlab_pygments       0.3.0              pyhd8ed1ab_1    conda-forge
jupyterlab_server         2.27.3             pyhd8ed1ab_0    conda-forge
jupyterlab_widgets        3.0.13             pyhd8ed1ab_0    conda-forge
kiwisolver                1.4.7           py310hc19bc0b_0    conda-forge
krb5                      1.20.1               heb0366b_0    conda-forge
langcodes                 3.4.0              pyhd8ed1ab_0    conda-forge
language-data             1.2.0              pyhd8ed1ab_0    conda-forge
lcms2                     2.15                 ha5c8aab_0    conda-forge
lerc                      4.0.0                h63175ca_0    conda-forge
libabseil                 20240116.2      cxx17_he0c23c2_1    conda-forge
libblas                   3.9.0              21_win64_mkl    conda-forge
libbrotlicommon           1.1.0                h2466b09_2    conda-forge
libbrotlidec              1.1.0                h2466b09_2    conda-forge
libbrotlienc              1.1.0                h2466b09_2    conda-forge
libcblas                  3.9.0              21_win64_mkl    conda-forge
libclang                  14.0.6          default_h77d9078_1    conda-forge
libclang13                14.0.6          default_h77d9078_1    conda-forge
libdeflate                1.17                 hcfcfb64_0    conda-forge
libexpat                  2.6.3                he0c23c2_0    conda-forge
libffi                    3.4.2                h8ffe710_5    conda-forge
libglib                   2.80.2               h0df6a38_0    conda-forge
libhwloc                  2.11.1          default_h8125262_1000    conda-forge
libiconv                  1.17                 hcfcfb64_2    conda-forge
libintl                   0.22.5               h5728263_3    conda-forge
liblapack                 3.9.0              21_win64_mkl    conda-forge
liblapacke                3.9.0              21_win64_mkl    conda-forge
libopus                   1.3.1                h8ffe710_1    conda-forge
libpng                    1.6.43               h19919ed_0    conda-forge
libpq                     12.17                h906ac69_0    defaults
libprotobuf               4.25.3               h503648d_0    conda-forge
libsodium                 1.0.18               h8d14728_1    conda-forge
libsqlite                 3.46.1               h2466b09_0    conda-forge
libtiff                   4.5.0                hf8721a0_2    conda-forge
libuv                     1.48.0               hcfcfb64_0    conda-forge
libwebp-base              1.4.0                hcfcfb64_0    conda-forge
libxcb                    1.13              hcd874cb_1004    conda-forge
libxml2                   2.12.7               h283a6d9_1    conda-forge
libzlib                   1.2.13               h2466b09_6    conda-forge
m2w64-gcc-libgfortran     5.3.0                         6    conda-forge
m2w64-gcc-libs            5.3.0                         7    conda-forge
m2w64-gcc-libs-core       5.3.0                         7    conda-forge
m2w64-gmp                 6.1.0                         2    conda-forge
m2w64-libwinpthread-git   5.0.0.4634.697f757               2    conda-forge
marisa-trie               1.1.0           py310h00ffb61_1    conda-forge
markdown-it-py            3.0.0              pyhd8ed1ab_0    conda-forge
markupsafe                2.1.5           py310ha8f682b_1    conda-forge
matplotlib                3.9.1           py310h5588dad_1    conda-forge
matplotlib-base           3.9.1           py310h37e0a56_2    conda-forge
matplotlib-inline         0.1.7              pyhd8ed1ab_0    conda-forge
mdurl                     0.1.2              pyhd8ed1ab_0    conda-forge
mistune                   3.0.2              pyhd8ed1ab_0    conda-forge
mkl                       2024.0.0         h66d3029_49657    conda-forge
mkl-devel                 2024.0.0         h57928b3_49657    conda-forge
mkl-include               2024.0.0         h66d3029_49657    conda-forge
msys2-conda-epoch         20160418                      1    conda-forge
munkres                   1.1.4              pyh9f0ad1d_0    conda-forge
murmurhash                1.0.10          py310h00ffb61_1    conda-forge
mutagen                   1.47.0                   pypi_0    pypi
nbclient                  0.10.0             pyhd8ed1ab_0    conda-forge
nbconvert-core            7.16.4             pyhd8ed1ab_1    conda-forge
nbformat                  5.10.4             pyhd8ed1ab_0    conda-forge
nest-asyncio              1.6.0              pyhd8ed1ab_0    conda-forge
notebook-shim             0.2.4              pyhd8ed1ab_0    conda-forge
numexpr                   2.7.3           py310hf5e1058_2    conda-forge
numpy                     1.26.4          py310hf667824_0    conda-forge
nvidia-ml-py3             7.352.0                    py_0    fastai
opencv-python             4.10.0.84                pypi_0    pypi
openh264                  2.4.1                h63175ca_0    conda-forge
openjpeg                  2.5.0                ha2aaf27_2    conda-forge
openssl                   3.3.2                h2466b09_0    conda-forge
overrides                 7.7.0              pyhd8ed1ab_0    conda-forge
packaging                 24.1               pyhd8ed1ab_0    conda-forge
pandas                    2.2.2           py310hb4db72f_1    conda-forge
pandocfilters             1.5.0              pyhd8ed1ab_0    conda-forge
parso                     0.8.4              pyhd8ed1ab_0    conda-forge
pcre2                     10.43                h17e33f8_0    conda-forge
pickleshare               0.7.5                   py_1003    conda-forge
pillow                    9.4.0           py310hdbb7713_1    conda-forge
pip                       24.2               pyh8b19718_1    conda-forge
pixman                    0.43.4               h63175ca_0    conda-forge
pkgutil-resolve-name      1.3.10             pyhd8ed1ab_1    conda-forge
platformdirs              4.3.3              pyhd8ed1ab_0    conda-forge
preshed                   3.0.9           py310h00ffb61_1    conda-forge
prometheus_client         0.20.0             pyhd8ed1ab_0    conda-forge
prompt-toolkit            3.0.47             pyha770c72_0    conda-forge
protobuf                  4.25.3          py310hb55946c_1    conda-forge
psutil                    6.0.0           py310ha8f682b_1    conda-forge
pthread-stubs             0.4               hcd874cb_1001    conda-forge
pthreads-win32            2.9.1                hfa6e2cd_3    conda-forge
pure_eval                 0.2.3              pyhd8ed1ab_0    conda-forge
pycparser                 2.22               pyhd8ed1ab_0    conda-forge
pycryptodomex             3.20.0                   pypi_0    pypi
pydantic                  2.9.1              pyhd8ed1ab_0    conda-forge
pydantic-core             2.23.3          py310hc226416_0    conda-forge
pygments                  2.18.0             pyhd8ed1ab_0    conda-forge
pyparsing                 3.1.4              pyhd8ed1ab_0    conda-forge
pyqt                      5.15.4          py310h8a704f9_0    conda-forge
pyqt5-sip                 12.9.0          py310h8a704f9_0    conda-forge
pysocks                   1.7.1              pyh0701188_6    conda-forge
python                    3.10.14         h4de0772_0_cpython    conda-forge
python-dateutil           2.9.0              pyhd8ed1ab_0    conda-forge
python-fastjsonschema     2.20.0             pyhd8ed1ab_0    conda-forge
python-json-logger        2.0.7              pyhd8ed1ab_0    conda-forge
python-tzdata             2024.1             pyhd8ed1ab_0    conda-forge
python_abi                3.10                    5_cp310    conda-forge
pytorch                   1.11.0          py3.10_cuda11.5_cudnn8_0    pytorch
pytorch-mutex             1.0                        cuda    pytorch
pytz                      2024.2             pyhd8ed1ab_0    conda-forge
pywin32                   306             py310h00ffb61_2    conda-forge
pywinpty                  2.0.13          py310h9e98ed7_1    conda-forge
pyyaml                    6.0.2           py310ha8f682b_1    conda-forge
pyzmq                     26.2.0          py310h656833d_1    conda-forge
qhull                     2020.2               hc790b64_5    conda-forge
qt-main                   5.15.2              h19c9488_10    defaults
referencing               0.35.1             pyhd8ed1ab_0    conda-forge
requests                  2.32.3             pyhd8ed1ab_0    conda-forge
rfc3339-validator         0.1.4              pyhd8ed1ab_0    conda-forge
rfc3986-validator         0.1.1              pyh9f0ad1d_0    conda-forge
rich                      13.8.1             pyhd8ed1ab_0    conda-forge
rpds-py                   0.20.0          py310hc226416_1    conda-forge
scipy                     1.14.1          py310h46043a1_0    conda-forge
send2trash                1.8.3              pyh5737063_0    conda-forge
sentry-sdk                2.14.0                   pypi_0    pypi
setproctitle              1.3.3                    pypi_0    pypi
setuptools                73.0.1             pyhd8ed1ab_0    conda-forge
shellingham               1.5.4              pyhd8ed1ab_0    conda-forge
sip                       6.5.1           py310h8a704f9_2    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
smart-open                7.0.4                hd8ed1ab_0    conda-forge
smart_open                7.0.4              pyhd8ed1ab_0    conda-forge
smmap                     5.0.1                    pypi_0    pypi
sniffio                   1.3.1              pyhd8ed1ab_0    conda-forge
soupsieve                 2.5                pyhd8ed1ab_1    conda-forge
spacy                     3.7.5           py310h82564ac_1    conda-forge
spacy-legacy              3.0.12             pyhd8ed1ab_0    conda-forge
spacy-loggers             1.0.5              pyhd8ed1ab_0    conda-forge
sqlite                    3.46.1               h2466b09_0    conda-forge
srsly                     2.4.8           py310h00ffb61_1    conda-forge
stack_data                0.6.2              pyhd8ed1ab_0    conda-forge
svt-av1                   2.1.0                he0c23c2_0    conda-forge
tbb                       2021.13.0            hc790b64_0    conda-forge
tensorboardx              2.6.2.2            pyhd8ed1ab_0    conda-forge
terminado                 0.18.1             pyh5737063_0    conda-forge
thinc                     8.2.5           py310h82564ac_0    conda-forge
tinycss2                  1.3.0              pyhd8ed1ab_0    conda-forge
tk                        8.6.13               h5226925_1    conda-forge
toml                      0.10.2             pyhd8ed1ab_0    conda-forge
tomli                     2.0.1              pyhd8ed1ab_0    conda-forge
torchaudio                0.11.0              py310_cu115    pytorch
torchvision               0.12.0              py310_cu115    pytorch
tornado                   6.4.1           py310ha8f682b_1    conda-forge
tqdm                      4.66.5             pyhd8ed1ab_0    conda-forge
traitlets                 5.14.3             pyhd8ed1ab_0    conda-forge
typer                     0.12.5             pyhd8ed1ab_0    conda-forge
typer-slim                0.12.5             pyhd8ed1ab_0    conda-forge
typer-slim-standard       0.12.5               hd8ed1ab_0    conda-forge
types-python-dateutil     2.9.0.20240906     pyhd8ed1ab_0    conda-forge
typing-extensions         4.12.2               hd8ed1ab_0    conda-forge
typing_extensions         4.12.2             pyha770c72_0    conda-forge
typing_utils              0.1.0              pyhd8ed1ab_0    conda-forge
tzdata                    2024a                h8827d51_1    conda-forge
ucrt                      10.0.22621.0         h57928b3_0    conda-forge
unicodedata2              15.1.0          py310h8d17308_0    conda-forge
uri-template              1.3.0              pyhd8ed1ab_0    conda-forge
urllib3                   2.2.3              pyhd8ed1ab_0    conda-forge
vc                        14.3                h8a93ad2_21    conda-forge
vc14_runtime              14.40.33810         ha82c5b3_21    conda-forge
vs2015_runtime            14.40.33810         h3bf8584_21    conda-forge
wandb                     0.18.1                   pypi_0    pypi
wasabi                    1.1.2           py310h5588dad_1    conda-forge
wcwidth                   0.2.13             pyhd8ed1ab_0    conda-forge
weasel                    0.4.1              pyhd8ed1ab_1    conda-forge
webcolors                 24.8.0             pyhd8ed1ab_0    conda-forge
webencodings              0.5.1              pyhd8ed1ab_2    conda-forge
websocket-client          1.8.0              pyhd8ed1ab_0    conda-forge
websockets                13.0.1                   pypi_0    pypi
wheel                     0.44.0             pyhd8ed1ab_0    conda-forge
widgetsnbextension        4.0.13             pyhd8ed1ab_0    conda-forge
win_inet_pton             1.1.0              pyhd8ed1ab_6    conda-forge
winpty                    0.4.3                         4    conda-forge
wrapt                     1.16.0          py310ha8f682b_1    conda-forge
x264                      1!164.3095           h8ffe710_2    conda-forge
x265                      3.5                  h2d74725_3    conda-forge
xorg-libxau               1.0.11               hcd874cb_0    conda-forge
xorg-libxdmcp             1.1.3                hcd874cb_0    conda-forge
xz                        5.2.6                h8d14728_0    conda-forge
yaml                      0.2.5                h8ffe710_2    conda-forge
yt-dlp                    2024.8.6                 pypi_0    pypi
zeromq                    4.3.5                he0c23c2_3    conda-forge
zipp                      3.20.2             pyhd8ed1ab_0    conda-forge
zlib                      1.2.13               h2466b09_6    conda-forge
zstandard                 0.23.0          py310he5e10e1_1    conda-forge
zstd                      1.5.6                h0ea2cb4_0    conda-forge

Could you please provide guidance on how to resolve this issue?

Thank you for your assistance!

[snynisada]

这是来自QQ邮箱的假期自动回复邮件。   您好，我最近正在休假中，无法亲自回复您的邮件。我将在假期结束后，尽快给您回复。