Open bhavika opened 1 month ago
I'm noticing that with cuDNN 9, multiprocess installation seems to stall indefinitely as seen below:
multiprocess
STEP 1/4: FROM pytorch/pytorch:2.4.0-cuda11.8-cudnn9-devel STEP 2/4: RUN pip3 install multiprocess==0.70.9 Collecting multiprocess==0.70.9 Downloading multiprocess-0.70.9.tar.gz (1.6 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.6/1.6 MB 8.8 MB/s eta 0:00:00 Preparing metadata (setup.py): started
Here's a test Dockerfile to reproduce this issue with:
FROM pytorch/pytorch:2.4.0-cuda11.8-cudnn9-devel RUN pip3 install multiprocess==0.70.9 WORKDIR /app CMD ["/bin/bash"]
I'm running Ubuntu 22.04.4 LTS on the host.
This is from one of our CI jobs that install many dependencies, multiprocess stalls for over 30 minutes and never finishes:
2024-08-06 12:35:52 UTC | #15 [stage-0 6/12] RUN pip install --no-deps --no-cache-dir -r requirements.txt -- | -- | 2024-08-06 12:35:52 UTC | #15 0.523 Collecting accelerate==0.31.0 (from -r requirements.txt (line 3)) | 2024-08-06 12:35:53 UTC | #15 0.567 Downloading accelerate-0.31.0-py3-none-any.whl.metadata (19 kB) | 2024-08-06 12:35:53 UTC | #15 0.779 Collecting aiohttp==3.9.5 (from -r requirements.txt (line 8)) | 2024-08-06 12:35:53 UTC | #15 0.783 Downloading aiohttp-3.9.5-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (7.5 kB) | 2024-08-06 12:35:53 UTC | #15 0.792 Collecting aiosignal==1.3.1 (from -r requirements.txt (line 13)) | 2024-08-06 12:35:53 UTC | #15 0.795 Downloading aiosignal-1.3.1-py3-none-any.whl.metadata (4.0 kB) | 2024-08-06 12:35:53 UTC | #15 0.803 Collecting annotated-types==0.7.0 (from -r requirements.txt (line 15)) | 2024-08-06 12:35:53 UTC | #15 0.807 Downloading annotated_types-0.7.0-py3-none-any.whl.metadata (15 kB) | 2024-08-06 12:35:53 UTC | #15 0.877 Collecting apache-beam==2.53.0 (from -r requirements.txt (line 17)) | 2024-08-06 12:35:53 UTC | #15 0.881 Downloading apache_beam-2.53.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.6 kB) | 2024-08-06 12:35:53 UTC | #15 0.902 Collecting async-timeout==4.0.3 (from -r requirements.txt (line 19)) | 2024-08-06 12:35:53 UTC | #15 0.906 Downloading async_timeout-4.0.3-py3-none-any.whl.metadata (4.2 kB) | 2024-08-06 12:35:53 UTC | #15 0.919 Collecting attrs==23.2.0 (from -r requirements.txt (line 21)) | 2024-08-06 12:35:53 UTC | #15 0.923 Downloading attrs-23.2.0-py3-none-any.whl.metadata (9.5 kB) | 2024-08-06 12:35:53 UTC | #15 0.940 Collecting bitsandbytes==0.43.2 (from -r requirements.txt (line 26)) | 2024-08-06 12:35:53 UTC | #15 0.944 Downloading bitsandbytes-0.43.2-py3-none-manylinux_2_24_x86_64.whl.metadata (3.5 kB) | 2024-08-06 12:35:53 UTC | #15 0.952 Collecting bitsandbytes-cuda110==0.26.0.post2 (from -r requirements.txt (line 28)) | 2024-08-06 12:35:53 UTC | #15 0.956 Downloading bitsandbytes_cuda110-0.26.0.post2-py3-none-any.whl.metadata (6.3 kB) | 2024-08-06 12:35:53 UTC | #15 0.969 Collecting cachetools==5.4.0 (from -r requirements.txt (line 30)) | 2024-08-06 12:35:53 UTC | #15 0.973 Downloading cachetools-5.4.0-py3-none-any.whl.metadata (5.3 kB) | 2024-08-06 12:35:53 UTC | #15 0.975 Requirement already satisfied: certifi==2024.7.4 in /opt/conda/lib/python3.11/site-packages (from -r requirements.txt (line 34)) (2024.7.4) | 2024-08-06 12:35:53 UTC | #15 1.027 Collecting charset-normalizer==3.3.2 (from -r requirements.txt (line 36)) | 2024-08-06 12:35:53 UTC | #15 1.031 Downloading charset_normalizer-3.3.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (33 kB) | 2024-08-06 12:35:53 UTC | #15 1.035 Requirement already satisfied: click==8.1.7 in /opt/conda/lib/python3.11/site-packages (from -r requirements.txt (line 38)) (8.1.7) | 2024-08-06 12:35:53 UTC | #15 1.046 Collecting cloudpickle==2.2.1 (from -r requirements.txt (line 40)) | 2024-08-06 12:35:53 UTC | #15 1.049 Downloading cloudpickle-2.2.1-py3-none-any.whl.metadata (6.9 kB) | 2024-08-06 12:35:53 UTC | #15 1.056 Collecting crcmod==1.7 (from -r requirements.txt (line 42)) | 2024-08-06 12:35:53 UTC | #15 1.060 Downloading crcmod-1.7.tar.gz (89 kB) | 2024-08-06 12:35:53 UTC | #15 1.075 Preparing metadata (setup.py): started | 2024-08-06 12:35:53 UTC | #15 1.250 Preparing metadata (setup.py): finished with status 'done' | 2024-08-06 12:35:53 UTC | #15 1.271 Collecting datasets==2.19.2 (from -r requirements.txt (line 44)) | 2024-08-06 12:35:53 UTC | #15 1.276 Downloading datasets-2.19.2-py3-none-any.whl.metadata (19 kB) | 2024-08-06 12:35:53 UTC | #15 1.299 Collecting db-dtypes==1.2.0 (from -r requirements.txt (line 49)) | 2024-08-06 12:35:53 UTC | #15 1.303 Downloading db_dtypes-1.2.0-py2.py3-none-any.whl.metadata (3.0 kB) | 2024-08-06 12:35:53 UTC | #15 1.305 Requirement already satisfied: decorator==5.1.1 in /opt/conda/lib/python3.11/site-packages (from -r requirements.txt (line 51)) (5.1.1) | 2024-08-06 12:35:53 UTC | #15 1.313 Collecting dill==0.3.1.1 (from -r requirements.txt (line 53)) | 2024-08-06 12:35:53 UTC | #15 1.321 Downloading dill-0.3.1.1.tar.gz (151 kB) | 2024-08-06 12:35:53 UTC | #15 1.335 Preparing metadata (setup.py): started | 2024-08-06 12:35:53 UTC | #15 1.512 Preparing metadata (setup.py): finished with status 'done' | 2024-08-06 12:35:54 UTC | #15 1.515 Requirement already satisfied: dnspython==2.6.1 in /opt/conda/lib/python3.11/site-packages (from -r requirements.txt (line 59)) (2.6.1) | 2024-08-06 12:35:54 UTC | #15 1.520 Collecting docopt==0.6.2 (from -r requirements.txt (line 61)) | 2024-08-06 12:35:54 UTC | #15 1.524 Downloading docopt-0.6.2.tar.gz (25 kB) | 2024-08-06 12:35:54 UTC | #15 1.530 Preparing metadata (setup.py): started | 2024-08-06 12:35:54 UTC | #15 1.698 Preparing metadata (setup.py): finished with status 'done' | 2024-08-06 12:35:54 UTC | #15 1.707 Collecting docstring-parser==0.16 (from -r requirements.txt (line 63)) | 2024-08-06 12:35:54 UTC | #15 1.710 Downloading docstring_parser-0.16-py3-none-any.whl.metadata (3.0 kB) | 2024-08-06 12:35:54 UTC | #15 1.720 Collecting einops==0.8.0 (from -r requirements.txt (line 67)) | 2024-08-06 12:35:54 UTC | #15 1.724 Downloading einops-0.8.0-py3-none-any.whl.metadata (12 kB) | 2024-08-06 12:35:54 UTC | #15 1.734 Collecting evaluate==0.4.2 (from -r requirements.txt (line 69)) | 2024-08-06 12:35:54 UTC | #15 1.739 Downloading evaluate-0.4.2-py3-none-any.whl.metadata (9.3 kB) | 2024-08-06 12:35:54 UTC | #15 1.894 Collecting fastavro==1.9.5 (from -r requirements.txt (line 71)) | 2024-08-06 12:35:54 UTC | #15 1.898 Downloading fastavro-1.9.5-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (5.5 kB) | 2024-08-06 12:35:54 UTC | #15 1.908 Collecting fasteners==0.19 (from -r requirements.txt (line 73)) | 2024-08-06 12:35:54 UTC | #15 1.911 Downloading fasteners-0.19-py3-none-any.whl.metadata (4.9 kB) | 2024-08-06 12:35:54 UTC | #15 1.929 Collecting filelock==3.15.4 (from -r requirements.txt (line 77)) | 2024-08-06 12:35:54 UTC | #15 1.932 Downloading filelock-3.15.4-py3-none-any.whl.metadata (2.9 kB) | 2024-08-06 12:35:54 UTC | #15 1.947 Collecting flash-attn==2.3.6 (from -r requirements.txt (line 84)) | 2024-08-06 12:35:54 UTC | #15 1.951 Downloading flash_attn-2.3.6.tar.gz (2.3 MB) | 2024-08-06 12:35:54 UTC | #15 1.969 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.3/2.3 MB 139.1 MB/s eta 0:00:00 | 2024-08-06 12:35:54 UTC | #15 2.422 Preparing metadata (setup.py): started | 2024-08-06 12:35:57 UTC | #15 4.663 Preparing metadata (setup.py): finished with status 'done' | 2024-08-06 12:35:57 UTC | #15 4.701 Collecting frozenlist==1.4.1 (from -r requirements.txt (line 86)) | 2024-08-06 12:35:57 UTC | #15 4.705 Downloading frozenlist-1.4.1-cp311-cp311-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (12 kB) | 2024-08-06 12:35:57 UTC | #15 4.726 Collecting fsspec==2023.10.0 (from -r requirements.txt (line 90)) | 2024-08-06 12:35:57 UTC | #15 4.730 Downloading fsspec-2023.10.0-py3-none-any.whl.metadata (6.8 kB) | 2024-08-06 12:35:57 UTC | #15 4.754 Collecting gcsfs==2023.10.0 (from -r requirements.txt (line 99)) | 2024-08-06 12:35:57 UTC | #15 4.757 Downloading gcsfs-2023.10.0-py2.py3-none-any.whl.metadata (1.6 kB) | 2024-08-06 12:35:57 UTC | #15 4.800 Collecting google-api-core==2.19.1 (from -r requirements.txt (line 101)) | 2024-08-06 12:35:57 UTC | #15 4.804 Downloading google_api_core-2.19.1-py3-none-any.whl.metadata (2.7 kB) | 2024-08-06 12:35:57 UTC | #15 4.819 Collecting google-apitools==0.5.31 (from -r requirements.txt (line 121)) | 2024-08-06 12:35:57 UTC | #15 4.823 Downloading google-apitools-0.5.31.tar.gz (173 kB) | 2024-08-06 12:35:57 UTC | #15 4.844 Preparing metadata (setup.py): started | 2024-08-06 12:35:57 UTC | #15 5.016 Preparing metadata (setup.py): finished with status 'done' | 2024-08-06 12:35:57 UTC | #15 5.074 Collecting google-auth==2.32.0 (from -r requirements.txt (line 123)) | 2024-08-06 12:35:57 UTC | #15 5.078 Downloading google_auth-2.32.0-py2.py3-none-any.whl.metadata (4.7 kB) | 2024-08-06 12:35:57 UTC | #15 5.088 Collecting google-auth-httplib2==0.1.1 (from -r requirements.txt (line 145)) | 2024-08-06 12:35:57 UTC | #15 5.091 Downloading google_auth_httplib2-0.1.1-py2.py3-none-any.whl.metadata (2.1 kB) | 2024-08-06 12:35:57 UTC | #15 5.102 Collecting google-auth-oauthlib==1.2.1 (from -r requirements.txt (line 147)) | 2024-08-06 12:35:57 UTC | #15 5.106 Downloading google_auth_oauthlib-1.2.1-py2.py3-none-any.whl.metadata (2.7 kB) | 2024-08-06 12:35:57 UTC | #15 5.135 Collecting google-cloud-aiplatform==1.52.0 (from -r requirements.txt (line 152)) | 2024-08-06 12:35:57 UTC | #15 5.141 Downloading google_cloud_aiplatform-1.52.0-py2.py3-none-any.whl.metadata (30 kB) | 2024-08-06 12:35:57 UTC | #15 5.222 Collecting google-cloud-bigquery==3.25.0 (from -r requirements.txt (line 156)) | 2024-08-06 12:35:57 UTC | #15 5.226 Downloading google_cloud_bigquery-3.25.0-py2.py3-none-any.whl.metadata (8.9 kB) | 2024-08-06 12:35:57 UTC | #15 5.252 Collecting google-cloud-bigquery-storage==2.25.0 (from -r requirements.txt (line 161)) | 2024-08-06 12:35:57 UTC | #15 5.256 Downloading google_cloud_bigquery_storage-2.25.0-py2.py3-none-any.whl.metadata (5.6 kB) | 2024-08-06 12:35:57 UTC | #15 5.281 Collecting google-cloud-bigtable==2.25.0 (from -r requirements.txt (line 163)) | 2024-08-06 12:35:57 UTC | #15 5.286 Downloading google_cloud_bigtable-2.25.0-py2.py3-none-any.whl.metadata (5.2 kB) | 2024-08-06 12:35:57 UTC | #15 5.307 Collecting google-cloud-core==2.4.1 (from -r requirements.txt (line 165)) | 2024-08-06 12:35:57 UTC | #15 5.310 Downloading google_cloud_core-2.4.1-py2.py3-none-any.whl.metadata (2.7 kB) | 2024-08-06 12:35:57 UTC | #15 5.337 Collecting google-cloud-datastore==2.19.0 (from -r requirements.txt (line 173)) | 2024-08-06 12:35:57 UTC | #15 5.342 Downloading google_cloud_datastore-2.19.0-py2.py3-none-any.whl.metadata (5.6 kB) | 2024-08-06 12:35:57 UTC | #15 5.364 Collecting google-cloud-dlp==3.20.0 (from -r requirements.txt (line 175)) | 2024-08-06 12:35:57 UTC | #15 5.369 Downloading google_cloud_dlp-3.20.0-py2.py3-none-any.whl.metadata (5.3 kB) | 2024-08-06 12:35:57 UTC | #15 5.387 Collecting google-cloud-language==2.14.0 (from -r requirements.txt (line 177)) | 2024-08-06 12:35:57 UTC | #15 5.394 Downloading google_cloud_language-2.14.0-py2.py3-none-any.whl.metadata (5.3 kB) | 2024-08-06 12:35:57 UTC | #15 5.429 Collecting google-cloud-pubsub==2.23.0 (from -r requirements.txt (line 179)) | 2024-08-06 12:35:57 UTC | #15 5.434 Downloading google_cloud_pubsub-2.23.0-py2.py3-none-any.whl.metadata (9.3 kB) | 2024-08-06 12:35:57 UTC | #15 5.450 Collecting google-cloud-pubsublite==1.11.1 (from -r requirements.txt (line 183)) | 2024-08-06 12:35:57 UTC | #15 5.457 Downloading google_cloud_pubsublite-1.11.1-py2.py3-none-any.whl.metadata (5.6 kB) | 2024-08-06 12:35:57 UTC | #15 5.473 Collecting google-cloud-recommendations-ai==0.10.12 (from -r requirements.txt (line 185)) | 2024-08-06 12:35:58 UTC | #15 5.477 Downloading google_cloud_recommendations_ai-0.10.12-py2.py3-none-any.whl.metadata (5.3 kB) | 2024-08-06 12:35:58 UTC | #15 5.498 Collecting google-cloud-resource-manager==1.12.4 (from -r requirements.txt (line 187)) | 2024-08-06 12:35:58 UTC | #15 5.502 Downloading google_cloud_resource_manager-1.12.4-py2.py3-none-any.whl.metadata (5.3 kB) | 2024-08-06 12:35:58 UTC | #15 5.531 Collecting google-cloud-spanner==3.48.0 (from -r requirements.txt (line 189)) | 2024-08-06 12:35:58 UTC | #15 5.536 Downloading google_cloud_spanner-3.48.0-py2.py3-none-any.whl.metadata (10 kB) | 2024-08-06 12:35:58 UTC | #15 5.574 Collecting google-cloud-storage==2.18.0 (from -r requirements.txt (line 191)) | 2024-08-06 12:35:58 UTC | #15 5.577 Downloading google_cloud_storage-2.18.0-py2.py3-none-any.whl.metadata (9.1 kB) | 2024-08-06 12:35:58 UTC | #15 5.650 Collecting google-cloud-videointelligence==2.13.5 (from -r requirements.txt (line 196)) | 2024-08-06 12:35:58 UTC | #15 5.655 Downloading google_cloud_videointelligence-2.13.5-py2.py3-none-any.whl.metadata (5.7 kB) | 2024-08-06 12:35:58 UTC | #15 5.679 Collecting google-cloud-vision==3.7.4 (from -r requirements.txt (line 198)) | 2024-08-06 12:35:58 UTC | #15 5.684 Downloading google_cloud_vision-3.7.4-py2.py3-none-any.whl.metadata (5.2 kB) | 2024-08-06 12:35:58 UTC | #15 5.710 Collecting google-crc32c==1.5.0 (from -r requirements.txt (line 200)) | 2024-08-06 12:35:58 UTC | #15 5.714 Downloading google_crc32c-1.5.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.3 kB) | 2024-08-06 12:35:58 UTC | #15 5.739 Collecting google-resumable-media==2.7.1 (from -r requirements.txt (line 204)) | 2024-08-06 12:35:58 UTC | #15 5.743 Downloading google_resumable_media-2.7.1-py2.py3-none-any.whl.metadata (2.2 kB) | 2024-08-06 12:35:58 UTC | #15 5.760 Collecting googleapis-common-protos==1.63.2 (from -r requirements.txt (line 208)) | 2024-08-06 12:35:58 UTC | #15 5.764 Downloading googleapis_common_protos-1.63.2-py2.py3-none-any.whl.metadata (1.5 kB) | 2024-08-06 12:35:58 UTC | #15 5.772 Collecting grpc-google-iam-v1==0.13.1 (from -r requirements.txt (line 213)) | 2024-08-06 12:35:58 UTC | #15 5.776 Downloading grpc_google_iam_v1-0.13.1-py2.py3-none-any.whl.metadata (3.3 kB) | 2024-08-06 12:35:58 UTC | #15 5.790 Collecting grpc-interceptor==0.15.4 (from -r requirements.txt (line 219)) | 2024-08-06 12:35:58 UTC | #15 5.794 Downloading grpc_interceptor-0.15.4-py3-none-any.whl.metadata (8.4 kB) | 2024-08-06 12:35:58 UTC | #15 6.269 Collecting grpcio==1.65.1 (from -r requirements.txt (line 221)) | 2024-08-06 12:35:58 UTC | #15 6.274 Downloading grpcio-1.65.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.3 kB) | 2024-08-06 12:35:58 UTC | #15 6.302 Collecting grpcio-status==1.62.2 (from -r requirements.txt (line 231)) | 2024-08-06 12:35:58 UTC | #15 6.306 Downloading grpcio_status-1.62.2-py3-none-any.whl.metadata (1.3 kB) | 2024-08-06 12:35:58 UTC | #15 6.317 Collecting hdfs==2.7.3 (from -r requirements.txt (line 236)) | 2024-08-06 12:35:58 UTC | #15 6.321 Downloading hdfs-2.7.3.tar.gz (43 kB) | 2024-08-06 12:35:58 UTC | #15 6.330 Preparing metadata (setup.py): started | 2024-08-06 12:35:58 UTC | #15 6.497 Preparing metadata (setup.py): finished with status 'done' | 2024-08-06 12:35:59 UTC | #15 6.509 Collecting httplib2==0.22.0 (from -r requirements.txt (line 238)) | 2024-08-06 12:35:59 UTC | #15 6.513 Downloading httplib2-0.22.0-py3-none-any.whl.metadata (2.6 kB) | 2024-08-06 12:35:59 UTC | #15 6.544 Collecting huggingface-hub==0.22.2 (from -r requirements.txt (line 244)) | 2024-08-06 12:35:59 UTC | #15 6.548 Downloading huggingface_hub-0.22.2-py3-none-any.whl.metadata (12 kB) | 2024-08-06 12:35:59 UTC | #15 6.559 Requirement already satisfied: idna==3.7 in /opt/conda/lib/python3.11/site-packages (from -r requirements.txt (line 253)) (3.7) | 2024-08-06 12:35:59 UTC | #15 6.559 Requirement already satisfied: jinja2==3.1.4 in /opt/conda/lib/python3.11/site-packages (from -r requirements.txt (line 257)) (3.1.4) | 2024-08-06 12:35:59 UTC | #15 6.576 Collecting joblib==1.4.2 (from -r requirements.txt (line 259)) | 2024-08-06 12:35:59 UTC | #15 6.580 Downloading joblib-1.4.2-py3-none-any.whl.metadata (5.4 kB) | 2024-08-06 12:35:59 UTC | #15 6.589 Collecting js2py==0.74 (from -r requirements.txt (line 261)) | 2024-08-06 12:35:59 UTC | #15 6.593 Downloading Js2Py-0.74-py3-none-any.whl.metadata (868 bytes) | 2024-08-06 12:35:59 UTC | #15 6.604 Collecting jsonpickle==3.2.2 (from -r requirements.txt (line 263)) | 2024-08-06 12:35:59 UTC | #15 6.607 Downloading jsonpickle-3.2.2-py3-none-any.whl.metadata (7.2 kB) | 2024-08-06 12:35:59 UTC | #15 6.634 Collecting jsonschema==4.23.0 (from -r requirements.txt (line 265)) | 2024-08-06 12:35:59 UTC | #15 6.638 Downloading jsonschema-4.23.0-py3-none-any.whl.metadata (7.9 kB) | 2024-08-06 12:35:59 UTC | #15 6.650 Collecting jsonschema-specifications==2023.12.1 (from -r requirements.txt (line 267)) | 2024-08-06 12:35:59 UTC | #15 6.654 Downloading jsonschema_specifications-2023.12.1-py3-none-any.whl.metadata (3.0 kB) | 2024-08-06 12:35:59 UTC | #15 6.676 Collecting lightning==2.3.0 (from -r requirements.txt (line 269)) | 2024-08-06 12:35:59 UTC | #15 6.680 Downloading lightning-2.3.0-py3-none-any.whl.metadata (53 kB) | 2024-08-06 12:35:59 UTC | #15 6.783 Collecting lightning-utilities==0.11.6 (from -r requirements.txt (line 271)) | 2024-08-06 12:35:59 UTC | #15 6.786 Downloading lightning_utilities-0.11.6-py3-none-any.whl.metadata (5.2 kB) | 2024-08-06 12:35:59 UTC | #15 6.802 Collecting markdown-it-py==3.0.0 (from -r requirements.txt (line 276)) | 2024-08-06 12:35:59 UTC | #15 6.806 Downloading markdown_it_py-3.0.0-py3-none-any.whl.metadata (6.9 kB) | 2024-08-06 12:35:59 UTC | #15 6.842 Collecting markupsafe==2.1.5 (from -r requirements.txt (line 278)) | 2024-08-06 12:35:59 UTC | #15 6.845 Downloading MarkupSafe-2.1.5-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.0 kB) | 2024-08-06 12:35:59 UTC | #15 6.852 Collecting mdurl==0.1.2 (from -r requirements.txt (line 280)) | 2024-08-06 12:35:59 UTC | #15 6.856 Downloading mdurl-0.1.2-py3-none-any.whl.metadata (1.6 kB) | 2024-08-06 12:35:59 UTC | #15 6.857 Requirement already satisfied: mpmath==1.3.0 in /opt/conda/lib/python3.11/site-packages (from -r requirements.txt (line 282)) (1.3.0) | 2024-08-06 12:35:59 UTC | #15 6.984 Collecting multidict==6.0.5 (from -r requirements.txt (line 284)) | 2024-08-06 12:35:59 UTC | #15 7.028 Downloading multidict-6.0.5-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.2 kB) | 2024-08-06 12:35:59 UTC | #15 7.043 Collecting multiprocess==0.70.9 (from -r requirements.txt (line 288)) | 2024-08-06 12:35:59 UTC | #15 7.047 Downloading multiprocess-0.70.9.tar.gz (1.6 MB) | 2024-08-06 12:35:59 UTC | #15 7.055 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.6/1.6 MB 246.0 MB/s eta 0:00:00 | 2024-08-06 12:35:59 UTC | #15 7.282 Preparing metadata (setup.py): started
I'm noticing that with cuDNN 9,
multiprocessinstallation seems to stall indefinitely as seen below:Here's a test Dockerfile to reproduce this issue with:
I'm running Ubuntu 22.04.4 LTS on the host.
This is from one of our CI jobs that install many dependencies,
multiprocessstalls for over 30 minutes and never finishes: