search

nipreps / fmriprep

fMRIPrep is a robust and easy-to-use pipeline for preprocessing of diverse fMRI data. The transparent workflow dispenses of manual intervention, thereby ensuring the reproducibility of the results.
https://fmriprep.org
Apache License 2.0
621 stars 290 forks source link

[Errno -3] Temporary failure in name resolution #3322

Closed WangYunHong98 closed 1 month ago

WangYunHong98 commented 1 month ago

What happened?

Hi @effigies @oesteban

I used fmriprep-24.0.0 to preprocess my data, but an issue occurred.

It seems like templateflow did not work well. Please see the details below.

Best, Yunhong

What command did you use?

export SINGULARITYENV_TEMPLATEFLOW_HOME="/templateflow"

subject=$( sed -n -E "$((${SLURM_ARRAY_TASK_ID} + 1))s/sub-(\S*)\>.*/\1/gp" ${BIDS_DIR}/participants.tsv )
age=$( sed -n -E "$((${SLURM_ARRAY_TASK_ID} + 1))s/sub-(\S*)\s+(\S*)\s+.*/\2/gp" ${BIDS_DIR}/participants.tsv )

echo "Running: $subject"
echo "Suject age: $age"

if (( $(echo "$age >= 7.5 && $age <= 13.5" | bc -l) )); then
  template="MNIPediatricAsym:cohort-4:res-1"
elif (( $(echo "$age >= 13 && $age <= 18.5" | bc -l) )); then
  template="MNIPediatricAsym:cohort-6:res-1"
elif (( $(echo "$age >= 4.5 && $age <= 8.5" | bc -l) )); then
  template="MNIPediatricAsym:cohort-2:res-1"
else
  template="Age is not in the range, you need choose another template"
fi
echo "Template is $template"

# bind container with host
cmd="singularity run \
        --cleanenv \
        -B $PROJECT:/rootdir \
        -B $BIDS_DIR:/data \
        -B $DERIVS_DIR:/output \
        -B $WORK_DIR:/work \
        -B ${TEMPLATEFLOW_HOST_HOME}:${SINGULARITYENV_TEMPLATEFLOW_HOME} \
        -B ${SINGULARITYENV_FS_LICENSE}:/freesurfer_license \
    ${FMRIPREP} /data /output \
    participant --participant_label $subject \
    -w /work \
    -vv \
    --fs-license-file /freesurfer_license \
    --fs-subjects-dir /output/freesurfer \
    --mem-mb 84000 \
    --n-cpus 14 \
    --write-graph \
    --output-spaces $template anat fsnative func \
    --return-all-components \
    "
echo "Running task: $SLURM_ARRAY_TASK_ID"
echo "Command: $cmd"
eval "$cmd"

What version of fMRIPrep are you running?

24.0.0

How are you running fMRIPrep?

Singularity

Is your data BIDS valid?

Yes

Are you reusing any previously computed results?

No

Please copy and paste any relevant log output.

(node:46029) Warning: Closing directory handle on garbage collection
(Use `node --trace-warnings ...` to show where the warning was created)
Downloading https://templateflow.s3.amazonaws.com/tpl-MNIPediatricAsym/cohort-4/tpl-MNIPediatricAsym_cohort-4_res-1_T1w.nii.gz
Process Process-2:
Traceback (most recent call last):
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/urllib3/connection.py", line 196, in _new_conn
    sock = connection.create_connection(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/urllib3/util/connection.py", line 60, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/socket.py", line 962, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
socket.gaierror: [Errno -3] Temporary failure in name resolution

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/urllib3/connectionpool.py", line 789, in urlopen
    response = self._make_request(
               ^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/urllib3/connectionpool.py", line 490, in _make_request
    raise new_e
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/urllib3/connectionpool.py", line 466, in _make_request
    self._validate_conn(conn)
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/urllib3/connectionpool.py", line 1095, in _validate_conn
    conn.connect()
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/urllib3/connection.py", line 615, in connect
    self.sock = sock = self._new_conn()
                       ^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/urllib3/connection.py", line 203, in _new_conn
    raise NameResolutionError(self.host, self, e) from e
urllib3.exceptions.NameResolutionError: <urllib3.connection.HTTPSConnection object at 0x7fcb06482610>: Failed to resolve 'templateflow.s3.amazonaws.com' ([Errno -3] Temporary failure in name resolution)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/requests/adapters.py", line 667, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/urllib3/connectionpool.py", line 843, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/urllib3/util/retry.py", line 519, in increment
    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='templateflow.s3.amazonaws.com', port=443): Max retries exceeded with url: /tpl-MNIPediatricAsym/cohort-4/tpl-MNIPediatricAsym_cohort-4_res-1_T1w.nii.gz (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x7fcb06482610>: Failed to resolve 'templateflow.s3.amazonaws.com' ([Errno -3] Temporary failure in name resolution)"))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/conda/envs/fmriprep/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/opt/conda/envs/fmriprep/lib/python3.11/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/fmriprep/cli/workflow.py", line 126, in build_workflow
    retval['workflow'] = init_fmriprep_wf()
                         ^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/fmriprep/workflows/base.py", line 94, in init_fmriprep_wf
    single_subject_wf = init_single_subject_wf(subject_id)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/fmriprep/workflows/base.py", line 317, in init_single_subject_wf
    anat_fit_wf = init_anat_fit_wf(
                  ^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/smriprep/workflows/anatomical.py", line 731, in init_anat_fit_wf
    anat_reports_wf = init_anat_reports_wf(
                      ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/smriprep/workflows/outputs.py", line 137, in init_anat_reports_wf
    template_iterator_wf = init_template_iterator_wf(spaces=spaces, sloppy=sloppy)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/smriprep/workflows/outputs.py", line 1131, in init_template_iterator_wf
    fetch_template_files(template, specs=None, sloppy=sloppy)
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/smriprep/interfaces/templateflow.py", line 205, in fetch_template_files
    files['t1w'] = tf.get(name[0], desc=None, suffix='T1w', **specs)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/templateflow/conf/__init__.py", line 72, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/templateflow/api.py", line 174, in get
    _s3_get(filepath)
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/templateflow/api.py", line 329, in _s3_get
    r = requests.get(url, stream=True, timeout=TF_GET_TIMEOUT)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/requests/api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/requests/adapters.py", line 700, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='templateflow.s3.amazonaws.com', port=443): Max retries exceeded with url: /tpl-MNIPediatricAsym/cohort-4/tpl-MNIPediatricAsym_cohort-4_res-1_T1w.nii.gz (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x7fcb06482610>: Failed to resolve 'templateflow.s3.amazonaws.com' ([Errno -3] Temporary failure in name resolution)"))

Additional information / screenshots

No response

effigies commented 1 month ago

This appears to be an issue with hitting the internet from your cluster.

https://www.nipreps.org/apps/singularity/#restricted-internet-access

WangYunHong98 commented 1 month ago

Thank you very much @effigies

I checked the cluster's internet and found that the internet is not accessible.