search

nipreps / fmriprep

fMRIPrep is a robust and easy-to-use pipeline for preprocessing of diverse fMRI data. The transparent workflow dispenses of manual intervention, thereby ensuring the reproducibility of the results.
https://fmriprep.org
Apache License 2.0
629 stars 291 forks source link

singularity build 1.3.1 fails immediately #1534

Closed oricon closed 5 years ago

oricon commented 5 years ago

Looks like there are some issues with the 1.3.1 build in singularity. I get a TemplateFlow warning and a bunch of DeprecationWarnings as soon as I submit. The standard output shows that BIDS compliance was checked, but there's nothing else in the log. The error log is below. My system has singularity v. 2.6.1. 

/usr/local/miniconda/lib/python3.7/importlib/_bootstrap.py:219: ImportWarning: can't resolve package from __spec__ or __package__, falling
 back on __name__ and __path__
  return f(*args, **kwds)
/usr/local/miniconda/lib/python3.7/importlib/_bootstrap.py:219: ImportWarning: can't resolve package from __spec__ or __package__, falling
 back on __name__ and __path__
  return f(*args, **kwds)
/usr/local/miniconda/lib/python3.7/importlib/_bootstrap.py:219: ImportWarning: can't resolve package from __spec__ or __package__, falling
 back on __name__ and __path__
  return f(*args, **kwds)
/usr/local/miniconda/lib/python3.7/importlib/_bootstrap.py:219: ImportWarning: can't resolve package from __spec__ or __package__, falling
 back on __name__ and __path__
  return f(*args, **kwds)
/usr/local/miniconda/lib/python3.7/site-packages/nilearn/datasets/neurovault.py:16: DeprecationWarning: Using or importing the ABCs from '
collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
  from collections import Container
/usr/local/miniconda/lib/python3.7/site-packages/templateflow/conf/__init__.py:22: ResourceWarning: TemplateFlow: repository not found at
/home/fmriprep/.cache/templateflow. Populating a TemplateFlow stub.
If the path reported above is not the desired location for Templateflow, please set the TEMPLATEFLOW_HOME environment variable.

  warn(_msg, ResourceWarning)
/usr/local/miniconda/lib/python3.7/site-packages/datalad/utils.py:75: DeprecationWarning: dist() and linux_distribution() functions are de
precated in Python 3.5
  = platform.linux_distribution()[:2]
/usr/local/miniconda/lib/python3.7/site-packages/datalad/utils.py:903: DeprecationWarning: Using or importing the ABCs from 'collections'
instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
  is_decorating = not kwargs and len(args) == 1 and isinstance(args[0], collections.Callable)
/usr/local/miniconda/lib/python3.7/site-packages/datalad/support/param.py:30: DeprecationWarning: inspect.getargspec() is deprecated, use
inspect.signature() or inspect.getfullargspec()
  _KNOWN_ARGS = getargspec(argparse.Action.__init__)[0] + ['action']
/usr/local/miniconda/lib/python3.7/site-packages/datalad/interface/base.py:321: DeprecationWarning: inspect.getargspec() is deprecated, us
e inspect.signature() or inspect.getfullargspec()
  args, varargs, varkw, defaults = getargspec(func)
effigies commented 5 years ago

Hi. Apologies for the noisy warnings. There was a recent change that we'd hoped would rein them in somewhat, but it looks like we'll need to look more closely. However, these warnings are not errors; did you run into an actual failure at some point?

oricon commented 5 years ago

The job just failed after 7 seconds and that error log is all that was produced. I'll try again with an increased verbosity.

Joseph M. Orr, Ph.D. Assistant Professor Department of Psychological and Brain Sciences Texas A&M Institute for Neuroscience Texas A&M University College Station, TX

On Mon, Mar 11, 2019 at 1:02 PM Chris Markiewicz notifications@github.com wrote:

Hi. Apologies for the noisy warnings. There was a recent change that we'd hoped would rein them in somewhat, but it looks like we'll need to look more closely. However, these warnings are not errors; did you run into an actual failure at some point?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_poldracklab_fmriprep_issues_1534-23issuecomment-2D471653866&d=DwMCaQ&c=ODFT-G5SujMiGrKuoJJjVg&r=ZKy1VO33u0kvO-PqY1gpb9Ld-AGhtT8c9PAcpsEyp70&m=cmUuEYGygoO7WowlQwiQTqNzNZWd2AhGUpTzmuWhPbo&s=p7uk7ZV6wnGAT77WGkw1Qx7Cu10OyvpxKPEAbuubpno&e=, or mute the thread https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AILZLySeQKSe9SVL5oFuQhiT2sIQUIezks5vVpoYgaJpZM4bpLQl&d=DwMCaQ&c=ODFT-G5SujMiGrKuoJJjVg&r=ZKy1VO33u0kvO-PqY1gpb9Ld-AGhtT8c9PAcpsEyp70&m=cmUuEYGygoO7WowlQwiQTqNzNZWd2AhGUpTzmuWhPbo&s=MxMqL-L9Q0CcCu9kpyfbm8QATbNOa69VAwFwliWGUWw&e= .

oricon commented 5 years ago

Ok here's the error: 


[INFO] Cloning https://github.com/templateflow/templateflow.git [1 other candidates] into '/home/fmriprep/.cache/templateflow'
[ERROR] Failed to clone from any candidate source URL. Encountered errors per each url were: (OrderedDict([('https://github.com/templateflow/templateflow.git', "Cmd('/usr/lib/git-annex.linux/git') failed due to: exit code(128)\n  cmdline: /usr/lib/git-annex.linux/git clone --progress -v https://github.com/templateflow/templateflow.git /home/fmriprep/.cache/templateflow [cmd.py:wait:415]"), ('https://github.com/templateflow/templateflow.git/.git', "Cmd('/usr/lib/git-annex.linux/git') failed due to: exit code(128)\n  cmdline: /usr/lib/git-annex.linux/git clone --progress -v https://github.com/templateflow/templateflow.git/.git /home/fmriprep/.cache/templateflow [cmd.py:wait:415]")]),) [install(/home/fmriprep/.cache/templateflow)]
/usr/local/miniconda/lib/python3.7/site-packages/datalad/distribution/dataset.py:504: DeprecationWarning: inspect.getargspec() is deprecated, use inspect.signature() or inspect.getfullargspec()
  orig_pos = getargspec(f).args
[WARNING] path not associated with any dataset [get(/home/fmriprep/.cache/templateflow)]
Process Process-2:
Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/usr/local/miniconda/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/cli/run.py", line 564, in build_workflow
    from ..workflows.base import init_fmriprep_wf
  File "/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/workflows/base.py", line 24, in <module>
    from niworkflows.interfaces.bids import (
  File "/usr/local/miniconda/lib/python3.7/site-packages/niworkflows/interfaces/__init__.py", line 9, in <module>
    from .registration import (FLIRTRPT as FLIRT,
  File "/usr/local/miniconda/lib/python3.7/site-packages/niworkflows/interfaces/registration.py", line 25, in <module>
    from .mni import (
  File "/usr/local/miniconda/lib/python3.7/site-packages/niworkflows/interfaces/mni.py", line 20, in <module>
    from templateflow.api import get as get_template
  File "/usr/local/miniconda/lib/python3.7/site-packages/templateflow/api.py", line 6, in <module>
    from .conf import TF_LAYOUT, TF_S3_ROOT
  File "/usr/local/miniconda/lib/python3.7/site-packages/templateflow/conf/__init__.py", line 33, in <module>
    install(path=str(TF_HOME), source=TF_GITHUB_SOURCE, recursive=True)
  File "/usr/local/miniconda/lib/python3.7/site-packages/datalad/interface/utils.py", line 491, in eval_func
    return return_func(generator_func)(*args, **kwargs)
  File "/usr/local/miniconda/lib/python3.7/site-packages/datalad/interface/utils.py", line 479, in return_func
    results = list(results)
  File "/usr/local/miniconda/lib/python3.7/site-packages/datalad/interface/utils.py", line 467, in generator_func
    msg="Command did not complete successfully")
datalad.support.exceptions.IncompleteResultsError: Command did not complete successfully [{'action': 'install', 'path': '/home/fmriprep/.cache/templateflow', 'type': 'dataset', 'status': 'error', 'message': ('Failed to clone from any candidate source URL. Encountered errors per each url were: %s', (OrderedDict([('https://github.com/templateflow/templateflow.git', "Cmd('/usr/lib/git-annex.linux/git') failed due to: exit code(128)\n  cmdline: /usr/lib/git-annex.linux/git clone --progress -v https://github.com/templateflow/templateflow.git /home/fmriprep/.cache/templateflow [cmd.py:wait:415]"), ('https://github.com/templateflow/templateflow.git/.git', "Cmd('/usr/lib/git-annex.linux/git') failed due to: exit code(128)\n  cmdline: /usr/lib/git-annex.linux/git clone --progress -v https://github.com/templateflow/templateflow.git/.git /home/fmriprep/.cache/templateflow [cmd.py:wait:415]")]),)), 'source_url': 'https://github.com/templateflow/templateflow.git'}, {'action': 'get', 'path': '/home/fmriprep/.cache/templateflow', 'refds': '/home/fmriprep/.cache/templateflow', 'raw_input': True, 'orig_request': '.', 'state': 'absent', 'status': 'impossible', 'message': 'path not associated with any dataset'}]
effigies commented 5 years ago

Thanks for the full error. I believe @oesteban can comment on how this is supposed to be handled.

oesteban commented 5 years ago

What is your exact command line?

oricon commented 5 years ago 

srun singularity run $image $bidsdir $outdir participant -w $workdir\
    --participant_label $subject --nthreads 30 --omp-nthreads 30 --mem-mb 60000\
    --low-mem --ignore slicetiming --output-space T1w fsnative template\
    --use-aroma --aroma-melodic-dimensionality -150 \
    --fs-license-file $outdir/license.txt -vv --notrack
oesteban commented 5 years ago

Thanks!

Can you try the following?

export TEMPLATEFLOW_HOME=/opt/templateflow
srun singularity run -B $HOME/.templateflow:/opt/templateflow $image $bidsdir $outdir \
        participant -w $workdir\
    --participant_label $subject --nthreads 30 --omp-nthreads 30 --mem-mb 60000\
    --low-mem --ignore slicetiming --output-space T1w fsnative template\
    --use-aroma --aroma-melodic-dimensionality -150 \
    --fs-license-file $outdir/license.txt -vv --notrack

Basically, 1) setting the $TEMPLATEFLOW_HOME variable to some folder within the container; and 2) then binding a folder on your home directory into the path pointed by $TEMPLATEFLOW_HOME.

Once that is tested, you could also give a try to this newer image @chrisfilo just baked last week: https://neurostars.org/t/filenotfounderror-errno-2-no-such-file-or-directory-home-fmriprep-cache/3770/7?u=oesteban In principle, that should work for you as well (and your tests will give us confidence that we really fixed these problems).

oricon commented 5 years ago

That gave an error

ERROR : Failed to get real path of /var/singularity/mnt/final/opt/templateflow: No such file or directory ABORT : Retval = 255 srun: error: c0135: task 0: Exited with exit code 255

I tried making a .templateflow directory in my home directory and in the singularity image, but kept getting the same message.

oesteban commented 5 years ago

Interesting. It seems that singularity does not like that the target path does not exist previously in the image.

What about this?:

unset TEMPLATEFLOW_HOME
srun singularity run -B $HOME/.templateflow:/home/fmriprep/.cache/templateflow \
        $image $bidsdir $outdir participant -w $workdir\
    --participant_label $subject --nthreads 30 --omp-nthreads 30 --mem-mb 60000\
    --low-mem --ignore slicetiming --output-space T1w fsnative template\
    --use-aroma --aroma-melodic-dimensionality -150 \
    --fs-license-file $outdir/license.txt -vv --notrack
oricon commented 5 years ago

Same thing again, and I confirmed that /home/fmriprep/.cache/templateflow is in the container image, so unfortunately that doesn't seem to be the hang-up.


[joseph.orr@login01 fmriprep$ singularity shell -B /home/.templateflow:/home/fmriprep/.cache/templateflow /apps/psyc/containers/fmriprep/1.3.1
ERROR  : Failed to get real path of /var/singularity/mnt/final/home/fmriprep/.cache/templateflow: No such file or directory
ABORT  : Retval = 255
[joseph.orr@login01 fmriprep$ ls -al /apps/psyc/containers/fmriprep/1.3.1/home/fmriprep/.cache/
total 35
drwxr-xrwx 5 joseph.orr psyc  5 Mar  6 12:05 .
drwxr-xrwx 7 joseph.orr psyc 10 Mar  7 15:42 ..
drwxr-xrwx 2 joseph.orr psyc  3 Mar  7 15:42 matplotlib
drwxr-xrwx 2 joseph.orr psyc  3 Mar  7 15:42 pip
drwxr-xrwx 9 joseph.orr psyc  9 Mar  6 12:05 templateflow
oesteban commented 5 years ago

Can you try Chris' image, that could be the only solution

On Mon, Mar 11, 2019, 13:18 Joseph M. Orr notifications@github.com wrote:

Same thing again, and I confirmed that /home/fmriprep/.cache/templateflow is in the container image, so unfortunately that doesn't seem to be the hang-up.

[joseph.orr@login01 fmriprep$ singularity shell -B /home/.templateflow:/home/fmriprep/.cache/templateflow /apps/psyc/containers/fmriprep/1.3.1 ERROR : Failed to get real path of /var/singularity/mnt/final/home/fmriprep/.cache/templateflow: No such file or directory ABORT : Retval = 255 [joseph.orr@login01 fmriprep$ ls -al /apps/psyc/containers/fmriprep/1.3.1/home/fmriprep/.cache/ total 35 drwxr-xrwx 5 joseph.orr psyc 5 Mar 6 12:05 . drwxr-xrwx 7 joseph.orr psyc 10 Mar 7 15:42 .. drwxr-xrwx 2 joseph.orr psyc 3 Mar 7 15:42 matplotlib drwxr-xrwx 2 joseph.orr psyc 3 Mar 7 15:42 pip drwxr-xrwx 9 joseph.orr psyc 9 Mar 6 12:05 templateflow

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/poldracklab/fmriprep/issues/1534#issuecomment-471711927, or mute the thread https://github.com/notifications/unsubscribe-auth/AAkhxioyVvfCMrJgrWqf_nLmHE-v13tXks5vVrotgaJpZM4bpLQl .

oricon commented 5 years ago

Sure, I'll try that tonight

oricon commented 5 years ago

I installed Chris's image, but I'm getting a new error now regarding git configuration for templateflow


[WARNING] It is highly recommended to configure git first (set both user.name and user.email) before using DataLad
. Failed to verify that git is configured: CommandError: command '['git', 'config', 'user.name']' failed with exit
code 1
| Failed to run ['git', 'config', 'user.name'] under None. Exit code=1. out= err= [cmd.py:run:542]CommandError: co
mmand '['git', 'config', 'user.email']' failed with exitcode 1
| Failed to run ['git', 'config', 'user.email'] under None. Exit code=1. out= err= [cmd.py:run:542].  Some operati
ons might fail or not perform correctly.
[ERROR] Failed to clone from any candidate source URL. Encountered errors per each url were: (OrderedDict([('https://github.com/templ
ateflow/templateflow.git', "Cmd('/usr/lib/git-annex.linux/git') failed due to: exit code(128)\n  cmdline: /usr/lib/git-annex.linux/gi
t clone --progress -v https://github.com/templateflow/templateflow.git /opt/templateflow [cmd.py:wait:415]"), ('https://github.com/te
mplateflow/templateflow.git/.git', "Cmd('/usr/lib/git-annex.linux/git') failed due to: exit code(128)\n  cmdline: /usr/lib/git-annex.
linux/git clone --progress -v https://github.com/templateflow/templateflow.git/.git /opt/templateflow [cmd.py:wait:415]")]),) [instal
l(/opt/templateflow)]
/usr/local/miniconda/lib/python3.7/site-packages/datalad/distribution/dataset.py:504: DeprecationWarning: inspect.getargspec() is dep
recated, use inspect.signature() or inspect.getfullargspec()
  orig_pos = getargspec(f).args
/usr/local/miniconda/lib/python3.7/site-packages/datalad/interface/base.py:682: DeprecationWarning: inspect.getargspec() is deprecate
d, use inspect.signature() or inspect.getfullargspec()
  argspec = getargspec(call)
[WARNING] path not associated with any dataset [get(/opt/templateflow)]
Process Process-2:
Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/usr/local/miniconda/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/cli/run.py", line 564, in build_workflow
    from ..workflows.base import init_fmriprep_wf
  File "/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/workflows/base.py", line 24, in <module>
    from niworkflows.interfaces.bids import (
  File "/usr/local/miniconda/lib/python3.7/site-packages/niworkflows/interfaces/__init__.py", line 9, in <module>
    from .registration import (FLIRTRPT as FLIRT,
  File "/usr/local/miniconda/lib/python3.7/site-packages/niworkflows/interfaces/registration.py", line 25, in <module>
    from .mni import (
  File "/usr/local/miniconda/lib/python3.7/site-packages/niworkflows/interfaces/mni.py", line 20, in <module>
    from templateflow.api import get as get_template
  File "/usr/local/miniconda/lib/python3.7/site-packages/templateflow/api.py", line 6, in <module>
    from .conf import TF_LAYOUT, TF_S3_ROOT
  File "/usr/local/miniconda/lib/python3.7/site-packages/templateflow/conf/__init__.py", line 33, in <module>
    install(path=str(TF_HOME), source=TF_GITHUB_SOURCE, recursive=True)
  File "/usr/local/miniconda/lib/python3.7/site-packages/datalad/interface/utils.py", line 491, in eval_func
    return return_func(generator_func)(*args, **kwargs)
  File "/usr/local/miniconda/lib/python3.7/site-packages/datalad/interface/utils.py", line 479, in return_func
    results = list(results)
  File "/usr/local/miniconda/lib/python3.7/site-packages/datalad/interface/utils.py", line 467, in generator_func
    msg="Command did not complete successfully")
datalad.support.exceptions.IncompleteResultsError: Command did not complete successfully [{'action': 'install', 'path': '/opt/templat
eflow', 'type': 'dataset', 'status': 'error', 'message': ('Failed to clone from any candidate source URL. Encountered errors per each
 url were: %s', (OrderedDict([('https://github.com/templateflow/templateflow.git', "Cmd('/usr/lib/git-annex.linux/git') failed due to
: exit code(128)\n  cmdline: /usr/lib/git-annex.linux/git clone --progress -v https://github.com/templateflow/templateflow.git /opt/t
emplateflow [cmd.py:wait:415]"), ('https://github.com/templateflow/templateflow.git/.git', "Cmd('/usr/lib/git-annex.linux/git') faile
d due to: exit code(128)\n  cmdline: /usr/lib/git-annex.linux/git clone --progress -v https://github.com/templateflow/templateflow.gi
t/.git /opt/templateflow [cmd.py:wait:415]")]),)), 'source_url': 'https://github.com/templateflow/templateflow.git'}, {'action': 'get
', 'path': '/opt/templateflow', 'type': 'directory', 'refds': '/opt/templateflow', 'raw_input': True, 'orig_request': '.', 'status':
'impossible', 'message': 'path not associated with any dataset'}]
srun: error: c0207: task 0: Exited with exit code 1
oesteban commented 5 years ago

I think I fixed this problem with the latest update to TemplateFlow. @oricon, could you build a singularity image from poldracklab/fmriprep:templateflow-0.1.2 ?

oricon commented 5 years ago

Sorry, different error this time, looks like a S3 AMS issue


/usr/local/miniconda/lib/python3.7/importlib/_bootstrap.py:219: ImportWarning: can't resolve package from __spec__ or __package__, falling back on __name__ and __path__
  return f(*args, **kwds)
/usr/local/miniconda/lib/python3.7/importlib/_bootstrap.py:219: ImportWarning: can't resolve package from __spec__ or __package__, falling back on __name__ and __path__
  return f(*args, **kwds)
/usr/local/miniconda/lib/python3.7/importlib/_bootstrap.py:219: ImportWarning: can't resolve package from __spec__ or __package__, falling back on __name__ and __path__
  return f(*args, **kwds)
/usr/local/miniconda/lib/python3.7/importlib/_bootstrap.py:219: ImportWarning: can't resolve package from __spec__ or __package__, falling back on __name__ and __path__
  return f(*args, **kwds)
/usr/local/miniconda/lib/python3.7/site-packages/nilearn/datasets/neurovault.py:16: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
  from collections import Container
/usr/local/miniconda/lib/python3.7/site-packages/templateflow/conf/__init__.py:24: ResourceWarning: TemplateFlow: repository not found at /opt/templateflow. Populating a TemplateFlow stub.
If the path reported above is not the desired location for TemplateFlow, please set the TEMPLATEFLOW_HOME environment variable.

  warn(_msg, ResourceWarning)
/usr/local/miniconda/lib/python3.7/site-packages/skimage/__init__.py:80: ResourceWarning: unclosed file <_io.TextIOWrapper name='/usr/local/miniconda/lib/python3.7/site-packages/pytest.py' mode='r' encoding='utf-8'>
  imp.find_module('pytest')
/usr/local/miniconda/lib/python3.7/site-packages/networkx/classes/reportviews.py:95: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
  from collections import Mapping, Set, Iterable
/usr/local/miniconda/lib/python3.7/site-packages/socks.py:58: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
  from collections import Callable
Downloading https://templateflow.s3.amazonaws.com/tpl-OASIS30ANTs/tpl-OASIS30ANTs_res-01_T1w.nii.gz
Process Process-2:
Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.7/site-packages/urllib3/connectionpool.py", line 594, in urlopen
    self._prepare_proxy(conn)
  File "/usr/local/miniconda/lib/python3.7/site-packages/urllib3/connectionpool.py", line 815, in _prepare_proxy
    conn.connect()
  File "/usr/local/miniconda/lib/python3.7/site-packages/urllib3/connection.py", line 324, in connect
    self._tunnel()
  File "/usr/local/miniconda/lib/python3.7/http/client.py", line 911, in _tunnel
    message.strip()))
OSError: Tunnel connection failed: 403 Forbidden

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.7/site-packages/requests/adapters.py", line 445, in send
    timeout=timeout
  File "/usr/local/miniconda/lib/python3.7/site-packages/urllib3/connectionpool.py", line 638, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/usr/local/miniconda/lib/python3.7/site-packages/urllib3/util/retry.py", line 398, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='templateflow.s3.amazonaws.com', port=443): Max retries exceeded with url: /tpl-OASIS30ANTs/tpl-OASIS30ANTs_res-01_T1w.nii.gz (Caused by ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed: 403 Forbidden')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/usr/local/miniconda/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/cli/run.py", line 765, in build_workflow
    err_on_aroma_warn=opts.error_on_aroma_warnings,
  File "/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/workflows/base.py", line 218, in init_fmriprep_wf
    err_on_aroma_warn=err_on_aroma_warn,
  File "/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/workflows/base.py", line 461, in init_single_subject_wf
    name="anat_preproc_wf",
  File "/usr/local/miniconda/lib/python3.7/site-packages/smriprep/workflows/anatomical.py", line 237, in init_anat_preproc_wf
    omp_nthreads=omp_nthreads)
  File "/usr/local/miniconda/lib/python3.7/site-packages/smriprep/workflows/anatomical.py", line 624, in init_skullstrip_ants_wf
    extensions=['.nii', '.nii.gz']))
  File "/usr/local/miniconda/lib/python3.7/site-packages/templateflow/api.py", line 39, in get
    _s3_get(filepath)
  File "/usr/local/miniconda/lib/python3.7/site-packages/templateflow/api.py", line 130, in _s3_get
    r = requests.get(url, stream=True)
  File "/usr/local/miniconda/lib/python3.7/site-packages/requests/api.py", line 72, in get
    return request('get', url, params=params, **kwargs)
  File "/usr/local/miniconda/lib/python3.7/site-packages/requests/api.py", line 58, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/miniconda/lib/python3.7/site-packages/requests/sessions.py", line 512, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/miniconda/lib/python3.7/site-packages/requests/sessions.py", line 622, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/miniconda/lib/python3.7/site-packages/requests/adapters.py", line 507, in send
    raise ProxyError(e, request=request)
requests.exceptions.ProxyError: HTTPSConnectionPool(host='templateflow.s3.amazonaws.com', port=443): Max retries exceeded with url: /tpl-OASIS30ANTs/tpl-OASIS30ANTs_res-01_T1w.nii.gz (Caused by ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed: 403 Forbidden')))
oesteban commented 5 years ago

what is your command line?

oricon commented 5 years ago

I was using


export TEMPLATEFLOW_HOME=/opt/templateflow
srun singularity run -B $HOME/.templateflow:/opt/templateflow $image $bidsdir $outdir participant -w $workdir\
    --participant_label $subject --nthreads 2 --omp-nthreads 2 --mem-mb 8000\
    --low-mem --ignore slicetiming --output-space T1w fsnative template\
    --use-aroma --aroma-melodic-dimensionality -150 \
    --fs-license-file $outdir/license.txt -vv --notrack
oesteban commented 5 years ago

Okay, that would be expected. Can you check without the templateflow boilerplate?

unset TEMPLATEFLOW_HOME
srun singularity run $image $bidsdir $outdir participant -w $workdir\
    --participant_label $subject --nthreads 2 --omp-nthreads 2 --mem-mb 8000\
    --low-mem --ignore slicetiming --output-space T1w fsnative template\
    --use-aroma --aroma-melodic-dimensionality -150 \
    --fs-license-file $outdir/license.txt -vv --notrack
oricon commented 5 years ago

I realized that as soon as you asked about command line. The job is currently running (>4 minutes) without errors and has actually progressed to processing the data.

Thanks!

oesteban commented 5 years ago

We'll be rolling a new release out very soon! Thanks for your precious feedback!