tfmgr add has no `--osf-project` parameter

[tsalo]

The TemplateFlow contributing instructions say to use the following command to add a new template, per Step 3:

tfmgr add tpl-Name --osf-project '<project_id>'

However, in version 20.0.0a2.dev23+gf4c596a (installed from master today), tfmgr add doesn't have an --osf-project parameter.

Here's the current help message:

Usage: tfmgr add [OPTIONS] TEMPLATE_ID

  Add a new template.

Options:
  --osf-user TEXT
  --osf-password TEXT
  --osf-overwrite
  --gh-user TEXT
  --gh-token TEXT
  --path PATH
  -j, --nprocs INTEGER RANGE  [x>=1]
  --help                      Show this message and exit.

[oesteban]

I think you'll need to refresh your browser - indeed the new workflow does not require an existing OSF project.

[tsalo]

Yes, sorry. I tried using the new instructions this morning, which just use tfmgr add, and got a different error message. I can close this issue, given that it no longer reflects the instructions, and open a new one if you'd like.

Here's the new error message, fwiw:

tfmgr add tpl-dHCPInfant
Has a RRID (research resource ID) already been assigned? []: n
Preparing Pull-Request (wd=/var/folders/ph/w3cbhj9n16s9bx4l7gq3dq6m0000gn/T/tmpjn0i7003).
Cloning into 'templateflow'...
warning: Could not find remote branch tpl-intake to clone.
fatal: Remote branch tpl-intake not found in upstream origin
/bin/sh: hub: command not found
Traceback (most recent call last):
  File "/opt/miniconda3/bin/tfmgr", line 8, in <module>
    sys.exit(cli())
  File "/opt/miniconda3/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/opt/miniconda3/lib/python3.8/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/opt/miniconda3/lib/python3.8/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/miniconda3/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/miniconda3/lib/python3.8/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/opt/miniconda3/lib/python3.8/site-packages/tfmanager/cli.py", line 205, in add
    run_command(
  File "/opt/miniconda3/lib/python3.8/site-packages/tfmanager/io.py", line 29, in run_command
    proc = run(cmd, capture_output=capture_output, shell=True, env=env, cwd=cwd,)
  File "/opt/miniconda3/lib/python3.8/subprocess.py", line 489, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/opt/miniconda3/lib/python3.8/subprocess.py", line 854, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/opt/miniconda3/lib/python3.8/subprocess.py", line 1702, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/var/folders/ph/w3cbhj9n16s9bx4l7gq3dq6m0000gn/T/tmpjn0i7003/templateflow'

My new template is in its own folder (i.e., not in a local clone of templateflow), if that matters.

[oesteban]

This is the culprit:

/bin/sh: hub: command not found

I need to update the instructions with the installation of hub. I think it is not in the setup.cfg because it needs to be installed from conda if I'm not wrong:

conda install -c conda-forge hub

[oesteban]

Also, a better error message would be nice :) /cc @mgxd

[tsalo]

Thanks! It definitely got further this time, but I still got an error. BTW I had to install six as well.

Has a RRID (research resource ID) already been assigned? []: n
Preparing Pull-Request (wd=/var/folders/ph/w3cbhj9n16s9bx4l7gq3dq6m0000gn/T/tmp3dl0so87).
Cloning into 'templateflow'...
warning: Could not find remote branch tpl-intake to clone.
fatal: Remote branch tpl-intake not found in upstream origin
Cloning into 'templateflow'...
fatal: unable to connect to github.com:
github.com[0: 140.82.112.4]: errno=Operation timed out

Traceback (most recent call last):
  File "/opt/miniconda3/bin/tfmgr", line 8, in <module>
    sys.exit(cli())
  File "/opt/miniconda3/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/opt/miniconda3/lib/python3.8/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/opt/miniconda3/lib/python3.8/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/miniconda3/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/miniconda3/lib/python3.8/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/opt/miniconda3/lib/python3.8/site-packages/tfmanager/cli.py", line 205, in add
    run_command(
  File "/opt/miniconda3/lib/python3.8/site-packages/tfmanager/io.py", line 29, in run_command
    proc = run(cmd, capture_output=capture_output, shell=True, env=env, cwd=cwd,)
  File "/opt/miniconda3/lib/python3.8/subprocess.py", line 489, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/opt/miniconda3/lib/python3.8/subprocess.py", line 854, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/opt/miniconda3/lib/python3.8/subprocess.py", line 1702, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/var/folders/ph/w3cbhj9n16s9bx4l7gq3dq6m0000gn/T/tmp3dl0so87/templateflow'

I tried with both a fine-grained GitHub PAT using the permissions in the submission instructions and a general token, but got the same error with both. I don't have any connectivity issues either.

[oesteban]

We probably should drop six (#9)

Regarding the GH issue - it looks like either the token doesn't have sufficient permissions or that GH actually timed out. If it were a permissions problem, GH usually informs properly.

fatal: unable to connect to github.com:
github.com[0: 140.82.112.4]: errno=Operation timed out

[oesteban]

Possibly related? #7

[tsalo]

The template folder contains ~500 MB of data, so it could be timing out, although it only ran for a couple of minutes. I don't know how to work around that problem though.

[oesteban]

I'm going to close this one as the original question has been addressed. @tsalo, please open as many new issues as they are needed (the manager is not in a very healthy state and I assume a few will be necessary).

Thanks a lot for your time :)