download data from openneuro instead of openfmri

[kalenkovich]

• it has both raw (which we don't need) and maxfiltered (which we do need) data,
• no archives involved,
• event are already extracted

Problems:

1. downloads often fail which I solved by running snakemake again and again and again. That could be improved, of course. I haven't found a way to tell snakemake that it should try to re-run failed jobs so, I guess, this has to be done in the rule itself. I am not sure how to correctly catch download problems so that we don't catch something we shouldn't. I looked for a package that could do that for us but haven't found anything.
2. if you ask openneuro for a file that is not in the dataset it will give you a text file that says "no such file in the git repository". We should probably also check for that as well.

We can try to deal with these problems now or create an issue to be dealt with later.

[levchenkoegor]

Tried to reproduce this branch locally, but after snakemake all --cores 1 it returns an error:

Traceback (most recent call last):
  File "E:\Egor_Levchenko\Software\Miniconda\envs\reproduction\lib\site-packages\snakemake\__init__.py", line 593, in snakemake
    workflow.include(
  File "E:\Egor_Levchenko\Software\Miniconda\envs\reproduction\lib\site-packages\snakemake\workflow.py", line 1087, in include
    if is_local_file(snakefile):
  File "E:\Egor_Levchenko\Software\Miniconda\envs\reproduction\lib\site-packages\snakemake\common.py", line 65, in is_local_file
    from smart_open import parse_uri
ImportError: cannot import name 'parse_uri' from 'smart_open' (E:\Egor_Levchenko\Software\Miniconda\envs\reproduction\lib\site-packages\smart_open\__init__.py)

I've checked if smart_open is installed and it is with version 1.9.0, but I guess the problem is somewhere else.

Do you have any ideas?

[kalenkovich]

@levchenkoegor, have you checked this one?

[levchenkoegor]

Yea, can be merged