Image generated by docker/singularity has issue importing python module

[ljhearne]

Hi Erin,

Thanks for the fantastic tool / all the work you have put in.

I'm trying to get ciftify up and running on the local HPC. To do so I created a singularity container using: singularity pull docker://tigrlab/fmriprep_ciftify:latest

Then to check it was working I open a shell in the container and ask for help on a ciftify function: Singularity shell --cleanenv <path-to-img> (within the container):~> ciftify_recon_all --help

However, I get an error when trying to import some pandas modules (below). I have been following the scripts on https://github.com/edickie/bids-on-scinet/blob/master/examples/sbatch_fmriprep_ciftify_p08.sh so it doesn't seem like I need to import anything else other than the container... Let me know if you have any thoughts!

> Traceback (most recent call last):
>   File "/home/code/ciftify/ciftify/bin/ciftify_recon_all", line 76, in <module>
>     import ciftify
>   File "/home/code/ciftify/ciftify/__init__.py", line 11, in <module>
>     from . import niio
>   File "/home/code/ciftify/ciftify/niio.py", line 11, in <module>
>     import pandas as pd
>   File "/usr/local/miniconda/lib/python3.6/site-packages/pandas/__init__.py", line 42, in <module>
>     from pandas.core.api import *
>   File "/usr/local/miniconda/lib/python3.6/site-packages/pandas/core/api.py", line 10, in <module>
>     from pandas.core.groupby import Grouper
>   File "/usr/local/miniconda/lib/python3.6/site-packages/pandas/core/groupby/__init__.py", line 2, in <module>
>     from pandas.core.groupby.groupby import (
>   File "/usr/local/miniconda/lib/python3.6/site-packages/pandas/core/groupby/groupby.py", line 42, in <module>
>     from pandas.core.dtypes.missing import isna, isnull, notna, _maybe_fill
> ImportError: cannot import name 'isna'

[edickie]

So it looks like singularity can't build the image properly from docker hub...

If it possible for you to use docker to build the image and then convert it to a singularity container. You would need a machine you can run docker on (like a local laptop).

$ docker build --privileged -t --rm tigrlab/fmriprep_ciftify: $ docker run --privileged -t --rm \ -v /var/run/docker.sock:/var/run/docker.sock \ -v D:\host\path\where\to\output\singularity\image:/output \ singularityware/docker2singularity \ tigrlab/fmriprep_ciftify:

Otherwise, we can try (tomorrow) to put the image up on singularity hub?

Also, I just built a newer version fo the container (with the fixes to some of the ciftify_recon_all options) which I haven't pushed yet (will tomorrow). I would suggest using the version number instead of "latest" to keep track of what version you have.

Thanks for letting me know! We'll update the docs with this issue

[ljhearne]

Thanks Erin, I'll download docker locally and build the singularity container that way.

I'll report back if I still get the same error.

edit: Seems like it was a docker issue before - I fixed by following the comments here. I'm using Singularity on our HPC and the help arguments are working, so I'm assuming this has fixed the problem! Great!