docker error when using singularity image to run ks4

[a-zmz]

Hello,

i have been running ks4 with singularity successfully in the past week, but today i got an error saying docker module not found:

ModuleNotFoundError                       Traceback (most recent call last)                                                                                                                                                                   
File ~/Documents/git/analysis/analysis/preprocessing/test_ks4.py:122                                                                                                                                                                          
    119 os.chdir(preprocess_folder)                                                                                                                                                                                                           
    121 # sort spikes                                                                                                                                                                                                                         
--> 122 sorting = ss.run_sorter(                                                                                                                                                                                                              
    123     sorter_name='kilosort4',                                                                                                                                                                                                          
    124     recording=ap_mcd,                                                                                                                                                                                                                 
    125     #folder=str(base_folder/"ks4_full"),                                                                                                                                                                                              
    126     singularity_image=True,                                                                                                                                                                                                           
    127     remove_existing_folder=True,                                                                                                                                                                                                      
    128     #**ks4_params,                                                                                                                                                                                                                    
    129 )                                                                                                                                                                                                                                     
    131 # curate                                                                                                                                                                                                                              
    132 sorting = scur.remove_duplicated_spikes(                                                                                                                                                                                              
    133     sorting,                                                                                                                                                                                                                          
    134     censored_period_ms=0.3,                                                                                                                                                                                                           
    135     method="keep_first_iterative",                                                                                                                                                                                                    
    136 )                                                                                                                                                                                                                                     

File ~/Documents/git/spikeinterface/src/spikeinterface/sorters/runsorter.py:210, in run_sorter(sorter_name, recording, folder, remove_existing_folder, delete_output_folder, verbose, raise_error, docker_image, singularity_image, delete_con
tainer_files, with_output, output_folder, **sorter_params)                                                                                                                                                                                    
    204         if not has_spython():                      
    205             raise RuntimeError(                    
    206                 "The python `spython` package must be installed to "                                           
    207                 "run singularity. Install with `pip install spython`"                                          
    208             )                                      
--> 210     return run_sorter_container(                   
    211         container_image=container_image,           
    212         mode=mode,                                 
    213         **common_kwargs,                           
    214     )                                              
    216 return run_sorter_local(**common_kwargs)

File ~/Documents/git/spikeinterface/src/spikeinterface/sorters/runsorter.py:525, in run_sorter_container(sorter_name, recording, mode, container_image, folder, remove_existing_folder, delete_output_folder, verbose, raise_error, with_output, delete_container_files, extra_requirements, installation_mode, spikeinterface_version, spikeinterface_folder_source, output_folder, **sorter_params)
    522     py_user_base_folder.mkdir(parents=True, exist_ok=True)
    523     py_user_base_unix = path_to_unix(py_user_base_folder)
--> 525 container_client = ContainerClient(mode, container_image, volumes, py_user_base_unix, extra_kwargs)
    526 if verbose:
    527     print("Starting container")

File ~/Documents/git/spikeinterface/src/spikeinterface/sorters/container_tools.py:102, in ContainerClient.__init__(self, mode, container_image, volumes, py_user_base, extra_kwargs)
     99     singularity_image = sif_file
    100 else:
--> 102     docker_image = self._get_docker_image(container_image)
    103     if docker_image and len(docker_image.tags) > 0:
    104         tag = docker_image.tags[0]

File ~/Documents/git/spikeinterface/src/spikeinterface/sorters/container_tools.py:127, in ContainerClient._get_docker_image(container_image)
    125 @staticmethod
    126 def _get_docker_image(container_image):
--> 127     import docker
    129     docker_client = docker.from_env(timeout=300)
    130     try:

ModuleNotFoundError: No module named 'docker'
> /home/amz/Documents/git/spikeinterface/src/spikeinterface/sorters/container_tools.py(127)_get_docker_image()
    125     @staticmethod
    126     def _get_docker_image(container_image):
--> 127         import docker
    128 
    129         docker_client = docker.from_env(timeout=300)

so then i installed docker, but got another error:

FileNotFoundError Traceback.txt

installed spikeinterface main on linux fedora 39.

Could someone please help? Thanks!

[a-zmz]

update: if i copy previously used kilosort4-base.sif to the running folder, it solves the issue...

[alejoe91]

it could be that the docker servers/internet connection were down

[a-zmz]

hmm internet connection is fine, i tried again just now, issue remains

[mhhennig]

Same problem here (system with apptainer). Happens on machines where docker cannot be run (no root in my case).

docker_client = docker.from_env(timeout=300) in _get_docker_image(container_image) fails.

This is not good, have not been able to find a solution yet.

[mhhennig]

Found a solution I think, PR coming.