nipype / pydra

Pydra Dataflow Engine
https://nipype.github.io/pydra/
Other
120 stars 59 forks source link

"IsADirectoryError" only occurs when using pydra for the same task #643

Open AaronReer opened 1 year ago

AaronReer commented 1 year ago

Hi all,

I am currently trying to build some workflows with pydra. They should include functions of a library, which we have developed. one step of the workflow would be reading in the config file. Consequently, I tried to set up a really simple workflow for doing that. However, when executing the workflow I get error the following error message:

"name": "IsADirectoryError",
"message": "[Errno 21] Is a directory: '/data2/egapontseva/MEG_QC_stuff/data/from_openneuro/ds003483'",

The workflow looks as follows:

import nest_asyncio

nest_asyncio.apply()

import pydra
import meg_qc  
from meg_qc.meg_qc_pipeline import make_derivative_meg_qc
from meg_qc.source.initial_meg_qc import get_all_config_params, Epoch_meg

# Define the task
@pydra.mark.task
def get_config_params(configfile):
    config_dict = get_all_config_params(configfile)
    return config_dict

# Define the Pydra workflow
workflow_name = "initial_processing"
input_spec = ["configfile"] 
initial_processing_wf = pydra.Workflow(name=workflow_name, input_spec=input_spec)

initial_processing_wf.add(get_config_params(name='read_config', configfile=path_to_config))

# Specify the output of the workflow
initial_processing_wf.set_output([
    ("config_dict", initial_processing_wf.read_config.lzout.out)
])

# Submit the workflow for execution using the 'cf' plugin
with pydra.Submitter(plugin='cf') as sub:
    sub(initial_processing_wf)

# View the output of the workflow
print(initial_processing_wf.result())

The problem is that when just running the function isolated like this: config_dict = get_all_config_params(path_to_config) config_dict it works perfectly and does not produce any error.

I would appreciate your help, since error does not really make sense to me. The directory_path in the dictionary is actually intended to be a path to a directory and that is what produces the error, in the case of running it with pydra, to my understanding.

I am using pydra 0.22.