Intermittent failures in setting shell outputs

What version of Pydra are you using?

9849b77f502fcf5aff728bdc4273689b5e2d003e

What were you trying to do?

Run the engine/tests/test_shelltask.py::test_shell_cmd_outputspec_7 test in Github Actions CI

What did you expect will happen?

Test to pass

What actually happened?

_______________ test_shell_cmd_outputspec_7[cf-result_submitter] _______________
[gw1] linux -- Python 3.11.4 /opt/hostedtoolcache/Python/3.11.4/x64/bin/python

tmp_path = PosixPath('/tmp/pytest-of-runner/pytest-0/popen-gw1/test_shell_cmd_outputspec_7_cf1')
plugin = 'cf', results_function = <function result_submitter at 0x7f91f5ee1300>

    @pytest.mark.parametrize("results_function", [result_no_submitter, result_submitter])
    def test_shell_cmd_outputspec_7(tmp_path, plugin, results_function):
        """
        providing output with output_file_name and using MultiOutputFile as a type.
        the input field used in the template is a MultiInputObj, so it can be and is a list
        """
        file = tmp_path / "script.sh"
        file.write_text('for var in "$@"; do touch file"$var".txt; done')

        cmd = "bash"
        new_files_id = ["1", "2", "3"]

        my_input_spec = SpecInfo(
            name="Input",
            fields=[
                (
                    "script",
                    attr.ib(
                        type=File,
                        metadata={
                            "help_string": "script file",
                            "mandatory": True,
                            "position": 1,
                            "argstr": "",
                        },
                    ),
                ),
                (
                    "files_id",
                    attr.ib(
                        type=MultiInputObj,
                        metadata={
                            "position": 2,
                            "argstr": "...",
                            "sep": " ",
                            "help_string": "list of name indices",
                            "mandatory": True,
                        },
                    ),
                ),
            ],
            bases=(ShellSpec,),
        )

        my_output_spec = SpecInfo(
            name="Output",
            fields=[
                (
                    "new_files",
                    attr.ib(
                        type=MultiOutputFile,
                        metadata={
                            "output_file_template": "file{files_id}.txt",
                            "help_string": "output file",
                        },
                    ),
                )
            ],
            bases=(ShellOutSpec,),
        )

        shelly = ShellCommandTask(
            name="shelly",
            executable=cmd,
            input_spec=my_input_spec,
            output_spec=my_output_spec,
            script=file,
            files_id=new_files_id,
        )

        res = results_function(shelly, plugin)
        assert res.output.stdout == ""
>       for file in res.output.new_files:
E       TypeError: '_Nothing' object is not iterable

/opt/hostedtoolcache/Python/3.11.4/x64/lib/python3.11/site-packages/pydra/engine/tests/test_shelltask.py:3115: TypeError

Note that it passes for Python < 3.10 and on the Mac CI test (all versions) so something a little strange is happening.

Can you replicate the behavior? If yes, how?

By rerunning the tests in the CI. However, I was not able to replicate on the issue on a similar Ubuntu VM,

Linux ip-10-0-2-164 5.19.0-1027-aws #28~22.04.1-Ubuntu SMP Wed May 31 18:30:36 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux with Python==3.10.6

Notes/thoughts

It appears switching from the "cf" plugin to the "serial" may fix the problem (it did for engine/tests/test_shelltask.py::test_shell_cmd_inputsspec_11 at least). So the bug could possibly be due to some problem gathering of results from different processes

nipype / pydra