CLI coiled install fails with NotImplementedError exception

[jgawad-nlmk]

Hi guys,

I have tried to install my coiled environment locally, using coiled install. The coiled environment already exists, and it's name is ml-ngo. The outcome is that coiled install fails with exception, output and traceback below:

> coiled install ml-ngo
Creating local conda environment for jgawad-nlmk/ml-ngo
Traceback (most recent call last):
  File "c:\users\j.gawad\miniconda3\envs\coiled\lib\runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "c:\users\j.gawad\miniconda3\envs\coiled\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\j.gawad\Miniconda3\envs\coiled\Scripts\coiled.exe\__main__.py", line 7, in <module>
  File "c:\users\j.gawad\miniconda3\envs\coiled\lib\site-packages\click\core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "c:\users\j.gawad\miniconda3\envs\coiled\lib\site-packages\click\core.py", line 782, in main
    rv = self.invoke(ctx)
  File "c:\users\j.gawad\miniconda3\envs\coiled\lib\site-packages\click\core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "c:\users\j.gawad\miniconda3\envs\coiled\lib\site-packages\click\core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "c:\users\j.gawad\miniconda3\envs\coiled\lib\site-packages\click\core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "c:\users\j.gawad\miniconda3\envs\coiled\lib\site-packages\coiled\cli\install.py", line 79, in install
    cloud._loop_runner.run_sync(
  File "c:\users\j.gawad\miniconda3\envs\coiled\lib\site-packages\distributed\utils.py", line 503, in run_sync
    return sync(self.loop, func, *args, **kwargs)
  File "c:\users\j.gawad\miniconda3\envs\coiled\lib\site-packages\distributed\utils.py", line 351, in sync
    raise exc.with_traceback(tb)
  File "c:\users\j.gawad\miniconda3\envs\coiled\lib\site-packages\distributed\utils.py", line 334, in f
    result[0] = yield future
  File "c:\users\j.gawad\miniconda3\envs\coiled\lib\site-packages\tornado\gen.py", line 762, in run
    value = future.result()
  File "c:\users\j.gawad\miniconda3\envs\coiled\lib\site-packages\coiled\cli\install.py", line 116, in create_local_env
    await create_conda_env(name=local_env_name, solved_spec=solved_spec)
  File "c:\users\j.gawad\miniconda3\envs\coiled\lib\site-packages\coiled\cli\install.py", line 179, in create_conda_env
    async for line in run_command_in_subprocess(conda_create_cmd):
  File "c:\users\j.gawad\miniconda3\envs\coiled\lib\site-packages\coiled\utils.py", line 367, in run_command_in_subprocess
    proc = await asyncio.create_subprocess_shell(
  File "c:\users\j.gawad\miniconda3\envs\coiled\lib\asyncio\subprocess.py", line 216, in create_subprocess_shell
    transport, protocol = await loop.subprocess_shell(
  File "c:\users\j.gawad\miniconda3\envs\coiled\lib\asyncio\base_events.py", line 1597, in subprocess_shell
    transport = await self._make_subprocess_transport(
  File "c:\users\j.gawad\miniconda3\envs\coiled\lib\asyncio\base_events.py", line 491, in _make_subprocess_transport
    raise NotImplementedError
NotImplementedError

As mentioned earlier, the corresponding coiled environment exists:

>coiled env inspect ml-ngo
container:
None

conda:
{'channels': ['defaults', 'conda-forge'],
 'dependencies': ['dask',
                  'dask-ml',
                  'distributed',
                  'lightgbm',
                  'munch',
                  'numpy',
                  'pandas',
                  'pip',
                  'pygmo',
                  'python-dotenv',
                  'python=3.8',
                  'scikit-learn',
                  'scipy',
                  'sklearn-contrib-py-earth',
                  'statsmodels',
                  'toolz'],
 'name': 'ML-NGO'}

pip:
None

post_build:
None

Locally I have neither ML-NGO nor ml-ngo conda environment.

As probably obvious from the traceback, the reported bug was found on a Windows machine with miniconda installed.

[FabioRosado]

Hello @jgawad-nlmk, thank you for raising this issue with us. Looking at the traceback, it seems that the issue is related to the asyncio support on windows. We will need to update the command that we call to create the software environment locally to work with Windows.

I am going to work on this and should be fixed on our next coiled version 0.0.38

[FabioRosado]

This is now implemented in our latest coiled release, thank you so much for reporting this issue