Prefect 2.0 Compatibility

[hayesgb]

Describe the bug A user reported:

a compatibility issue when running coiled+prefect 2.0+jupyterlab

import dask
import coiled
from prefect import task, flow

@task def test_cluster(): cluster = coiled.Cluster(n_workers=2,...) cluster.close()

@flow def t_c(): test_cluster()

t_c()

which returned the following

File "/tmp/ipykernel_25136/1775350701.py", line 8, in test_cluster cluster = coiled.Cluster(n_workers=2,software="fugue-env") File "/usr/local/lib/python3.8/site-packages/coiled/_beta/cluster.py", line 228, in init self._requested_or_fetched_cluster = asyncio.Event() File "/usr/local/lib/python3.8/asyncio/locks.py", line 260, in init self._loop = events.get_event_loop() File "/usr/local/lib/python3.8/asyncio/events.py", line 639, in get_event_loop raise RuntimeError('There is no current event loop in thread %r.'

**Reference Links, optional**
Is there a GitHub issue or Slack thread with more details?
https://coiled-users.slack.com/archives/C018QQ3SLLF/p1660065945844799

**Issue Priority, optional**
- [ ] Low
- [ ] Medium
- [ ] High

**Environment, optional**
- Python version:
- coiled-runtime version:
If not using coiled-runtime, then:
- Dask version:
- Coiled version:

**Diagnostics, optional**
Please run `coiled.diagnostics` locally and paste the result below:

<details>
<summary>Coiled Diagnostics Output:</summary>

</details>

**Additional context**
Attempts to create either of the two following conda environments fails to solve successfully:
`conda create -n test python==3.8.12 coiled  "prefect >=2"`
`conda create -n test python==3.8.12 dask distributed "prefect >=2"`

[phobson]

Oh wow. Prefect 2.0 has some pretty big changes.

And yeah, even their docs only show installation via some form a pip (conda is mentioned, but not demonstrated).

Trying to upgrade my own conda env, I get:

Encountered problems while solving:
  - nothing provides asgi-lifespan >=1.0 needed by prefect-2.0.1-pyhd8ed1ab_1

[fjetter]

I think there is a problem with the prefect 2.0 conda package itself already

conda create -n prefect2.0 prefect=2.0
Collecting package metadata (current_repodata.json): done
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: -
Found conflicts! Looking for incompatible packages.
This can take several minutes.  Press CTRL-C to abort.
failed

UnsatisfiableError:
Note that strict channel priority may have removed packages required for satisfiability.

If we try a solve with, e.g. dask

conda create -n prefect2.0 prefect=2.0 dask

prefect=2.0 -> aiofiles[version='>=0.7.0'] -> python[version='>=2.6|>=3.10,<3.11.0a0|>=3.10,<3.11.0a0|>=3.6|>=3.9,<3.10.0a0|>=3.9,<3.10.0a0|>=3.8,<3.9.0a0|>=3.8,<3.9.0a0|>=3.6.1|>=2.7,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*|>=3|>=3.6.2|>=2.7',build=*_cpython]
prefect=2.0 -> python[version='>=3.7']

It appears that prefect already has an internal conflict with aiofiles. Not sure if some of this pinning spaghetti is the cause or the build=*_cpython flag. Not familiar with conda sufficiently to tell.

Disentangled aiofiles dependency spaghetti

``` >=2.6 | >=3.10 <3.11.0a0 | >=3.10 <3.11.0a0 | >=3.6 | >=3.9 <3.10.0a0 | >=3.9 <3.10.0a0 | >=3.8 <3.9.0a0 | >=3.8 <3.9.0a0 | >=3.6.1 | >=2.7 !=3.0.* !=3.1.* !=3.2.* !=3.3.* !=3.4.* | >=3 | >=3.6.2 | >=2.7 ``` I _think_ this simplifies to "any non-alpha version >=3.8"

[phobson]

With fully pip'd environment I reproduce the error with:

1. the example as is
2. using if __name__ == "__main__"

The example runs correctly if you use a distributed.LocalCluster instead of a coiled cluster.

[dchudz]

n.b. user has temporary workaround:

import asyncio
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
client = Client(coiled.Cluster(name=some_cluster_name))

[dchudz]

Root cause is probably similar to https://github.com/coiled/feedback/issues/173.

[shughes-uk]

Yeah cause of this seems to be a weird situation where starting the cloud object does not end up creating an event loop attached to the current thread. @graingert 's MR I think should handle it adequately

[dchudz]

@FabioRosado @graingert this is resolved now, right?

[FabioRosado]

Yes this is resolved and released on the latest Coiled version.

Anyone using an older version might still see this issue but I think we can close it for now?