Streamlit / Coiled RuntimeError: There is no current event loop in thread 'ScriptRunner.scriptThread'

[tomgallagher]

Using following package versions:

dask==2022.6.1
coiled==0.2.12
streamlit_folium==0.6.13
numpy==1.21.4
folium==0.12.1.post1
pandas==1.4.3
streamlit==1.10.0

I'm getting this error

This bug occurs in very standard streamlit / coiled code

def get_cloud_client():
    dask.config.set({"coiled.token": st.secrets['coiled_token']})
    cluster = coiled.Cluster(
        n_workers=10,
        name='coiled-streamlit',
        software="datafile-comparison-v1",
        shutdown_on_close=False, 
    )
    client = Client(cluster)
    return client

#Option to fire up cluster
st.sidebar.subheader("Starting your cluster")
st.sidebar.write("Always check the cluster is running before loading data.")
if st.sidebar.button("Start your cluster"):
    cluster_state.write("Starting or connecting to Coiled cluster...")
    client = get_cloud_client()
    cluster_state.write(f"Your Coiled cluster is up! Click the link to access the Dask Dashboard: {st.session_state.client.dashboard_link}")

I'm not sure why I seem to have the problem. Any help hugely appreciated.

[phobson]

Thanks for bringing this to our attention. I'm not sure what's going on yet either. We'll take a look as soon as we can.

[mrocklin]

cc @rrpelgrim

[tomgallagher]

import coiled
import dask
import asyncio
from dask.distributed import Client
import streamlit as st

def get_cloud_client():
    dask.config.set({"coiled.token": st.secrets['coiled_token']})
    loop = asyncio.new_event_loop()
    asyncio.set_event_loop(loop)
    cluster = coiled.Cluster(
        n_workers=10,
        name='coiled-streamlit',
        software="datafile-comparison-v1"
    )
    client = Client(cluster)
    return client

So this fixes the bug, looks like it's the latest version of Streamlit, which includes pages, is not launching event loop in page threads.

[gjoseph92]

The coiled client should be managing the event loop itself, though. That's not something you'd normally need to touch.

[phobson]

@hayesgb FYI

[hayesgb]

@tomgallagher -- following up here. Can you comment on the status of this issue?

[hayesgb]

Should be fixed by latest release of Coiled. Please reopen if this issue is not resolved.