Remote spawned server running but ... waiting to connect

[rp2927]

JupyterHub vers. 3.0.0 (python 3.9.7) -- Batchspawner for GridEngine successfully submits the remote server task, the job starts and binds to a reachable port, the Hub acknowledges the running job, but reports "...waiting to connect" and remains this way until (300sec) timeout (then successfully deletes the running job). Turns out the Hub never receives the random port number generated in the /barchspawner/singleuser.py, run by the spawned job, because the _api_request method was never awaited (a discrete warning is posted in the spawner log).

This suggested fix adds a proper call to the now __apirequest async method, and also changes the payload format to match the expected HTTPRequest arguments.

[welcome[bot]]

Thanks for submitting your first pull request! You are awesome! :hugs:
If you haven't done so already, check out Jupyter's Code of Conduct. Also, please make sure you followed the pull request template, as this will help us review your contribution more quickly. You can meet the other Jovyans by joining our Discourse forum. There is also a intro thread there where you can stop by and say Hi! :wave:
Welcome to the Jupyter community! :tada:

[jbeal-work]

Just spent the whole day chasing this down. With my sons help.

Seeing if this fixes the issues for me :)

[curtywang]

This totally fixes the issue for SlurmSpawner too... I was going nuts chasing this down as well!

Installed latest jupyterhub, batchspawner from conda-forge.

Versions from jupyter --version

Selected Jupyter core packages...
IPython          : 8.12.0
ipykernel        : 6.22.0
ipywidgets       : not installed
jupyter_client   : 8.2.0
jupyter_core     : 5.3.0
jupyter_server   : 2.5.0
jupyterlab       : 3.6.3
nbclient         : 0.7.3
nbconvert        : 7.3.1
nbformat         : 5.8.0
notebook         : 6.5.4
qtconsole        : not installed
traitlets        : 5.9.0

[consideRatio]

I think this is a dupliate of https://github.com/jupyterhub/batchspawner/pull/247, and may not be needed nay more.

@minrk I saw that this PR opted to not introduce a dependency on requests, while #247 introduced it without explicitly specifying it as a dependency.. Do you think we should do how its done here without introducing requests?

[consideRatio]

Thank you @rp2927 @jbeal-work @curtywang for working this, the equivalent fix from #247 will be part of version 1.3.0 I'm preparing to get released.

[rp2927]

Pleased to help. Best, Razvan

-- Razvan N. Popescu Research Computing Director Office: (212) 851-9298 @.**@.>

Columbia Business School At the Very Center of Business

From: Erik Sundell @.> Date: Saturday, March 16, 2024 at 8:06 AM To: jupyterhub/batchspawner @.> Cc: Popescu, Razvan @.>, Mention @.> Subject: Re: [jupyterhub/batchspawner] Remote spawned server running but ... waiting to connect (PR #250)

Thank you @rp2927https://github.com/rp2927 @jbeal-workhttps://github.com/jbeal-work @curtywanghttps://github.com/curtywang for working this, the equivalent fix from #247https://github.com/jupyterhub/batchspawner/pull/247 will be part of version 1.3.0 I'm preparing to get released.

— Reply to this email directly, view it on GitHubhttps://github.com/jupyterhub/batchspawner/pull/250#issuecomment-2001966021, or unsubscribehttps://github.com/notifications/unsubscribe-auth/A35PBA6YXKFWPMEUJAO7JPLYYQYTFAVCNFSM6AAAAAASDZ3TNWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBRHE3DMMBSGE. You are receiving this because you were mentioned.Message ID: @.***>