swc-shell-novice scaffolds fail to build in binder

[bielsnohr]

Upon clicking the JupyterLab or JupyterHub link for the swc-shell-novice scaffolds, the session fails to build and launch in binder. Screenshot of build log errors:

I have tried this in both Firefox and Chromium, so it is unlikely to be a browser issue. I also have confirmation from a colleague that these failed to build for them.

[zkamvar]

I'm getting the same build failure on my end. It looks like everything is working until the process where it is starting the notebook itself. This is a problem on the mybinder end, so I'll raise an issue there and see if there is any solution.

For posterity, these are the raw logs I'm getting:

Found built image, launching...
Launching server...
Server requested
2021-10-19T15:37:46.813352Z [Normal] Successfully assigned prod/jupyter-carpentries-2dscaffolds-2dniahnsqi to gke-prod-user-202009-b9c03ca0-6qx4
2021-10-19T15:37:47Z [Normal] Container image "jupyterhub/mybinder.org-tc-init:2020.12.4-n655.hfe65496" already present on machine
2021-10-19T15:37:47Z [Normal] Created container tc-init
2021-10-19T15:37:47Z [Normal] Started container tc-init
2021-10-19T15:37:48Z [Normal] Pulling image "gcr.io/binderhub-288415/r2d-staging-g5b5b759-carpentries-2dscaffolds-a0305a:264cd6761c4079de4ee2aea4779b5a66b26564bc"
2021-10-19T15:38:16Z [Normal] Successfully pulled image "gcr.io/binderhub-288415/r2d-staging-g5b5b759-carpentries-2dscaffolds-a0305a:264cd6761c4079de4ee2aea4779b5a66b26564bc" in 27.748927076s
2021-10-19T15:38:16Z [Normal] Created container notebook
2021-10-19T15:38:16Z [Normal] Started container notebook
Spawn failed: pod prod/jupyter-carpentries-2dscaffolds-2dniahnsqi did not start in 300 seconds!
Launch attempt 1 failed, retrying...
Server requested
2021-10-19T15:42:45.713490Z [Normal] Successfully assigned prod/jupyter-carpentries-2dscaffolds-2dnftf9xuv to gke-prod-user-202009-b9c03ca0-ku80
2021-10-19T15:42:47Z [Normal] Container image "jupyterhub/mybinder.org-tc-init:2020.12.4-n655.hfe65496" already present on machine
2021-10-19T15:42:47Z [Normal] Created container tc-init
2021-10-19T15:42:47Z [Normal] Started container tc-init
2021-10-19T15:42:48Z [Normal] Pulling image "gcr.io/binderhub-288415/r2d-staging-g5b5b759-carpentries-2dscaffolds-a0305a:264cd6761c4079de4ee2aea4779b5a66b26564bc"
2021-10-19T15:43:16Z [Normal] Successfully pulled image "gcr.io/binderhub-288415/r2d-staging-g5b5b759-carpentries-2dscaffolds-a0305a:264cd6761c4079de4ee2aea4779b5a66b26564bc" in 27.769661977s
2021-10-19T15:43:19Z [Normal] Created container notebook
2021-10-19T15:43:19Z [Normal] Started container notebook
Spawn failed: pod prod/jupyter-carpentries-2dscaffolds-2dnftf9xuv did not start in 300 seconds!
Launch attempt 2 failed, retrying...
Server requested
2021-10-19T15:47:32.330215Z [Normal] Successfully assigned prod/jupyter-carpentries-2dscaffolds-2dp957w176 to gke-prod-user-202009-b9c03ca0-nrh6
2021-10-19T15:47:33Z [Normal] Container image "jupyterhub/mybinder.org-tc-init:2020.12.4-n655.hfe65496" already present on machine
2021-10-19T15:47:33Z [Normal] Created container tc-init
2021-10-19T15:47:33Z [Normal] Started container tc-init
2021-10-19T15:47:33Z [Normal] Pulling image "gcr.io/binderhub-288415/r2d-staging-g5b5b759-carpentries-2dscaffolds-a0305a:264cd6761c4079de4ee2aea4779b5a66b26564bc"
2021-10-19T15:47:51Z [Normal] Successfully pulled image "gcr.io/binderhub-288415/r2d-staging-g5b5b759-carpentries-2dscaffolds-a0305a:264cd6761c4079de4ee2aea4779b5a66b26564bc" in 17.768284863s
2021-10-19T15:47:51Z [Normal] Created container notebook
2021-10-19T15:47:52Z [Normal] Started container notebook
Spawn failed: Timeout
Launch attempt 3 failed, retrying...
Server requested
2021-10-19T15:52:48.510152Z [Normal] Successfully assigned prod/jupyter-carpentries-2dscaffolds-2dc4bkd2b3 to gke-prod-user-202009-b9c03ca0-nrh6
2021-10-19T15:52:49Z [Normal] Container image "jupyterhub/mybinder.org-tc-init:2020.12.4-n655.hfe65496" already present on machine
2021-10-19T15:52:49Z [Normal] Created container tc-init
2021-10-19T15:52:49Z [Normal] Started container tc-init
2021-10-19T15:52:50Z [Normal] Container image "gcr.io/binderhub-288415/r2d-staging-g5b5b759-carpentries-2dscaffolds-a0305a:264cd6761c4079de4ee2aea4779b5a66b26564bc" already present on machine
2021-10-19T15:52:50Z [Normal] Created container notebook
2021-10-19T15:52:50Z [Normal] Started container notebook
Spawn failed: pod prod/jupyter-carpentries-2dscaffolds-2dc4bkd2b3 did not start in 300 seconds!
Spawn failed: pod prod/jupyter-carpentries-2dscaffolds-2dc4bkd2b3 did not start in 300 seconds!

[fomightez]

Issue seems to be the start file needs to end with exec and not eval.

For easy copying, the following is good as last line of a start file:

exec "$@"

The postBuild to install the Table of Contents (TOC) feature is no longer necessary as it is built in to JupyterLab 3 now.

[zkamvar]

Thank you @fomightez! I updated the branches with:

for i in $(git branch | grep -v main); do 
  git checkout $i
  sed -i -r -e 's/eval/exec/' .binder/start
  rm -f .binder/postBuild
  git add .
  git commit -m "update ${i} to fix #7"
  git push
  git checkout -
done

[zkamvar]

The binder instances should now build properly

[MikeTrizna]

FYI, @zkamvar, I was having trouble getting "man" to work in our own Binder repos, and found your fix in https://github.com/carpentries/scaffolds/issues/1. However, I kept running into this same hanging Binder issue described here because I copied the "start" file from the dc-genomics branch: https://github.com/carpentries/scaffolds/tree/dc-genomics which still has "eval" instead of "exec".

All of that is to say that it appears that at least 1 branch here still has the old version of the start file, so your for-loop may have missed some fixes.

[zkamvar]

this is why I should have checked for all upstream branches :facepalm:

[zkamvar]

Thank you for pointing this out!