"Failed to connect to event stream" frequently for xarray's binder

[max-sixty]

Firstly, thanks for the awesome product!

I'm frequently seeing "Failed to connect to event stream" on the xarray examples (e.g. https://mybinder.org/v2/gh/pydata/xarray/master?urlpath=lab/tree/doc/examples/multidimensional-coords.ipynb). I tried using both Lab and normal notebooks. Is there somewhere we can see more logs to assess what the problem is?

I tried the Dask one and it works well: https://mybinder.org/v2/gh/dask/dask-examples/master?urlpath=lab. It doesn't specify the file; I tried changing xarray's to the path but no luck. Potentially the different is in the requirements?

Ref #155, which had a similar error message. It was closed awaiting more reports.

[betatim]

The error is annoying but harmless, you can reload the page to "reconnect". Thanks for reporting it though because otherwise we don't notice how often this happens!

We still don't have a good idea when exactly it happens, other than "sometimes and shaky internet connections or large amounts of output make it worse". For example for me the xarray binder built just fine on the first try now :-/

This means it is a bit tricky to know what the actionable thing is to do here. Maybe something to think about is to automate the reloading of the page when this error happens as that is what we tell users to do manually. What do you think?

[max-sixty]

Thanks for the fast reply.

But I don't think reloading makes it work: I did try that previously, at least half a dozen times over ~15 minutes. My colleague tried too. I also just tried again (i.e. 4 hours after the original issue) and I see the same error...

[choldgraf]

@max-sixty could you try launching it on a different BinderHub instance? e.g. hard-code gke. or gesis. before the "mybinder.org" bit...maybe it's a hub-specific thing?

[max-sixty]

It now works (on https://hub-binder.mybinder.ovh/user/pydata-xarray-am8ztkkl/lab, though I guess that's temporary)

Are there any logs to work out what's happening / diagnose the problem? I'm happy to help...

[choldgraf]

unfortunately not - the annoying thing about this particular bug is that it is the logs communication breaking down, so it is hard to pinpoint what's going wrong :-(

[max-sixty]

I just checked again, and it's broken again :(

https://mybinder.org/v2/gh/pydata/xarray/master?urlpath=lab/tree/doc/examples/multidimensional-coords.ipynb

@max-sixty could you try launching it on a different BinderHub instance? e.g. hard-code gke. or gesis. before the "mybinder.org" bit...maybe it's a hub-specific thing?

I tried adding gke and gesis to the front (i.e. https://gke.mybinder.org/v2/gh/pydata/xarray/master?urlpath=lab/tree/doc/examples/multidimensional-coords.ipynb, https://notebooks.gesis.org/binder/v2/gh/pydata/xarray/master?urlpath=lab/tree/doc/examples/multidimensional-coords.ipynb), and both failed with the same error

Could it be that xarray's environment.yml isn't fully pinned? I'm trying to see the difference between Dask's successful launches and xarray's failures

[betatim]

The place I'd search is a difference in the amount of output produced in each build and the network connection you are on. The build continues happily in the background even when you are disconnected. The error is purely about the network connection between your browser and mybinder.org

[max-sixty]

The place I'd search is a difference in the amount of output produced in each build and the network connection you are on.

Where can we see the output?

[betatim]

You see the log output in the same place as the error message you reported. I don't know if a difference in the amount of text produced there is a reason for the error to happen but I could imagine that it isn't impossible :)

[max-sixty]

Thanks for engaging @betatim . Unfortunately I only see that line, and can't seem to scroll to any other output. (FWIW it seems to break more often than work, so if you click on the links above you can probably see the same thing). Am I missing something?

[betatim]

That is the weird thing, for me it works (unscientifically) 99% of the time, at least I hardly ever see the error. I tried refreshing your links many times to see if that would provoke it or some other repos that have caused this problem in the past. My hunch is that this has something to do with the network between the user and mybinder.org, but don't know what it is. This is based on the fact that there are people for whom this isn't a problem at all and people for who it is reproducibly&frequently a problem.

Some wild guesses: Different browsers? Different corporate network settings/firewalls? Mobile vs wired internet? Ad blockers?

[max-sixty]

Another oddity is that it's only xarray that seems to fail. As above, the dask examples work fine, and I just tried Beaker here.

It's a shame because Binder is such a great resource for a library like xarray, but people's first impressions may be that it's poorly maintained.

Any ideas why xarray might fail while others succeed?

[betatim]

I'll close this issue as it is a duplicate of https://github.com/jupyterhub/binderhub/issues/743 which tracks the same problem. I don't believe this is a problem with the source repository so we only need one issue to track the problem and work on a solution.

Any ideas why xarray might fail while others succeed?

As I suggested before: My hunch is that this has something to do with the network between the user and mybinder.org, but don't know what it is. This is based on the fact that there are people for whom this isn't a problem at all and people for who it is reproducibly&frequently a problem.

Some wild guesses: Different browsers? Different corporate network settings/firewalls? Mobile vs wired internet? Ad blockers?

Because this seems to depend on the user not the repo we need one of the effected users to debug this.