Restore lost websocket connections?

[WolframKahl]

Should I report issues here at all, or should they all go into the haste-compiler issue tracker?

Haste.App.Client contains:

sendOverWS :: Endpoint -> Blob -> Client ()
sendOverWS ep blob = do
  -- TODO: use endpoint map to handle disconnects?

I would be very grateful of that was done --- loss of connection is apparently something my students are getting a lot.

(Separate question really: Is there a timeout on the WebSocket? If yes, it would be nice to have an interface for switching that off.)

[valderman]

Fixed in commit 9ba32893fa43085c8ce46e444aa1d7d8b0dbaaf9. The current solution is quite rough however, as reconnects are just mindlessly retried until they succeed; it's not really a fix, but rather the first step towards one.

I'm thinking that the best thing to do here might be to trigger some onDisconnect event handler specified in the Haste.App config, from where the user can choose to try to reconnect, display a pretty "please wait while we're reconnecting" message, or something. The default handler should probably try to reconnect repeatedly, but with an exponential backoff or similar.

[WolframKahl]

That was fast! :smile:

I managed to compile the new version (into a freash GHC-7.10.3 installation) --- there was a cabal hickup in the middle where I needed to give --force-reinstalls and then reinstall the four broken packages. I will attach the log that shows the package lists pulled in by haste-compiler and haste-app; the only thing that strikes my eye is a downgrade of cereal.

After re-installing the four broken packages, ghc-pkg check is happy, but it would be nice if haste-compiler and haste-app could agree better on their dependencies... :wink:

It works for me locally, and I am not sure how to even test it remotely --- you make it sound like I should better not yet deploy this?

Github does not let me upload a .txt file, helpfully suggesting: Try again with a PNG, GIF, JPG, DOCX, PPTX, XLSX, TXT, PDF, or ZIP., but renaming to .TXT does not help either.

Now included here:

# cd haste-compiler
# cabal install --dry-run
#   puills in the following:
MonadRandom-0.4.2.3
StateVar-1.1.0.4
async-2.1.0
base-compat-0.9.1
base-orphans-0.5.4
base64-bytestring-1.0.0.1
blaze-builder-0.4.0.2
byteable-0.1.1
bzlib-0.5.0.5
case-insensitive-1.2.0.7
data-default-class-0.1.2.0
cookie-0.4.2.1
distributive-0.5.0.2
dlist-0.8.0.2
exceptions-0.8.3
fail-4.9.0.0
cereal-0.5.3.0
ghc-paths-0.1.0.9
ghc-simple-0.3 (latest: 0.4)
hourglass-0.2.10
http-types-0.9.1
memory-0.13
asn1-types-0.3.2
asn1-encoding-0.9.4
asn1-parse-0.9.4
cryptonite-0.19 (latest: 0.20)
mime-types-0.1.0.7
mmorph-1.0.6
pem-0.2.2
prelude-extras-0.4.0.3
socks-0.5.5
streaming-commons-0.1.16
http-client-0.4.31.1 (latest: 0.5.3.2)
system-filepath-0.4.13.4
system-fileio-0.3.16.3
tagsoup-0.13.10 (latest: 0.14)
tar-0.5.0.3
temporary-1.2.0.4
shellmate-0.3.4
time-locale-compat-0.1.1.3
transformers-base-0.4.4
monad-control-1.0.1.0
lifted-base-0.2.3.8
resourcet-1.1.8
conduit-1.2.8
utf8-string-1.0.1.1
vector-0.11.0.0
scientific-0.3.4.9
attoparsec-0.13.1.0
aeson-1.0.2.0
conduit-extra-1.1.13.3
void-0.7.1
contravariant-1.4
comonad-5
bifunctors-5.4.1
profunctors-5.2
semigroupoids-5.1
free-4.12.4
either-4.4.1.1
x509-1.6.4
x509-store-1.6.2
x509-system-1.6.4
x509-validation-1.6.5
tls-1.3.8
connection-0.2.6
http-client-tls-0.2.4.1 (latest: 0.3.3)
http-conduit-2.1.11 (latest: 2.2.3)
xml-1.3.14
feed-0.3.11.1
shellmate-extras-0.3.4
haste-compiler-0.6.0.0

# cd ../haste-lib
# cabal install --dry-run

data-binary-ieee754-0.4.4
monads-tf-0.1.0.3

aste-app $ cabal install --dry-run
Resolving dependencies...
In order, the following would be installed:
SHA-1.6.4.2 (via: websockets-0.9.7.0) (new package)
ansi-terminal-0.6.2.3 (via: wai-extra-3.0.18 ansi-wl-pprint-0.6.7.3) (new package)
ansi-wl-pprint-0.6.7.3 (via: optparse-applicative-0.13.0.0) (new package)
appar-0.1.4 (via: iproute-1.7.1) (new package)
auto-update-0.1.4 (via: warp-3.2.8 fast-logger-2.4.7) (new package)
blaze-markup-0.7.1.0 (via: wai-app-static-3.1.6.1 blaze-html-0.8.1.2) (new package)
blaze-html-0.8.1.2 (via: wai-app-static-3.1.6.1) (new package)
byteorder-1.0.4 (via: wai-logger-2.3.0 iproute-1.7.1) (new package)
bytestring-builder-0.10.8.1.0 (via: warp-3.2.8 http2-1.6.2 wai-3.2.1.1 fast-logger-2.4.7) (new package)
cereal-0.4.1.1 (latest: 0.5.3.0) (via: tls-1.3.8 data-embed-0.1.0.0) (new version)
cmdargs-0.10.14 (via: aeson-pretty-0.8.2) (new package)
aeson-pretty-0.8.2 (via: http2-1.6.2) (new package)
easy-file-0.2.1 (via: fast-logger-2.4.7) (new package)
entropy-0.3.7 (via: websockets-0.9.7.0) (new package)
executable-path-0.0.3 (via: data-embed-0.1.0.0) (new package)
data-embed-0.1.0.0 (via: haste-app-0.1.0.0) (new package)
file-embed-0.0.10 (via: wai-websockets-3.0.1.1 wai-app-static-3.1.6.1) (new package)
hex-0.1.2 (via: http2-1.6.2) (new package)
http-date-0.0.6.1 (via: wai-app-static-3.1.6.1 warp-3.2.8) (new package)
iproute-1.7.1 (via: warp-3.2.8 wai-extra-3.0.18) (new package)
network-info-0.2.0.8 (via: haste-app-0.1.0.0) (new package)
optparse-applicative-0.13.0.0 (via: wai-app-static-3.1.6.1) (new package)
psqueues-0.2.2.2 (via: http2-1.6.2) (new package)
simple-sendfile-0.2.25 (via: warp-3.2.8) (new package)
stringsearch-0.3.6.6 (via: wai-extra-3.0.18) (new package)
tls-1.3.8 (via: warp-tls-3.2.2) (reinstall) (changes: cereal-0.5.3.0 -> 0.4.1.1)
unix-compat-0.4.2.0 (via: wai-app-static-3.1.6.1 warp-3.2.8 wai-extra-3.0.18) (new package)
unix-time-0.3.7 (via: wai-logger-2.3.0 fast-logger-2.4.7) (new package)
fast-logger-2.4.7 (via: wai-extra-3.0.18 wai-logger-2.3.0) (new package)
vault-0.3.0.6 (via: warp-3.2.8 wai-extra-3.0.18 wai-3.2.1.1) (new package)
wai-3.2.1.1 (via: haste-app-0.1.0.0 warp-tls-3.2.2 wai-websockets-3.0.1.1 wai-app-static-3.1.6.1 warp-3.2.8 wai-extra-3.0.18 wai-logger-2.3.0) (new package)
wai-logger-2.3.0 (via: wai-extra-3.0.18) (new package)
websockets-0.9.7.0 (via: haste-app-0.1.0.0 wai-websockets-3.0.1.1) (new package)
word8-0.1.2 (via: warp-3.2.8 wai-extra-3.0.18 http2-1.6.2) (new package)
http2-1.6.2 (via: warp-3.2.8) (new package)
wai-extra-3.0.18 (via: wai-app-static-3.1.6.1) (new package)
warp-3.2.8 (via: haste-app-0.1.0.0 warp-tls-3.2.2 wai-websockets-3.0.1.1 wai-app-static-3.1.6.1) (new package)
wai-app-static-3.1.6.1 (via: wai-websockets-3.0.1.1) (new package)
wai-websockets-3.0.1.1 (via: haste-app-0.1.0.0) (new package)
warp-tls-3.2.2 (via: haste-app-0.1.0.0) (new package)
haste-app-0.1.0.0 (new package)
Warning: The following packages are likely to be broken by the reinstalls:
http-client-tls-0.2.4.1
http-conduit-2.1.11
shellmate-extras-0.3.4
connection-0.2.6
Use --force-reinstalls if you want to install anyway.

haste-app $ cabal install  http-client-tls-0.2.4.1 http-conduit-2.1.11 shellmate-extras-0.3.4 connection-0.2.6
Resolving dependencies...
In order, the following will be installed:
socks-0.5.5 (via: connection-0.2.6) (reinstall) (changes: cereal-0.5.3.0 -> 0.4.1.1)
connection-0.2.6 (via: http-client-tls-0.2.4.1) (reinstall) (changes: tls-1.3.8 added)
http-client-tls-0.2.4.1 (latest: 0.3.3) (via: http-conduit-2.1.11) (reinstall) (changes: tls-1.3.8 added)
http-conduit-2.1.11 (latest: 2.2.3) (via: shellmate-extras-0.3.4) (reinstall)
shellmate-extras-0.3.4 (reinstall)
Warning: Note that reinstalls are always dangerous. Continuing anyway...
Configuring socks-0.5.5...
Building socks-0.5.5...
Installed socks-0.5.5
Configuring connection-0.2.6...
Building connection-0.2.6...
Installed connection-0.2.6
Configuring http-client-tls-0.2.4.1...
Building http-client-tls-0.2.4.1...
Installed http-client-tls-0.2.4.1
Configuring http-conduit-2.1.11...
Building http-conduit-2.1.11...
Installed http-conduit-2.1.11
Configuring shellmate-extras-0.3.4...
Building shellmate-extras-0.3.4...
Installed shellmate-extras-0.3.4

[valderman]

It's not that I'd say "don't use the reconnect feature", it's more that it's going to change again soon, in a way that may or may not break compatibility with the current implementation.

Also, I'll take a look at the mismatching dependencies. A few of them may be due to the haste-* libraries expecting the exact same version of the compiler; the 0.6 libraries are under pretty heavy development and are only being tested with the matching version of the compiler.

[WolframKahl]

On Wed, Oct 19, 2016 at 04:16:31AM -0700, Anton Ekblad wrote:

It's not that I'd say "don't use the reconnect feature", it's more that it's going to change again soon, in a way that may or may not break compatibility with the current implementation.

How soon? (I'd be more looking forward to improved interfaces and features than being afraid of changes.)

I'll probably try whether the reconnect attempts freeze my browsers, and might even take a stab at some (capped) exponential backoff if they do. But not before tonight (Canada time).

Also, I'll take a look at the mismatching dependencies. A few of them may be due to the haste-* libraries expecting the exact same version of the compiler; the 0.6 libraries are under pretty heavy development and are only being tested with the matching version of the compiler.

The reported installations were from fresh github checkouts of all pieces (haste-cabal, haste-compiler, haste-app) into a fresh ghc-7.10.3 installation using cabal-install-1.22.9.0, and including cabal update.

(This is how I avoid interference between different versions.)

Thank you!

Wolfram

[valderman]

As of commit 60c3bce48003a9ad1cf124d1788c3d88c39ccd4c there is now support for custom handlers for disconnection and reconnection. The disconnection handler is called once for each disconnected endpoint, and the reconnection handler once for each successful reconnection. The default disconnection handler pops up an error message (covering the entire page, preventing interaction in disconnected state) and tries to reconnect with exponential backoff. Custom disconnection handlers should do something similar, since there is no other automatic reconnection.

Currently, initially failing to connect to an endpoint does not count as a disconnection, and instead just tries to reconnect over and over until a connection is established. I'm leaning towards counting a connection failure as a disconnection however, as there is little conceptual difference between the two.

[WolframKahl]

Thank you very much!

I factored out the retrying component to make it re-usable --- feel free to include if you feel that this is appropriate.

-- |whileRetrying ep n| may somehow display a message like ``Retrying in |n| seconds''.
reconnectAndThen  :: (Endpoint -> Int -> Client ())
                  -> (Endpoint -> Client ()) -> (Endpoint -> Client ())
reconnectAndThen whileRetrying atReconnection = let
    keepRetrying :: Int -> Endpoint -> Client ()
    keepRetrying n ep = do
      whileRetrying ep (n `div` 1000)
      success <- reconnect ep
      if success
        then atReconnection ep
        else do
          sleep n     -- |n| are milliseconds
          keepRetrying (min (1000 * 20) $ max n (n*2)) ep
   in keepRetrying 250

For my purposes, I wanted something unobtrusive, so I only change the page background.

[WolframKahl]

I'm leaning towards counting a connection failure as a disconnection however, as there is little conceptual difference between the two.

This sounds reasonable to me.