Timeout during serving voila application

[ricky-lim]

Hi Dan,

I was testing a basic voila application using cdsdashboards on z2h, with similar setup as in issue #43

The voila application repo: https://github.com/ricky-lim/wealth-nederland The application was also deployed at heroku (as a test) here: http://wealth-of-nederland.herokuapp.com/

Upon first launch, the application was served properly, however for the subsequent requests (next users), unfortunately it gave TimeoutError

Further digging into the log, I found this error from jhsingle_native_proxy.

Still puzzled what went wrong. I would be very much grateful to hear your advice to solve this issue, please.

Cheers and thanks in advance for your time and support.

future: <Task finished coro=<SuperviseAndProxyHandler.ensure_process.<locals>.pipe_output() done, defined at /opt/conda/lib/python3.7/site-packages/jhsingle_native_proxy/proxyhandlers.py:645> exception=ValueError('Separator is found, but chunk is longer than limit')>
Traceback (most recent call last):
  File "/opt/conda/lib/python3.7/asyncio/streams.py", line 496, in readline
    line = await self.readuntil(sep)
  File "/opt/conda/lib/python3.7/asyncio/streams.py", line 592, in readuntil
    'Separator is found, but chunk is longer than limit', isep)
asyncio.streams.LimitOverrunError: Separator is found, but chunk is longer than limit

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/conda/lib/python3.7/site-packages/jhsingle_native_proxy/proxyhandlers.py", line 650, in pipe_output
    line = await stream.readline()
  File "/opt/conda/lib/python3.7/asyncio/streams.py", line 505, in readline
    raise ValueError(e.args[0])
ValueError: Separator is found, but chunk is longer than limit

[ricky-lim]

Aha I found it, it was caused by the log with a long line, as I enabled debug mode

Do you have any advice for enabling c.Spawner.debug without hitting this limit ?

[danlester]

Fantastic - thanks for reporting this and the solution.

There is a related issue here, to think about how to limit the debug output instead of having nothing or everything: https://github.com/ideonate/jhsingle-native-proxy/issues/7

So I would like to have a think about how to handle this better. In any case, I would like to fix the underlying overflow problem too.

[danlester]

Thanks again for reporting this. It's handled much better in version 0.4.1, just released. Logs are truncated rather than letting it break everything!

In addition, all logs are passed through from the subprocess by default, even if --debug is not set. (This can be turned off with --no-logs passed to jhsingle-native-proxy).

Great notebook repo, by the way!