progress ipywidget should handle cancelled futures

[ogrisel]

As of now, the progress notebook widget seems to treat all cancelled futures in the same way as it reported pending futures:

• the time spent counter goes on indefinitely as if waiting for the cancelled tasks to process,
• the progress bar show white background for remaining tasks (that are all cancelled).

[ogrisel]

This is a not a big deal but I think this feature request could be labeled "help wanted" in the tracker to emphasize this issue to people who would like to start contributing to the project.

[jakirkham]

Agree we should do something about this. The situation is worse for the non-ipywidget progress bar where it blocks indefinitely.

Can think of a couple options. There may be others.

1. Treat "cancelled" like "finished".
2. Ignore "cancelled" and adjust number of Futures accordingly.
3. Raise a CancelledError on first "cancelled" seen.

Thoughts?

[mrocklin]

I would have expected this to err immediately. This makes sense for dask collections (array, dataframe) although it may not make sense for more real-time applications with futures, for which options one or two seem to make the most sense.

I don't have enough experience with workloads that cancel features to have strong opinions on this one. Happy to defer to others' judgment.

[jakirkham]

If Dask collections start submitting tasks within tasks, one or two might be reasonable as well.

We could also compose the options in different ways as well. For example still show progress continue, but log about cancelled errors next to the progress bar. Could also add an option to suppress this logging behavior should cancellations be expected.

[mrocklin]

Note that the current ipywidgets progress bar doesn't track new futures, only those that are given to it. For the full state of all tasks we point people to the dashboard, which is probably the target of most UI development these days.

On Sat, Mar 24, 2018 at 7:32 PM, jakirkham notifications@github.com wrote:

If Dask collections start submitting tasks within tasks, one or two might be reasonable as well.

We could also compose the options in different ways as well. For example still show progress continue, but log about cancelled errors next to the progress bar. Could also add an option to suppress this logging behavior should cancellations be expected.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/dask/distributed/issues/289#issuecomment-375932701, or mute the thread https://github.com/notifications/unsubscribe-auth/AASszFO8tuGIL6zr09O8m0U7Kdpmne6cks5thteHgaJpZM4Il7bH .

[max-sixty]

Relatedly — I found it surprising that a single future error-ing will cause distributed.progress to exit, while distributed.wait waits for all to complete — I had expected those to behave similarly. (After the initial confusion with ProgressBar(): distrubited.wait(fs) works fine though...)