Handling of background requests/downloads

[The-Compiler]

I'd really appreciate some input on this:

Currently, no automatic background actions are done in qutebrowser. Those are currently necessary for adblocking, and probably soon to get a TLD list as well (see #731).

There are various problems with this approach:

• Some users don't notice the "Run :adblock-update to get adblock lists." notification and think qutebrowser doesn't have adblocking capabilities.
• Users rarely/never update the adblock lists.
• For the TLD list, it doesn't make sense to have a dedicated command just for that.

Another solution would be this:

• Background tasks are shown by some spinner in the statusbar (as it'd be confusing to have the download bar appear automatically)
• By default, the background tasks would run on the first start, and in a fixed interval (e.g. every week).
• There's a setting to customize the interval or deactivate it.
• As long as the block lists/URL lists are not available, some fallback is used (no adblocking, QUrl::topLevelDomain with Qt's lists).
• Or maybe ship a default adblock/URL list with each release which then gets overwritten on updates?

This would mean the user doesn't have to think about updating these things, but it also means the browser automatically attempts to download a few MBs on the first start...

[arp242]

There's already a download manager which runs in the background. If you add a download_finished_callback (to emit some signal to signal that the adblock/suffix list should be re-read) and download_hidden option, then that would be enough for the adblock update and toplevel domain update?

[The-Compiler]

Yes, that's how the adblock updating works currently (modulo the download_hidden thing).

The question is more a policital/philosophical than a technical one.

[lamarpavel]

So the question is when to do the background tasks and how to present them to the user?

I would not bother the user whenever updates are available, that is rather annoying and may lead to people ignoring them. Do it after qutebrowsers startup. That means at least once every update of the browser, though most people probably restart their applications more often.

For the question on how to present them to the user I have a new take on this: Other programs do these things via desktop notifications, but they have several drawbacks:

• dependant on the users system configuration
• annoyance
• easy to miss if you don't look at the screen for a few seconds
• might vanish before the user can read them
• you are limited to a short text, which might confuse rather than inform the user what is happening

Instead add "notification tabs" for that purpose. Meaning a new tab that opens (in the background) but that stays open until the user selects and closes it. The tab would be color-highlighted and named Update notification or something along these lines. Closing the tab accidentally before reading it would be no problem as they could be re-opened via :undo and on that tab we have all the space we need to explain what is happening. For now this could be as simple as showing a message such as

There are updates for your adblock-list, you may download them with :adblock-update or configure qutebrowser to do this automatically via :set adblock auto-update.

This could later be expanded to show a more sophisticated page that sums up the status of all plugins and more.

[The-Compiler]

So the question is when to do the background tasks and how to present them to the user?

Exactly - and mainly if it's appropriate to download some small files without the user's explicit consent, but in return keeping things up-to-date.

Instead add "notification tabs" for that purpose. Meaning a new tab that opens (in the background) but that stays open until the user selects and closes it. The tab would be color-highlighted and named Update notification or something along these lines. Closing the tab accidentally before reading it would be no problem as they could be re-opened via :undo and on that tab we have all the space we need to explain what is happening.

To know that there are new adblock updates, I already have to download the lists anyways, so I think there isn't any need for such a notification, and the downloaded list could be applied automatically.

The main question really is if it should be periodically downloaded automatically, and how to present that to the user (e.g. with a spinner in the statusbar). It's a tradeoff between "the user should be in control" and "things should be up to date even if the user is too lazy".

[lamarpavel]

I would generally prefer these types of checks to happen only when I start a program, because then I'm not in any kind of workflow and notifications don't bother me as much. I would also like to give consent at least once, that includes knowing that there is something to give consent to.

Right now the user has to manually invoke :adblock-update, this could implicitly enable the auto-update when executed the first time. When qutebrowser is started for the first time (or without config) and the help page is shown, there could be a note on the help page that invoking :adblock-update would enable automatic updates for it, which can be disabled in the settings.

It would generally make sense to have a section "important decisions" on the help page that lists all the options that require consent while showing their default/current value.

[amacfie]

Just for completeness, a very rough workaround is for users to schedule adblock updating with cron or jobber.

[crocket]

@amacfie How can I make cron execute adblock-update for qutebrowser?

qutebrowser --nowindow :adblock-update

doesn't quit after executing adblock-update.

[crocket]

Doesn't python have scheduler libraries? It should be possible to allow people to configure qutebrowser to run commands on regular intervals. High-level options like adblock.auto_update can utilize qutebrowser's scheduler.

[The-Compiler]

There isn't really a need for a library, Qt's QTimer is probably enough for this.

[crocket]

My concept of scheduled tasks.

scheduled_tasks.internal.monthly: [":history-sql-vacuum", ...]
scheduled_tasks.user.daily: [":adblock-update", ":command2", ...]
scheduled_tasks.user.weekly: [...]
...

With this, users can see and configure qutebrowser's internal scheduled tasks and their own tasks.

[crocket]

I also recommend scheduling tasks executed when qutebrowser launches or exits.

scheduled_tasks.internal.on-startup: [":cmd1", ":cmd2", ...],
scheduled_tasks.user.on-startup: [":cmd1", ":cmd2", ...],
scheduled_tasks.internal.on-exit: [":cmd1", ":cmd2", ...],
scheduled_tasks.user.on-exit: [":cmd1", ":cmd2", ...]

[The-Compiler]

FWIW anything more generic and based on commands will first require a way for commands to signal that they are finished, see #3007. I'm also not entirely convinced that this is useful in general, I imagine the commands suitable for background tasks like those are rather limited.

[crocket]

You can also allow users to schedule tasks in python code rather than commands.

I would like to configure background tasks.. in some ways.

[The-Compiler]

That'd be something for the extension API (#30), but I'd rather just start with a way to cover the most common use cases (mainly adblock list update, perhaps sanitize-on-exit) rather than over-engineering this in a way that it requires a lot more work before it's possible.

[crocket]

Sure. I'd like to configure automatic adblock update on a regular basis and perhaps make qutebrowser wipe history on exit.

[crocket]

I found a workaround for automatic adblock update.

~/.local/bin/qutebrowser

#!/bin/sh

exec /usr/bin/qutebrowser :adblock-update "$@"

[amano-kenji]

My workaround for the lack of automatic adblock update is to disable adblock in qutebrowser and update dnsmasq block list everyday through a daily cron job. This blocks domains only, but I don't have to manually manage anything.