List of trusted/blocked domains and apps to process some links automatically

[CharHeap]

Describe a related problem (optional)

There are links under domains which I trust and which I would like to be just sent to my browser directly, without the need for the URLCheck dialog. There are apps which I consider safe such as my notekeeping app where any links were added by myself and could also just be waved through.

On the other hand there are apps from which I never want to open a link such as my SMS messenger. Here I would like URLCheck to give a notification that the app from which the hyperlink was (most likely accidentally) tapped is untrusted and to automatically reject the request. There could also be a list of untrusted domains which are handled similarly.

Describe your suggested feature

The user gets the option to maintain a list of trusted and untrusted domains and apps which are then handled as described above.

The list of trusted domains should be simple to add to, either prompt the user on every new domain or add a little button to the dialog. How the other lists are assembled could be further discussed.

The lists can then be edited elsewhere and possibly even imported/exported.

Describe alternatives you've considered for your suggested feature

No response

Other details

I am not sure if this is in the scope of this project or if it would make the app too cluttered. But I believe this could make the app less intrusive in the sense that for some links/apps the user experience will be as without the app (just open it, maybe allow for some automatic query removal and application of rules) and (more importantly) at the same time allows for reliably blocking phishing links via e.g. SMS which could still be accidentally opened (although with URLCheck it now takes an additional button press, which is already a lot better).

I'd happily explain and discuss my ideas further with you.

Acknowledgements

• [X] I have searched the existing issues and this is a new ticket, NOT a duplicate or related to another open issue.
• [X] I have written a short but informative title.
• [X] I will fill out all of the requested information in this form.

[pressRtowin]

Some questions:

• I assume if a trusted domain comes from an untrusted app, or an untrusted domain comes from a trusted app, the untrusted attribute takes priority, right?
• If a link matches a trusted domain or comes from a trusted app, what determines the behavior of how (in which app) that link is actually opened? If that behavior is to be saved, it can't simply be just a list of trusted apps or domains. It would have to be a list of (app, app) or (domain, app) pairs that tells URLCheck how to actually handle the link once trust has been determined.
• If example.com is set to always auto open in com.example.myfirstbrowser and all links from the app com.example.myapp are set to open in com.example.mysecondbrowser, what happens when I try to open a link to example.com from com.example.myapp?

Other thoughts:

Maybe two settings could be added to the open/share module (or modules depending on what happens with the open/share split and the ideas for multiple versions of the modules for different UI choices being bounced around) that adds:

1. A whitelist and blacklist of package names along with paired package names for the whitelist
2. A whitelist and blacklist of domains (with regex support for experienced users) along with paired package names for the whitelist

Both could have an option to "quick add" to from the main UI as @CharHeap mentioned. I'm thinking a set of (mutually exclusive) Trusted/Untrusted check boxes for both the source app and domain that, when checked, adds the app/domain (along with the destination app) to the whitelist or blacklist upon selecting the app to open with or choosing to dismiss URLCheck (as you wouldn't select an app if you're blacklisting something).

I'd say make some sort of "Trust Module" that handles this instead to keep the UI (and already very crowded Open/Share module) clean, but I know @TrianguloY has mentioned not wanting to much inter-module interaction/dependency, so I'm not quite sure that's an option.

[pressRtowin]

Also just noting here that trust status would have to be ignored when URLCheck is summoned from an editable text field for the "Apply" feature being discussed here to work as intended.

[TrianguloY]

There are other issue with similar requests like https://github.com/TrianguloY/UrlChecker/issues/21 I'm still not sure how to tackle these interactions so that it allows for multiple different workflows. If possible I will try to start with something simple and improve it later. Something like an auto-open flag which:

• is always false if the url matches a "bad urls" configurable regexp
• can be at true if the url matches a "good urls" configurable regexp
• is always false if the origin app is blacklisted
• can be true if the origin app is whitelisted
• is always false if the first available app is blacklisted
• can be true if the first available app is whitelisted

So, if the flag can be set, the url is automatically opened in the first app from the list (same behavior as auto-clicking the open button).

In the future it could be improved by detecting "warnings" from other modules to disable the flag, or maybe match some urls with some apps. In any case communication between modules is needed, I'm not opposed to that but to keep it to just the necessary (otherwise it's more complex to ensure it all works).

In any case my main concern is that, as the app is programmed right now, this behaviour is not possible unless the open module is the last one on the list (otherwise modules below will not have a chance to analyze the url). I'm planning on a medium refactor that should fix this, but as usual I don't know when I'll have the time to do it :(

[pressRtowin]

There's a question of how that first app can be changed then, if that's not the app you want to open the link with by default. How is the current order determined? I know it's related to usage frequency, but is there a shared global ranking or is it determined per domain?

[TrianguloY]

Right now it's global but configurable to be for each domain.

You are right, apart from the flag something else is necessary. I guess the regexp->app relations could be a solution.

[pressRtowin]

I'm wondering if that could cause any unexpected behaviors. Wouldn't your primary browser slowly rise in rank as it's selected repeatedly to handle multiple other URLs, such that when you try to open an app specific URL (e.g. Twitter), unless you open Twitter links a lot, the Twitter app would fall behind in ranking?

Less of an issue-specific problem if those regexp->app relations relations are used, and it just becomes more of a general UX problem.

On the other hand if both trusted domains and trusted source apps are paired to destination apps, the potential conflict I mentioned before comes into play if a trusted domain comes from a trusted app but their destination apps are different. Some potential options:

1. Make an arbitrary determination of priority, e.g. trusted domain rule will always take precedence (and accept that some users will always wish it was the other way around).
2. Add a priority value as an optional third parameter, which still could lead to conflicts, but at least there's a way for users who care enough to go in and resolve them the way they want to. (Default priority for each app/domain->app pair is 0. Any integer can be added as a priority and in the case a link is both from a trusted app and goes to a trusted domain, the rule with higher priority executes.)

[PabloOQ]

This has been on my mind for quite some time, but I think it might be too complex, still, I'll share it now since we are discussing automatization.

A module, or something different, that sits between the first time the URL is passed to the modules and the initialization of the dialog. Customizable with some kind of structure data file (XML, JSON). The structure would follow a linear order (script like) and each step has different fields:

• Command: Calls a static function from a module to change attributes of the current url (ClearURL, Open with, share to,...).
• Command parameters: Extra info for the command.
• Condition: Calls a function from a info module, based on the result it decides to execute the command or not (Hosts, Patterns checker,...). Can also be used to see how the app was called: Open, share, clipboard shortcut, tile, #191
• Whitelist
• Blacklist

Whitelist and blacklist can call to another file which could have groups of regex to match. I.e.: OpenInApp:[twitter.com, youtube.com], OpenInBrowser:[instagram.com, reddit.com, etc], Maybe with some form of conditional flow (if, elif, else)? After the execution of a command, the "script" would be run again, it repeats until a command with a "finisher action" (open, share, continue to MainDialog) ,it reachs a maximum of loops (to avoid infinite loops), or there are no more steps left. Lastly, a button, to run the "script" automatically from the dialog.

Pros:

• Once implemented it would be easy to maintain, each module and future modules would only need to implement commands or/and conditions, which is just transforming their main purpose to a static method with a return value.
• Abstracts auto-apply, whitelist, blacklist from each module, no need to worry on implementing this on each module. I.e. unshorten module only needs to automatically-apply when encountering shortened URLs.
• Allows to skip what to do with URLs that are encountered daily (social media, job thingies,...)
• Allows to use a module without it being present in the view (i.e. currently if URLCleaner is applied automatically then the view's only purpose is to inform when it matches), making up space for open and share actions (recents discussions have made clear they need more space).
• Customizable

Cons:

• As I said, this seems too convoluted, complex, and maybe overkill, hopefully it sparks some other ideas.
• Lots of refactoring
• TOO customizable, maybe overwhelming (wiki could be very useful).

[pressRtowin]

My crystal ball says there's a "basic/pro" toggle switch in the near future.

[pressRtowin]

Right now it's global but configurable to be for each domain.

I just realized what you meant by this. Now I know what that toggle does. 😅

[TrianguloY]

That automatization sure is another big change, but one I'm starting to like. Right now what I would like to do (and will probably be the first thing to do) is:

(warning: technical details that may not match the code (I don't remember the exact function names))

Right now each modules have a single function onNewUrl, and the app core has a changeUrl one that can be called by anyone at any time (usually at launch and when you press a button). That changeUrl runs onNewUrl for each of the modules in order, and when all are processed, it checks whether changeUrl was called in the process. If so repeats (up until a fixed times, to avoid infinite changes). This has the disadvantage that all the modules will check, analyze and prepare all urls, even if the url is later modified to a different one. This was made to not break modules like the Log module (maybe someone here remember that originally it needed to be the first or it would miss updates). This solution was not perfect, and some modules like the scanner have checks to cancel scanning urls if they are changed immediately. In the original report I even reported a better (and harder) solution. I think it's time to implement it, with some changes:

Now (when I can implement it I mean) each module will have three optional functions: onRecordUrl, onModifyUrl and onDisplayUrl (names may change). The core will then: 1 call onRecord for all modules. This function should be used if a module wants to know all urls that are checked (like the Log module). No url modification are allowed at this step. 2 call onModify on each module. This function should return a new url if needed (or nothing). If a new url is needed the whole process is canceled and starts again on step 1, which means that some modules may miss this callback if not needed. 3 call onDisplay on each module. This function should update the UI or initialize any background process needed. At this step the url can't be changed, so it won't be unnecessary work.

I'm still not sure what to do with calling changeUrl while the loop runs. I guess I'll just disable it (with an error in debug mode).

As for future auto-open, all the work in step 3 will do be unnecessary, so maybe I can include a 2.5 step to cancel the process at that time if the app is meant to be closed. This will be easy to add afterwards so no problem there.

[CharHeap]

I very much appreciate the lively discussion to which I have little to nothing to add as I lack experience with the app (only recently started using it) and don't know anything about its innerworkings. I can answer the initial question from @pressRtowin though: yes, I agree untrust should take priority and I had overlooked the "open in" functionalities which indeed could make things complicated. However you seem to have come up with several ways how that could be handled.

@TrianguloY said you would like to start simple. Implementing just a blacklist could be relatively straightforward and would suffice for now? In my opinion the whitelist is more of a quality-of-life feature than a necessity. But then again you would probably have to rework it all when adding the other automation, so I understand if you just would like to give this more time and thought and do it right.

Again, I very much appreciate your responses to the feature request and am curious what the final solution will look like.

[pressRtowin]

An idea I thought of that is related closely enough to this that I didn't want to open a new issue right away, but also a bit distinct: There may be cases where a you perform a certain set of actions in a certain order within URLCheck quite often (e.g. run url cleaner --> follow a redirect --> follow a redirect again --> remove a certain query) before opening or sharing a link, but you don't want to automatically run it on every link from an app or domain as discussed above.

For that purpose, a macro module could be very useful. It could either be configured from within the module settings, or what might be really neat is to have a small record/stop button directly within the module, allowing you to press record, run the actions you want, press stop, and then be prompted for a name to save it under. The module could then also have a drop down list of saved macros which you could quickly select and run in the future as needed.

[pressRtowin]

Maybe if you leave the macro recording when you select the final action to open with in an app (or long press to share), it immediately asks you for a name to save the macro before executing the final action, then also saves that final open (or share) action as part of the macro.

[pablo03v]

I want that too. For example, my insurance app opens a link every time on start up to authenticate. It connects to a certain domain every time. I trust it, but still need to allow it every single time.

For the meantime I just use Linksheet before URLCheck.

[lucassilvas1]

Any progress on this? I would like to be able to trust hosts and apps as well. I think the feature can be simplified by having just two lists and a toggle:

• Trusted source apps (trust any link that originates from these apps);
• Trusted hostnames (trust any link from this hostname, regardless of where it comes from);

Then a toggle can be used to turn these lists into block lists instead.

[TrianguloY]

No work yet, I've been busy with the backup feature.

I have an idea that I want to try, and that may help with this (and other automation) requests, or maybe not. In any case, I would like now to focus on some bugs that have been reported recently that I think are important to adress. As usual, PRs are welcomed.