search

atlas-engineer / nyxt

Nyxt - the hacker's browser.
https://nyxt-browser.com/
9.87k stars 413 forks source link

Adblock/Umatrix like functionality? #32

Open schvabodka-man opened 6 years ago

schvabodka-man commented 6 years ago

Hello. Are there plans on implementing per domain based settings to block stuff like ads, javascript, pictures, etc like umatrix?

jmercouris commented 6 years ago

Hi, thank you for opening your issue! I share your sentiment, as a user, Adblock is immensely important to me/blocking images/js etc

So to answer your question, yes there are definitely plans for this, I think probably in version 0.07 or 0.08! I'll keep you updated!

-John

stellarstriker commented 6 years ago

+1

jmercouris commented 6 years ago

Hey everyone, just to provide a quick update on this. I am still working hard on the new version which will finally enable this functionality. I anticipate a release within a couple of weeks. Thank you for your patience!

-John

impaktor commented 6 years ago

Will it be some native solution, or will it rely on existing addons, like uBlock origin or Adblock plus? The former being much preferred.

jmercouris commented 6 years ago

Well, it turns out that my integration efforts have been in vain, because I'd have to make my own webkit port on MacOS to actually use the purported content blocking rules (only present in Safari), if any one has any other information they are privy to, or knows how to do it, please let me know.

As of now, the follow up plan is to implement advertisement blocking by updating the hosts file, not exactly the most elegant, but it works...

jgkamat commented 6 years ago

For simple host based blocking (at least), this can be done a bit cleaner (and without editing the system hosts file, which will probably need root) in qtwebkit by setting a network access manager which you define yourself to block whatever domains you want. This the example of how qb does this on qtwebkit. I'm pretty sure there should be something similar in webkitgtk, as luakit has a hostblocker built in, but I'm much less familiar with that (you might be able to get some pointers by reading the relevant sections of their source!)

jmercouris commented 6 years ago

Hi, to everyone wanting this ad-blocking feature. I am hard at work, but I need your help to make it happen, as part of a new architecture, I did finally figure out how to actually make ad-block possible.

If you would be able to, please support, or spread awareness about my crowdfunding campaign so that I can pay the C/C++ developer who I'll be working with, thank you for your time!

https://www.indiegogo.com/projects/next-browser-nix-support#/

Ambrevar commented 5 years ago

I'm on this! Expect at least a host-based blocker + image blocker + javascript toggle from the work I'm going to put on this in the coming days :D

Ambrevar commented 5 years ago

I've just pushed the resource blocker. Javascript toggle coming soon. Not too sure about the image blocker actually, does anyone know how to deal with image in WebKitGTK?

impaktor commented 5 years ago

Nice! I assume it will be documented as well?

Ambrevar commented 5 years ago

It already is (a little): https://github.com/atlas-engineer/next/blob/master/documents/MANUAL.org#resource-blodking-ad-blocking

whirm commented 5 years ago

Would it be hard to integrate this: https://github.com/brave/ad-block ?

Ambrevar commented 5 years ago

We can't (or don't want to) "integrate" brace/ad-block directly since we would like to stick to a full-Lisp code base as much as possible, but sure the current algorithm can be expanded to follow more closely what Ublock-origin and brace/ad-block do.

It's a bit of work, but certainly doable. Specialists in ad-blocking are welcome to pitch in :D

Ambrevar commented 4 years ago

See also #464.

whirm commented 4 years ago

Does this issue have any priority? I'd really like this to be implemented as It's the only thing preventing me from switching to Next as far as I've seen.

Ambrevar commented 4 years ago

It has high priority indeed, post 2.0 (which is due in hopefully just a few of weeks).

One way to handle it would be to implement support WebExtensions, then we would get uBlock for free.

Help is more than welcome! :)

dudik commented 4 years ago

Hello. I'm working on a WebKitGTK extension similar to uBlock Origin as my bachelors thesis. I wanted to ask if Nyxt supports loading WebKitWebExtensions written in C/C++. Based on this cl-webkit code I think it should, but I want to be sure so I can potentially refer to Nyxt as a reason for why the WebKitGTK ecosystem would benefit from having such an extension and that the user base of these browsers is not negligible. Thank you.

Ambrevar commented 4 years ago

We were just looking for a reason to merge WebExtension support!

https://github.com/joachifm/cl-webkit/pull/41

@aartaka Any comments?

aartaka commented 4 years ago

It seems we found the reason to complete WebKitWebExtension support :tada:

I can look into it next week and test it with example extensions I found. Shouldn't be hard, so by the end of the next week we should have support for WebKitWebExtensions loading!

@dudik, thank you for this motivation!

dudik commented 4 years ago

It makes me extremely happy to see your enthusiasm. Sadly, I have just started writing my thesis and the first half is centered around theory. I'm also still in the middle of a normal semester, so I won't be able to show you anything worthwhile until February. Maybe sooner, but it's hard to say right now. I made some research and it is also in my own interest to have a WebKitGTK content blocker, so I'm positive about creating something useful. I'm really sorry, but also very thankful for the motivation.

aartaka commented 4 years ago

No pressure, you're not obliged to work on it just because we found your idea to be in-line with what we do :)

In any way, good luck!

aartaka commented 3 years ago

@dudik, good news: we've merged the WebKitWebExtension support in cl-webkit! This means that Nyxt can use WebKit extensions and you can refer to Nyxt as a use-case for your thesis :)

kskyten commented 3 years ago

Qutebrowser recently merged an adblocker based on Brave qutebrowser/qutebrowser#5317. Perhaps worth looking at how they did it. As far as I know, it is implemented in Rust, so it would add a non-lisp part into the code base, though.

jmercouris commented 3 years ago

It should not be part of the core, we would like to support the actual webextensions API so firefox and other plugins can be easily used and ported

jmercouris commented 3 years ago

In other words, no Nyxt specific hacks should be necessary

jmercouris commented 3 years ago

congratulations to the team for the achievement!

dudik commented 3 years ago

So after working on a pure C adblock for a month, I realized that because of performance reasons, I would have to implement it with inter-process communication in mind. That gave me the freedom to use any language I want so I ended up putting the C adblock aside and just creating a "bridge" between the Brave Rust adblocker and the C extension. This is the result. At this time it's pretty barebones and the code quality is questionable, but it's working both for cosmetic and network filtering so I felt it was the right time to make it public. While writing this comment I realized that I'm not freeing some memory in the C extension. This will get fixed tommorow when I wake up as I'm pretty sleepy. (it's fixed) Anyways, in the meantime I would like to hear if it works on your platform and how well. Thanks!

Ambrevar commented 3 years ago

Thank you for much for sharing your work!

I just had a look at the code: so it looks like if we want to integrate this in Nyxt, we just need to write FFI bindings to void webkit_web_extension_initialize(WebKitWebExtension *extension), is this correct?

aartaka commented 3 years ago

No, we needn't :)

This function is an entry point for all WebKit extensions, so the only thing we need is to compile the extension, put it where Nyxt can find it (see WebKitWebExtension documentation), run adblock server, and enjoy the extension being enabled in Nyxt on the next startup.

EDIT: don't -> needn't

Ambrevar commented 3 years ago

You mean we already have this binding?

aartaka commented 3 years ago

Yes and no 🙃

We don't have it as a CL-CFFI binding, but we don't need to have it because WebKit manages its extensions automatically. The only change to Nyxt that we need to do is to set the extensions directory. This is trivial, I'll open the PR, most probably in a couple of hours.

EDIT: it's -> its :/

whirm commented 3 years ago

@aartaka are you still planning on open that PR? Cheers!

aartaka commented 3 years ago

@aartaka are you still planning on open that PR? Cheers!

Actually, it's long merged :D

The directory is .config/nyxt/extensions/. However, it's likely to change soon (as per https://github.com/atlas-engineer/nyxt/pull/1505), so keep an eye on the gtk-extensions-path of your Nyxt buffers :)

whirm commented 3 years ago

Amazing, I'll give it a go ASAP. Thanks a lot!

crocket commented 3 years ago

Wouldn't it be possible to write extensions in common lisp?

Ambrevar commented 3 years ago

Of course, and some exist already! See https://github.com/atlas-engineer/nyxt/blob/master/documents/EXTENSIONS.org.

As for ad-blocking, the problem is mostly with maintenance; it's a huge endeavor. We would be better off reusing a well maintained ad-blocker than rolling out our own.

crocket commented 3 years ago

Qutebrowser uses brave browser's adblocker written in Rust.

Ambrevar commented 3 years ago

Indeed, that's also an option that was mentioned in the past.

jbmestelan commented 3 years ago

As an extra reference: the Webmacs alternative browser - written in Python as a replacement for Conkeror - also uses Brave browser's adblocker. The code is pulled as a submodule, then compiled here: https://github.com/parkouss/webmacs/blob/master/setup.py

and used in this way: https://github.com/parkouss/webmacs/blob/master/webmacs/adblock.py

crocket commented 3 years ago

I think uBlock Origin is the only extension I really need. You don't need WebExtension to get advanced adblock. Other extensions can be written in common lisp.

Ambrevar commented 3 years ago

Isn't uBLock Origin based on WebExtension? How would you load uBlock Origin in Nyxt?

crocket commented 3 years ago

Brave's Rust adblock library is almost the same as uBlock Origin.

arne-cl commented 3 years ago

The directory is .config/nyxt/extensions/. However, it's likely to change soon (as per #1505), so keep an eye on the gtk-extensions-path of your Nyxt buffers :)

@aartaka Could you please give an example of how to load a WebKitWebExtension in Nyxt?

aartaka commented 3 years ago

@aartaka Could you please give an example of how to load a WebKitWebExtension in Nyxt?

Sure! I assume you are aware of the fact that WebKitWebExtensions are .so C libraries. The steps are:

One concern of mine, that we currently discussed in #1781, is that it can actually not load the extension due to WebKit being picky about the directories. To trespass the restriction, you can do export WEBKIT_FORCE_SANDBOX=0 (or add (setf (uiop:getenv "WEBKIT_FORCE_SANDBOX") "0") to your config), until we come up with a better solution. Beware: this can disable extension sandboxing, so be careful about what you install!

EDIT: Change the directory to what it currently (Jan 2022) is.

crocket commented 3 years ago

Which one between WebExtension and Rust adblock library is better?

aartaka commented 3 years ago

Which one between WebExtension and Rust adblock library is better?

I'd say WebExtensions, as those can hijack JS requests and modify the page opened in the browser, while any external adblocking library is usually restricted to host filtering. However perfect host filtering can be, that's not the only way tracking/advertisement works. Thus, WebExtensions are a priority.

However, if you want to use Brave's adblocker, you can try installing blockit as a WebKitWebExtension. I haven't managed to wrap my head around Rust building and thus haven't tried it, but it should run fine in Nyxt.

crocket commented 3 years ago

I'd say WebExtensions, as those can hijack JS requests and modify the page opened in the browser, while any external adblocking library is usually restricted to host filtering.

Perhaps, that's why qutebrowser can't do cosmetic filtering, yet?

crocket commented 3 years ago

blockit already supports cosmetic filtering. I think people should test nyxt with blockit.

glyh commented 2 years ago

I think it'll be better if we support web-extension apis, and then a lot of functionality can be done at the plugin side rather than embrace everything in the browser itself, which can ease the pain of maintaining a big project.

aartaka commented 2 years ago

Now that #1762 is merged, there's a solid base to build uBlock support on top of. Here's a list of things to do for uBlock to work (copy-pasted from the linked pull request):