A block module based on JS

[737simpilot]

With my idea of blocking the block page to those without JS on, I have another another idea. What if there can be an optional module install to allow to block any request without JS on? I'm sure 9 out of 10 requests all use JS and those without JS are nothing more than bots. I came up with this idea because I think it's damn hard to block every single proxy out there and with my recent hit with someone trying to register only to be blocked by my captcha it might be prudent to just block all requests without JS. This would probably help secure a site where a hacker tries to probe your site. Granted a search engine pass though would need to be crafted.

What's say you?

[DanielRuf]

I'm sure 9 out of 10 requests all use JS and those without JS are nothing more than bots

This is not correct. Many disabled users use special tools and browsers who do not need or support JS and many block it due to security reasons. Punishing people for wanting to be secure (many things can be solved without JS) is a really bad idea.

[DanielRuf]

This would probably help secure a site where a hacker tries to probe your site.

This is not related to JS in general.

[ghost]

What if there can be an optional module install to allow to block any request without JS on?

Many bots parses JS these days as well. I would not rely on that as an indicator.

[dev-101]

In my opinion, this will only put additional and unnecessary maintenance task and move away from CIDRAM's main purpose and goal.

Detection of JS disabled with PHP (server side) cannot be done on "first load", as first load is essentially the page request itself. So, you need to add a landing generic page, processing and a redirect, which is a bad idea.

[737simpilot]

The module I propose is optional to install and use. It's not forced on anyone. If anyone ops to block the parsing of JS in their browser that's on them. I did this largely with NoScript, but it broke many websites and now a days many websites are JS heavy. So now I use Sandboxie for my browser and allow all JS to run. Likewise, my website's all use JS, so if a user blocks JS they can't sign up or even interact with the website. I find that anyone blocking JS is probably more or less using Tor, and since I already block Tor it's a moot point.

I feel the main reason to have an optional JS block module would be to help mitigate bots that don't parse JS. I just had a proxy successfully connect to my site and it was only stopped at the registration page because it couldn't use the captcha extension I have installed. It was very disconcerting to see this freely available proxy server make some shady requests. Even though I use CIDRAM, Ninjafirewall, CloudFlare and have mod_security installed I believe in layers in terms of security. CloudFlare does have a JS check, but it's like half ass. All it does when JS is off is run down a small timer and then allows the request to go through. I can see the point in the timer, but it's not near as long enough for a bot to simply give up. Despite that some ASNs that I block with a JS challenge that may belong to cell phone providers are JS checked. But like I mention, it's like a 5 second timer before the page finally loads.

Reemphasizing this. This module, if created is an optional one. It's not forced for anyone to install like the other modules. You can enable them as you please.

[DanielRuf]

In my opinion, this will only put additional and unnecessary maintenance task and move away from CIDRAM's main purpose and goal.

Definitely out of scope imo.

I guess you want to add ZB Block features to CIDRAM. CIDRAM in general should only block based on CIDR rules.

[Maikuolan]

Just replying quickly to let anyone concerned know that I've seen and read this issue. Will share my views in more details possible tomorrow morning or thereabouts (plus catch up on the various other issues which I haven't yet properly responded to). Just been a little tired the past few days due to unrelated things which have kept me busy.

[ghost]

CIDRAM in general should only block based on CIDR rules.

We are already way beyond that, both Maikuolan and I supply blocks based on UA, hoster and some security checks. So that point is moot.

But I agree that we should keep CIDRAM slimmed down enough to be a stable and trustworthy core. It is after all easy to write up own modules and add-in for anyone who needs a special function.

[737simpilot]

But I agree that we should keep CIDRAM slimmed down enough to be a stable and trustworthy core. It is after all easy to write up own modules and add-in for anyone who needs a special function.

I'd code it myself, but I don't know PHP. Like I said, it's really optional as to whether anyone want's to install it or not. I do realize it is beyond the scope of CIDRAM.

[Maikuolan]

Definitely would not be integrating this into the core, but the idea, in theory, insofar as writing it as an optional module, could be usable. The concept that bots probably interpret JavaScript and other such things differently than browsers, is something that most of us have likely exploited at some point or another, to some extent (I know that I've done so, at least).

As a catch-all approach to covering all parts of a website though, there are quite a few boxes we'd need to tick I think, in terms avoiding potential problems that could come with it (e.g., potentially accidentally blocking search engines, users on devices which don't have JavaScript enabled, like HipTop phones, some types of screen readers and such, the risk that it might simply be ineffectual in some cases, where some of the smarter bots are beginning to be able to interpret JavaScript and such things in an attempt to work around similar such measures implemented elsewhere already in the past). In the past, I've tried to exploit differences between how browsers work versus how bots work to exclude possible human users from being able to accidentally trigger honeypots, to protect login and registration pages and such things, but not so much as a catch-all measure (somewhat different considerations, I think).

If being able to tick all these boxes per se, in order to be able to come up with something effectual, ends up becoming a complex task (e.g., consisting of multiple "concerns", in the programmatic sense, or requiring multiple operation phases in order to function correctly), or ends up requiring regular maintenance in order to remain effective into the future (e.g., if such a module ended up needing its own, specific set of signatures, in order to identify different entities which must be treated differently than others, such as to avoid accidentally blocking search engines, or any known good entities which aren't able to process things like JavaScript), it may be better to approach the idea as an entirely separate project, rather than as a module.

Approaching the idea as an entirely separate project (i.e., starting a new, separate package, rather than just creating a new module), would at least go some way to preventing any maintainability problems from bleeding across into the CIDRAM project, and would resolve any questions that anyone might have about scope (i.e., preventing scope-creep in CIDRAM to some extent, and preventing it from becoming an unmanageable mess, which isn't necessarily a problem for CIDRAM currently, nor necessarily a problem from implementing just one or two new ideas, but something which could maybe happen in the future if all new ideas which might be suggested from everyone in the future get implemented, so I mention this as a general thing, rather than something specific to this particular idea).

An example of the "unmanageable mess" thing: I had some users in the past suggest possibly merging the CIDRAM and phpMussel packages into one, singular package. I do often develop and work on both of these packages side-by-side (particularly when it comes to new front-end features), and there are many similarities, as well as quite a lot of duplicated code and documentation, but despite that, I think that merging them entirely would be a bad idea, due to that merging them wouldn't only be merging their capabilities, but would also be merging their complexity, their ongoing maintenance cost, and any possible future technical debt that might be incurred by handing off such a project to other developers in the future, and plus would completely blur their originally intended scopes and purpose, possibly eroding trust in the eventual merged package. Obviously there is a huge difference between that example, and the idea actually proposed in this case (I don't mean to imply that this idea would be anything on that scale), but I mention this example purely to elaborate on what I mean by "becoming an unmanageable mess".

In any case, if this idea was to be implemented somewhere somehow (whether as a module, as a separate package, or whatever else), the first step would be to come up with some workable test cases, to be able to demonstrate an implementation of this idea which would actually be effective in production at all. I suspect that this is something which would probably take some time, due to that any such test cases would likely need to be tested in a live environment (though preferable not on any important, commercial websites or such things), to be able to see how effectual they are against actual bots in the wild right now, and to see how well they stand up against possible negative consequences like damaged search engine reputation and such.

I could look into this at some point, to try to come up with something workable. Ideally though, I would like to try working through more of the project backlog before working on new ideas like this (there's still some work that needs to be done on CIDRAM's new two-factor authentication feature, for example, plus some other things which I'd already promised to get finished before the next version is released).

As always though, and as with any of the other ideas and suggestions proposed for the project thus far, if anyone else wanted to have a crack at it, they're welcome to do so. :-)

The TL;DR of it though: Whether the idea gets canned, or eventually implemented, instead of deciding right now, my thoughts on the idea currently are to just leave this issue open for now, give it an extended "planning" period, let it be discussed, and if someone wants to have a crack at it, we'll see what they can come up with and judge it by the results they can show for it. Otherwise, if nobody else wants to have a crack at it, I could give it a go once I've gotten through some of the current project backlog, or if there isn't any interest in the idea anymore by that point in time, we just close the issue.

Hopefully the interest in this idea isn't urgent at this point, but I'm leaning this way in part because I'm a little concerned about falling behind on development due to doing too many things at once (not just here, but with other projects elsewhere as well, plus balancing time between these various projects, actual paid work, other commitments and so on).

[737simpilot]

No problem. I'm not in any rush to have a JS blocking module. I was thinking perhaps the module could somehow use the same code as search engine verification for bypasses. So if a search engine comes along without JS it's let through.

A new project just for blocking JS sounds alright I guess. I was thinking about a phpBB extension and a plugin for Wordpress since those are the two platforms I run.

[737simpilot]

Just checking the statues on this. I know, I'm a PITA, but it's almost been a month since my last comment.

[DanielRuf]

Nothing new and I still see not much sense in this feature request.

[Maikuolan]

Haven't looked at this yet since my previous reply. Quite a few other higher priorities at the moment, so it may be a while longer before I look at this more deeply.

[Maikuolan]

So.. It's not a module, and in fact, actually something integrated into the CIDRAM core itself, but.. Due to #211, I think we can probably now mark this issue here as implemented/resolved and close it, since we now have a way of using JavaScript (given that CAPTCHAs are effectively a kind of JavaScript challenge in their own right) to test every single inbound request if so desired.

@737simpilot What do you reckon?

[737simpilot]

Hey Caleb. Long time since I created this suggestion.

The reason why I brought this idea up was that Cloudflare has it and DOES work. Cloudflare can use a JS challenge or a captcha challenge currently. The captcha they used to use was Google's. Now they use Hcaptcha. Back when I created this issue, Cloudflare's JS challenge was broken, and I even told them about it. What would happen is if you had a JS check on and disabled JS in the browser, when you fetched your website you'd get the JS check like you're supposed to, but in about 5 seconds you were let though. I thought that was totally stupid how they implemented it.

Now Cloudflare has their JS challenge implemented like it should have been. If you don't parse JS you're 403ed. No 5 second timer crap.

Now I've seen the effectiveness of a JS challenge first hand with whole ASNs being JS challenged where they contained host and residential end points. So while there are bots, scripts and what not that may parse JS, not all do.

While a Google captcha is using JS, the principal of a JS challenge and a captcha are different. A JS challenge will be less cumbersome to the user where a majority of browsers already parse JS by their very nature. So this would be an indication the connection is good based on that single metric. With a captcha, that could be cumbersome if it's not an invisible captcha. And many people probably don't like solving captchas. Also, if I can remember, an invisible captcha is not as strong as a normal captcha from what I read a Google. For that reason, it's why I use version 2, none invisible captcha. And I do like to balance security with ease of use so the captcha is only used for certain things that need a little more security and of course its fired off if a dat file calls for it.

While Cloudflare can do this, unfortunately it's not like CIDRAM's auxillary rules with the use of wild card URLs. For that you'd have to JS challenge whole countries, and I don't think the free Cloudflare account allows that. I'd have to check. But doing that you'd have to create some 100+ rules in the Cloudflare firewall. LOL.

So having said all that, I still think a JS challenge module would have its merits based on the differentilality of the two types of challenges I explained above.

Now as noted from a commenter, this probably would require the use of some code that is JS to be fetched and if that fails a 403 (or whatever web code the CIDRAM user uses) is issued. So I don't know if this would be hard to code or not, especially with the module being interlaced with search engine verification.

So I guess I'm tentative on the module idea. But at the same time, I do know the real PITA connections may be hacked ISP routers from a script here at Github, and turned into a zombie botnet. So we see here that unless CIDRAM can utilize either the Shodan, Census and Greynoise APIs (and willing to pay for those APIs), making sure a residential IP is not a zombie botnet is problematic. A JS challenge could be another tool in the arsenal where a lot of these hacker Python scripts may not parse JS for their connections.

I typed all this on my phone so there may be grammical or spelling errors.

[737simpilot]

I think I'll look at the invisible captcha again and run it through the ringer looking for possible flaws with certain page requests and altered headers and what not to see it it'll stop me.

[Maikuolan]

Sounds good. :-)