search

micro-analytics / micro-analytics-cli

Public analytics as a Node.js microservice. No sysadmin experience required! 📈
MIT License
734 stars 39 forks source link

IP Blacklist #11

Open AriTheElk opened 7 years ago

AriTheElk commented 7 years ago

Obviously this could be implemented in the client-side on a per-project basis, but I think it would be a decent idea to implement some sort of IP blacklisting system so that stats don't get bloated during development/QA.

Maybe something along the lines of providing a .ipignore file that follows the same general rules of .gitignore files?

127.0.0.1
192.102.*
*.512.*

Again, not sure if you're trying to keep this as micro as possible leaving things like this up to the individual developer. But seems like a feature that many could benefit from.

mxstbr commented 7 years ago

I was going to say this shouldn't live in core and people should take care of that themselves, but then I realized that you cannot get the IP address of the user in e.g. the browser context.

Is there a module that contains some well-known IPs that shouldn't be tracked like e.g. localhost, the Google/Bing/Yahoo bot, etc.?

AriTheElk commented 7 years ago

Here: http://www.iplists.com/

My only concern with using a static list, is that I think it might be unfair to block IPs like localhost out of the box. If I find this library and install it, I'm going to have expectations of seeing my page loads getting tracked (even if I don't want this in production). If I install it, load my page, and see no activity tracked; I'm going to assume that it's not working properly.

It's probably a good idea to block IPs of web crawlers out of the box though.

mxstbr commented 7 years ago

Let's check for NODE_ENV=development and not block, but if it's production we block localhost, crawlers etc? (possibly with an option to disable in prod, --no-ip-block or something)

sean-roberts commented 7 years ago

bringing the conversation here. I think allowing some sort of extension module by the user would be ideal for this and any other situation that is general or unique to the user's needs.

So the proposal is to allow the user to specify micro-analytics --extension=analyticsExtension.js naming totally open for grabs.

That module would be able to export any extension api that we would want configuration on. But for this and favicon issue we could allow them to export a filterInput function that can receive the request and if they return true, it will be passed into the ingestion of core for processing.

sean-roberts commented 7 years ago

another tool to help users filter https://referrerspamblocker.com/ they have a solid list of referrers that could be filtered or blacklisted