search

Ultimate-Hosts-Blacklist / dev-center

The place to talk about our infrastructure or everything related to the Ultimate Hosts Blacklist project.
MIT License
11 stars 2 forks source link

[Feedback] Week 13 of 2019: Tuesday, March 26 – Sunday, March 31, 2019 #28

Closed funilrys closed 5 years ago

funilrys commented 5 years ago

Hello, World!

Please report here every infrastructure/build/test issues (suspected or confirmed) of this week.

Week covered by this issue: Tuesday, March 26 – Sunday, March 31, 2019

Note: Issue that implies big changes or not related to infrastructure or build/test are welcome as a new issue!

Cheers, Nissar

dnmTX commented 5 years ago

@funilrys all the repos that i'm monitoring aren't updating their lists.It keeps on filtering though,Travis Cl is reporting that it's all good and build is passing but no lists on the front page got updated.Please check.

P.S. https://github.com/Ultimate-Hosts-Blacklist/justdomains_mirror1.malwaredomains.com is on it's fifth day since last updated. Definitely something is wrong.

funilrys commented 5 years ago

@dnmTX, https://github.com/Ultimate-Hosts-Blacklist/justdomains_mirror1.malwaredomains.com distributed a new {clean,volatile,whitelisted}.list hours ago.

On my side, after looking at many green builds, everything is normal :)

dnmTX commented 5 years ago

Well...eventually did updated...after five days(i should've take a screenshot so you can see). I'll keep monitoring,looks like it takes much longer for one filtering to finish.

funilrys commented 5 years ago

@dnmTX Yes it take longer but it has nothing to do with our infrastructure, it's all Travis CI :man_facepalming:

dnmTX commented 5 years ago

it's all Travis CI

speed improvements are always welcome 😉

funilrys commented 5 years ago

@dnmTX It's what takes my free time currently :smile_cat:

dnmTX commented 5 years ago

@funilrys let me hear your thoughts on something.It's about adding the www. to every domain,in particular domains that ends with .info .online .biz .top etc etc. They're all either malicious or adware but they always are activated in the background and from my observations so far(going trough many page codes) they never use www..I consider them more like subdomains then anything else.Maybe you can skip the generation of www. on those if it will save some resources. Here is one example: Capture

mitchellkrogza commented 5 years ago

For the hosts file generation the www. could remain as unfortunately wildcarding does not work in hosts.

For domains.list and dotted-dotted-format.list we can probably drop the www altogether as DNSMasq and Pi-Hole (to the best of my knowledge) can be configured to wildcard.

But then again, any domain could just use w.domain.info x.domain.info whatever.domain.info

dnmTX commented 5 years ago

@mitchellkrogza appreciate your input but looks like you misunderstood me on this one. As of recent(https://github.com/Ultimate-Hosts-Blacklist/dev-center/issues/14#issuecomment-463747086) @funilrys implemented new option to add www. to each domain before the filtering starts as many were listed without their www. counterpart,therefore,especially on mobile devices those domains were still accessible(not blocked). We discussed it,winded out the subdoimains(they didn't need it) so i'm just trying to figure out what else doesn't need the extra www. in order to save some resources,possibly speed up a bit the whole filtering.

P.S.

But then again, any domain could just use w.domain.info x.domain.info whatever.domain.info

They don't use it,haven't spotted any subdomains that ends with .info etc yet.I just considering them as such.

funilrys commented 5 years ago

Hi @dnmTX and hey @mitchellkrogza told you you will have a lot to read when you comes back :wink:!

So back to this, I'm not sure it is a good idea to implement such things but will look deeper into this.

Also for the context, it's actually cheap for us to generate all of those www as the system automatically retest them over time if they are INACTIVE.

Cheers, Nissar