search

Ultimate-Hosts-Blacklist / dev-center

The place to talk about our infrastructure or everything related to the Ultimate Hosts Blacklist project.
MIT License
11 stars 2 forks source link

[Feedback] Week 12 of 2019: Monday, March 18 – Moday, March 25, 2019 #24

Closed funilrys closed 5 years ago

funilrys commented 5 years ago

Hello, World!

Please report here every infrastructure/build/test issues (suspected or confirmed) of this week.

Week covered by this issue: Monday, March 18 – Monday, March 25, 2019

Note: Issue that implies big changes or not related to infrastructure or build/test are welcome as a new issue!

Weekly Note: As I took some break out of OSS development last week, I'm a bit late but all issues from the last week that are still not fixed were flagged/moved into a new issue.

Cheers, Nissar

dnmTX commented 5 years ago

Reporting: Found one more,this one has been stuck for a 2 months 😃 https://github.com/Ultimate-Hosts-Blacklist/Global-Advert-Servers-Blocklist---Personal-Edition

This one has no repo,clicking on it takes you straight to the source's website: Ultimate Hosts Blacklist

Same with this one: Blacklist

dnmTX commented 5 years ago

@funilrys as a suggestion,you can skip the generation of volatile.list and whitelisted.list on all the repos which host only IP addresses if it saves some resources of course. Also the generation of /output/hosts/* is not needed as well. You know,such a thing will never work in any hosts file: 0.0.0.0 103.27.200.66

funilrys commented 5 years ago

Hi @dnmTX, as you may have seen (cf: https://pypi.org/user/mitchellkrog/ && https://github.com/Ultimate-Hosts-Blacklist/dev-center/branches/active) , we (mostly I :wink:) are moving all our scripts into one place so that it easier to maintain.

Indeed, I do know my code but it's hard to maintain accross all input sources. Because they copy the scripts at the repository-structure and run. I do not like that design. Because in case of a major upgrade or GitHub is down or whatever, nothing works.

So, I'm taking some good quality time to unify everything so that it's easier for me (and us in the future) to add new features or fix bugs and improve perfomance accross all the scripts/tools we have into our infrastructure. That's my priority for now.

shallalist is in the top5, blocklist too :joy_cat:

When I'm done with the migration to PyPi, I'll add a proper handler for those cases after fixing the other issues :+1:

dnmTX commented 5 years ago

I'm just throwing some ideas so they're documented for any feature enhancements. Good luck with all those tasks ☝️ . I'm here if you need me 👍

funilrys commented 5 years ago

@dnmTX Thanks! Can you drop them all at https://github.com/Ultimate-Hosts-Blacklist/dev-center/projects/2 (freshly created :wink:)

dnmTX commented 5 years ago

DONE and DONE. I like that PROJECTS repo already. 😄

funilrys commented 5 years ago

@dnmTX If you have time can your review/fix https://github.com/mitchellkrogza/Ultimate.Hosts.Blacklist/issues/505 for us ? Feel free to create PR to the whitelisting list or directly push :)

Reminder: We read/use the domains.list of https://github.com/Ultimate-Hosts-Blacklist/whitelist for whitelisting :wink:

dnmTX commented 5 years ago

How about i'll review it and post my findings in the same issue so at least you or @mitchellkrogza have a look before merging them into the whitelist? I kind of feel better with more eyes when it comes to especially whitelisting.

dnmTX commented 5 years ago

Reminder: We read/use the domains.list of https://github.com/Ultimate-Hosts-Blacklist/whitelist for whitelisting 😉

This goes straight to the project repo: Don't generate any lists or directories in the WHITELIST REPO as they're not used anyway.

funilrys commented 5 years ago

Than create a new PR at https://github.com/Ultimate-Hosts-Blacklist/whitelist and link it to the issue. That way we can edit/review the changes before merging into the whitelist list!

funilrys commented 5 years ago

@dnmTX Will be done in the coming hours I'm pushing a new version of the whitelisting script asap :wink:

dnmTX commented 5 years ago

Than create a new PR at https://github.com/Ultimate-Hosts-Blacklist/whitelist and link it to the issue. That way we can edit/review the changes before merging into the whitelist list!

Yep,that will work 👍

@dnmTX Will be done in the coming hours I'm pushing a new version of the whitelisting script asap 😉

I guess you one step ahead of me 😃

P.S. Tomorrow as the latest i'll review those domains in the issue mentioned above

dnmTX commented 5 years ago

@funilrys https://github.com/Ultimate-Hosts-Blacklist/quidsup_malicious-sites finished filtering about 2 hours ago and no lists(clean.list,volatile.list etc.) got updated.You mind take a look.Thanks.

funilrys commented 5 years ago

Just pushed the new whitelisting tool in production. So normally #23 should be fixed in the coming days/hours.

I'm restarting all passed and failed in the coming hour.

funilrys commented 5 years ago

Starting the restart (only last build part) of all failed build.

funilrys commented 5 years ago

Finished the restart (only last build part) of all failed build.

funilrys commented 5 years ago

Starting the test launch of all passed build.

dnmTX commented 5 years ago

Ok,i'll let you know if anything.

P.S. The thing with the last problem i reported(above) is that Travis Cl reported that the bill is passing,no problems but still none of the lists got updated.Let's hope it's relevant to the fix you just pushed and it's not some new issue to deal with.

funilrys commented 5 years ago

Finished the test launch of all passed build.

funilrys commented 5 years ago

@dnmTX The new update.py do not have any changes in the way we run tests. It's only maintenance:

If build passed and no changes, that mostly neans that there is no changes between the new and the previous version of {clean,volatile,whitelisted}.list.

If you still find some I'll take the time to check them all as usually :wink:

dnmTX commented 5 years ago

Ok,i see,if i understood correctly all those lists didn't get updated because there were no changes from the last filtering which make sense.This particular list is small so it's likely not to expect any changes every time it gets filtered.

funilrys commented 5 years ago

You may ask yourself what are the differences between both whitelisting tools:

Why (I reserved) the new woldwide namespace ultimate_hosts_blacklist ? It's simple: it allows us to have a lot of code in one place and installable with pip. So the reusability is way better. Before this new namespace I had to copy/past some helpers (for example) over and over and over and so on...

Okay time to sleep (Berlin Time). Closing. Please report to #28 for further reports/feedbacks.