Ultimate-Hosts-Blacklist / dev-center

The place to talk about our infrastructure or everything related to the Ultimate Hosts Blacklist project.
MIT License
11 stars 2 forks source link

[Feedback] Saturday, November 09 2019 - Friday, February 21, 2020 #42

Closed funilrys closed 4 years ago

funilrys commented 4 years ago

Hello, World!

Please report here every infrastructure/build/test issues (suspected or confirmed).

Note: Issue that implies big changes or not related to infrastructure or builds/tests are welcome as a new issue!

Cheers, Nissar

funilrys commented 4 years ago

TODO:

dnmTX commented 4 years ago

I'll be 👀 . Too many fixes,hopefully there will be no major disturbances. Thanks for taking the time and fixing it @funilrys 👍

dnmTX commented 4 years ago

@funilrys it looks like after filtering there are missing domains(active for sure) and i couldn't find any trace of it(i was looking for a particular one) anywhere even in the output directory. For example: analytics.getshogun.com appear to be present in the domains.list but not in any other lists or directories.Can you please see if you can trace it somehow.It smells like a bug to me. Thanks 👍

P.S. Just FYI,it's not present in the whitelist either.

dnmTX commented 4 years ago

@funilrys just giving you update. After the recent fixes the lists(volatile.list etc.) are not updating again. Can you please see if you can fix it soon as i'm always way behind the updates from the original lists. Thank you 👍

funilrys commented 4 years ago

@dnmTX Planned in my management board for this night/tomorrow morning :wink:

funilrys commented 4 years ago

@dnmTX, You deserve a :medal_sports:. The patch I'm writing because of analytics.getshogun.com will improve PyFunceble's behavior drastically!! :medal_sports:

funilrys commented 4 years ago

@dnmTX, Once PyFunceble-dev 2.45.0 will be released (in the coming 30 minutes) our infrastructure will pick it and work with it. So in the coming hours/days issue should be fixed.

Let me know if something went wrong (I'll try to check in the morning but can't promise as it's Monday... so it will probably be in the night (Berlin Time))

Thanks again for your time!

Cheers, Nissar

P.S: I'm going to sleep but version/release can be checked here https://pypi.org/project/PyFunceble-dev/

dnmTX commented 4 years ago

I saw the fixes,great work as always @funilrys. Hopefully there will be smooth sailing from now on. Yeah,that analytics.getshogun.com domain...it was luck(most of it) and small part blaming the curator of the original lists(still feel bad about it) but we got it fixed so that's the important thing. I'll be on a lookout as always,if anything you'll definitely read about it 😉 Good night 😴

dnmTX commented 4 years ago

@funilrys lists are still not updating,some are 14 or more days old.Please take a look when you have time.Thank you 👍

funilrys commented 4 years ago

@dnmTX, I could not see anything that needs my attention (code side) after looking at the logs. It looks like as we approach December there is less update from upstream :joy:

As the same launcher is used everywhere if one is not updating, every other should not update too which is by far not the case.

So the only thing I didn't check yet is the raw_link index which varies from an input source to another. I will check tomorrow if all raw_link are available.

funilrys commented 4 years ago

TODO:

  • [x] Delete the -p argument from the central repository.
  • [x] Wait for 48h for every job to synchronize.
  • [x] Safely remove the -p flag from the tool.

Completed with https://github.com/Ultimate-Hosts-Blacklist/dev-center/commit/611e63fbb1a369e2b713c9bce8b81b77ea80ad54.

dnmTX commented 4 years ago

@funilrys this is just a sample of the latest commits which are by far three days apart: https://github.com/lightswitch05/hosts/commits/master The latest update here of the domains.list was nine days ago and the others lists haven't been updated so far for a 15 days. Not sure what else to write(as it's started happening every time when there is a problem) to convince you that something is definitely wrong 🤔

P.S. Another example from the same repo is that analytics.getshogun.com is still not added after your last fix. Ok,going through all the repos that i'm monitoring it looks like they are updating but just the one that i posted samples from. I guess only lightswitch05 will require some attention.

funilrys commented 4 years ago

I did see that but when I see another list updated 2 hours ago I could not admit that it was a general issue. That way I said that I need to check each raw_link (manually) from my machine to catch the problem 😉

funilrys commented 4 years ago

I didn't say I won't look deeply into this @dnmTX :stuck_out_tongue_winking_eye: I trust your input so there is probably a problem somewhere. But by looking at the logs I could not find one.

dnmTX commented 4 years ago

@funilrys got it,and sorry,didn't mean to point fingers(i'm not that kind).I edited my post with more info. I guess it's just that repo that has some problem.

dnmTX commented 4 years ago

Success at last,lists finally got updated 👍 Will continue monitoring though 👀

dnmTX commented 4 years ago

@funilrys when you have time,please check smed79's repo.It's failing and it doesn't update so far 5 days and the domains.list's last update is from 30 days ago(he just did update like 5 days ago).Thank you 👍

funilrys commented 4 years ago

✔️

funilrys commented 4 years ago

Note

Because of funilrys/PyFunceble#67 and everything related to it, you may see the whole infrastructure failing to start.

I just deployed the fix of our infrastructure launcher. The infrastructure should restart by itself in the coming minutes/hour.

If it's not the case later in the morning (Berlin Time), I'll look deeper into it.

dnmTX commented 4 years ago

@funilrys lists are not updating again 🤔 No exception this time,no list got updated on everything that i'm monitoring probably for more then a week.

funilrys commented 4 years ago

Cancelled all builds.

funilrys commented 4 years ago

Restarted all builds.

dnmTX commented 4 years ago

@funilrys all the lists that i'm monitoring are failing(last lists update as far as 13 days ago).Please check,see if you can get to the bottom of this,once and for all. Thank you 👍

funilrys commented 4 years ago

@dnmTX Thanks for the feedback! I was busy trying to dig deep into the bugs and how to elegantly resolve them ... It's back!

I did not check the list update issue though. But now that PyFunceble is back, it should be okay. Will check that tomorrow after work.

Cheers!