Prigent-Adult list unavailable

[ghost]

Hey @WaLLy3K, first of all, a big thank-you for painstakingly maintaining your Big Blocklist Collection. I've noticed that the list with the URL "https://v.firebog.net/hosts/Prigent-Adult.txt" has not been available for quite some time:

I'm using this list on my Pi-hole and thought I'd let you know that it's unavailable.

[WaLLy3K]

Thanks for the ping @zackiff51, and I now notice it's been a bit hit or miss. I've just made a separate parser for that list (set to update once a week) which seems to let the script process the final 100mb text file successfully.

Please feel free to hit me up if it stops working as expected in the future 🙂

[ghost]

Thanks for the speedy reply and fix!

I will hit you up if it stops working again. :)

[ghost]

Hey @WaLLy3K, once again, the Prigent-Adult list is unavailable (404 Not Found).

[WaLLy3K]

Noted, and looking into it. Thanks @ytoruec80

[WaLLy3K]

While I've resolved it by manually running the script (after adding a few more debug lines), I haven't actually pinned down exactly where and why the script fails, only that it stops after successfully extracting the file with 7Zip. I've re-opened this with the knowledge that this is an ongoing issue and will likely reoccur in the future.

Whomever notices this next time, please @ me about this :)

[ghost]

Thanks a lot for the quick fix and the good work! :)

[ghost]

@WaLLy3K Unfortunately, it's happening again. :/

[WaLLy3K]

Thank you @nindilo51, I think I've pinned down the issue to PHP kicking up a stink trying to ensure every line is unique - which definitely isn't a check that's needed for this particular list. I really need to use a better language for the job, but hey.

The list is back up again now :)

[ghost]

Yep, it's back to normal now. Great work! :) Well, PHP is a special programming language. :D

[mastermc0]

@WaLLy3K looks like it's missing again.

[WaLLy3K]

Getting closer to figuring this out; have got it back up for now.

[ghost]

Hey @WaLLy3K, the list is unavailable again.

[WaLLy3K]

The script processes everything, but fails to save everything at the end like it's running out of memory despite nothing showing up in PHP error logs, or looking at the server via htop. I've removed all line validation (which wasn't doing a whole lot according to the debug output) and that let the final output save on first attempt.

If I had another box available (that wasn't either self hosted or work hosted), I'd whip up the script in another non-trash language and we'd be fine. Unfortunately, this is the one 'cost' for hosting everything for free on a reliable and trustworthy box :)

[ghost]

Either way, I really appreciate the effort you're putting into this. Thank you! :)

[codygp]

@WaLLy3K Its down again.

Are you looking for server hosting or something like that. I might be able to help. I have a server I pay for that I barely use.

[WaLLy3K]

Hey @codygp, thanks for letting me know, it's back up. I've got a server I can do the processing on, I've just never had the time AND drive to rewrite this from the ground up. I've got a week off from work, so I'll try to bang something out to fix this properly.

Edit: I've completed rewriting it all, so I'll spend the next few days validating the output of various types of content.

[WaLLy3K]

All done. Anything done with the new parser will have some comments up the top, and I'll get around to migrating everything over to it as it's considerably faster than what I had.

https://v.firebog.net/hosts/Prigent-Adult.txt

If there are any issues, please let me know.

[WaLLy3K]

Everything's been migrated to the new parser now, which can be seen at https://v.firebog.net/hosts