search

Ultimate-Hosts-Blacklist / dev-center

The place to talk about our infrastructure or everything related to the Ultimate Hosts Blacklist project.
MIT License
11 stars 2 forks source link

[bug] cameleon_at_sysctl.org #6

Closed dnmTX closed 6 years ago

dnmTX commented 6 years ago

@mitchellkrogza there is a issue with the formatting in the output folder.I'm using the lists with (g)awk to to filter the duplicates against StevenBlack's lists which are a lot ( like 13 000+) and simply does not work.It outputs the same number as the input,different command combinations results in wrong numbers.If i download the lists from the clean.list works like charm.I'm using (g)awk with other lists as well and have no problems but here,and only if i fetch it from the output folder. @funilrys's clean.list is empty so i had to start downloading it from here but his repo got the same formatting issue in the output folder as well. If for some reason you don't believe me or can't do nothing about it.please,at least keep the clean.list populated and most importantly updated so i can be able to download it and filter it.Thank you.

funilrys commented 6 years ago

@dnmTX Thanks for reporting.

I do not understand what's the point here as https://github.com/Ultimate-Hosts-Blacklist/cameleon_at_sysctl.org/blob/master/clean.list is not empty ...

Please keep in mind that we do not touch nothing. I do check the whole system on a private machine once or twice every weeks but the whole process is automated.

Closing.

dnmTX commented 6 years ago

@funilrys the mentioned clean.list in your repo is this one: https://github.com/dead-hosts/cameleon_web_sysctl.org/blob/master/clean.list

That's where i was originally downloading it from and since last week it's empty. The Dead-Hosts(https://github.com/dead-hosts) is your project isn't it? There are quite a few lists on rotation there and i was keeping an eye on the cameleon lists for any changes but as of late looks more and more abandoned. And you didn't even addressed the main issue here,the formatting in the output folder.There is something wrong there,which i can't explain it but you as a highly professional can at least check it out before you cut me off and close it.

funilrys commented 6 years ago

@dnmTX You are writing an issue which mention cameleon_at_sysctl.org which is a unique repository name in this organization not the Dead-Hosts project.

You have to understand that if you write an issue here, it does not have something to do with the Dead-Hosts organization.

I do understand your pain and I'll investigate the right issue but please next time give the right information at the right place.

Reporting to https://github.com/dead-hosts/cameleon_web_sysctl.org/issues/2.

dnmTX commented 6 years ago

@funilrys in my defense i was reporting bug that exists in cameleon_at_sysctl.org and meanwhile mentioned that the same bug(about the formatting in output folder) exist in dead-hosts as well and also mentioned that in dead-hosts's cameleon_web_sysctl.org the clean.list is empty. My MAIN PROBLEM is the formatting in the output folder HERE and BIG CONCERN is the clean.list HERE to stay populated and not empty like in dead-hosts because this is my only source at the present.

funilrys commented 6 years ago

@dnmTX Can you please detail the formatting issue at https://github.com/funilrys/PyFunceble so I can investigate it. Indeed, the output directory is generated by PyFunceble.

Cheers, Nissar.

dnmTX commented 6 years ago

@funilrys you mentioned that you found the problem here: https://github.com/dead-hosts/cameleon_web_sysctl.org/issues/2

Do you still want me to report it in: https://github.com/funilrys/PyFunceble or you good?

Will wait for reply.

dnmTX commented 6 years ago

@funilrys you need to restart this one as well.

funilrys commented 6 years ago

Hi @dnmTX,

The process is currently running : https://travis-ci.org/Ultimate-Hosts-Blacklist/cameleon_at_sysctl.org.

For the formatting issue you take in consideration that other file which are not clean.list starts with comments ?

dnmTX commented 6 years ago

@funilrys before i do any duplicate filtering both lists are cleaned out of comments,empty space etc. etc. And i know you still think that the problem is on my end(which is not),let me post you the commands i'm using on both lists just to be more convincing or you can try to replicate it if you want:

For StevenBlack's lists: sed -i '/^# Start KAD/,/^# End KAD/d; /^# Start CoinB/,/^# End CoinB/d; /^#<Wind/,/^#<\/Wind/d; /^127/d; /^255/d; /::/d; / 0.0./d; /#/d; /^$/d' stevenblack.txt

For Cameleon: sed -i /^#/d; /^$/d' cameleon.txt (when using hosts from output folder) sed -i '/^$/d; s/^/0.0.0.0 /g' cameleon.txt (when using clean.list)

Replicate it,and then try to filter the duplicates with the awk command that i posted in dead-hosts issues. Just make sure that the chosen Cameleon lists is from the output folder and the result will be the same as the original.Next,do the same but use the clean.list and the result will be something around 7749 unique entries.