search

RooneyMcNibNug / pihole-stuff

blocklists, scripts, etc.
44 stars 10 forks source link

Please Remove Duplicate Lines #55

Closed sr093906 closed 1 year ago

sr093906 commented 1 year ago

First, I am a human not a bot. A bot will not have the interest to send such a pull request.

The duplicates are easy to spot. Please copy all content starting from the Line 17 of SNAFU.txt to any online tool or offline editor.

I use https://www.somacon.com/p568.php

The result is here

https://pastebin.com/a2APKWaS

I cannot just past it here because Github reminds me of the issue 'There was an error creating your Issue: body is too long (maximum is 65536 characters). '.

RooneyMcNibNug commented 1 year ago

Oh, apologies for that - been dealing with a lot of questionably human things lately so I think that mindset just drifted into this for me.

I definitely do see the duplications you are referring to now grepping through some things and checking out the links you provided. I ran this perl one-liner to go ahead and de-duplicate things and build the fixed SNAFU.txt file per your point:

perl -ne 'print unless $seen{$_}++;' SNAFU.txt > SNAFUdedupe.txt

I then renamed the file after comparing and double-checking between the two.

This has been completed in the following commit: https://github.com/RooneyMcNibNug/pihole-stuff/commit/0a68a5bb520cae750751180844e303e07d19d29c

Thanks for calling this out - I want to make sure this list doesn't get messy, so this was very helpful along the way.

Safe surfing, -Rooney