Publishing the source code of blacklist generation's program

[ClementDerouet]

Greetings, I like very much the Ipsum project. I believe this blacklist is a good idea, but there is a problem : we are dependent on your git repository. If you interrupt the feeding of the IP list, we would have no protection anymore. So, I think you should have publish the source code of the program that allows to generate the list. Thus, we can have our own alternative repository.

[stamparm]

update-ipsum: # with redacted $GITHUB_USERNAME and $GITHUB_PASSWORD

#!/bin/bash

cd /tmp
rm -rf maltrail ipsum master.zip*
git clone --depth 1 "https://$GITHUB_USERNAME:$GITHUB_PASSWORD@github.com/stamparm/ipsum" ipsum
cat ipsum/header.txt > ipsum/ipsum.txt
echo "# Last update: `date -R`" >> ipsum/ipsum.txt
echo "#" >> ipsum/ipsum.txt
echo -e "# IP\tnumber of (black)lists" >> ipsum/ipsum.txt
echo "#" >> ipsum/ipsum.txt
wget https://github.com/stamparm/maltrail/archive/master.zip
unzip master.zip
mv maltrail-master maltrail
cd maltrail
rm -rf trails/feeds/bambenekconsulting*
python core/update.py -r 2>>/tmp/ipsum/ipsum.txt
cd /tmp/ipsum
if [ `stat -c %s ipsum.txt` -ge 10000 ]; then
cat ipsum.txt | grep -v "#" | cut -f 1 > levels/1.txt
cat ipsum.txt | grep -v "#" | grep -v -E "\s[1]$" | cut -f 1 > levels/2.txt
cat ipsum.txt | grep -v "#" | grep -v -E "\s[1-2]$" | cut -f 1 > levels/3.txt
cat ipsum.txt | grep -v "#" | grep -v -E "\s[1-3]$" | cut -f 1 > levels/4.txt
cat ipsum.txt | grep -v "#" | grep -v -E "\s[1-4]$" | cut -f 1 > levels/5.txt
cat ipsum.txt | grep -v "#" | grep -v -E "\s[1-5]$" | cut -f 1 > levels/6.txt
cat ipsum.txt | grep -v "#" | grep -v -E "\s[1-6]$" | cut -f 1 > levels/7.txt
cat ipsum.txt | grep -v "#" | grep -v -E "\s[1-7]$" | cut -f 1 > levels/8.txt
grep -B100000 "Wall of shame" README.md | grep -v "Wall of shame" > _README.md
if [ -s _README.md ]
then
mv _README.md README.md
fi
cat >> README.md << EOF
Wall of shame ($(date +'%Y-%m-%d'))
----

|IP|Number of (black)lists|
|---|--:|
EOF
cat ipsum.txt | grep -v "#" | grep -v -E "\s[1-7]$" | tr '\t' '|' >> README.md
git commit -am "Automatic update"
git push origin master
fi
cd /tmp
rm -rf maltrail ipsum master.zip*

header.txt:

# IPsum Threat Intelligence Feed
# (https://github.com/stamparm/ipsum)
#

crontab -l:

...
0 23 * * * $HOME/ipsum/update-ipsum

[stamparm]

p.s. I'll just leave it here as it is without pushing it to the repo

[ClementDerouet]

Thank you very much.