search

tosdr / tosback2

Reimplementing TOSBack with Ruby and using git to see TOS changes!
http://tosback.org
GNU General Public License v2.0
125 stars 32 forks source link

Crawler stopped (lack of inodes) #76

Closed michielbdejong closed 3 years ago

michielbdejong commented 3 years ago

https://tosback.org/ shows a directory listing, suggesting it's running Apache without Passenger, probably? The root cause seems to be disk quota:

tosback3@dragon:/var/www/ToSBack3/public$ touch test.txt 
touch: cannot touch 'test.txt': Disk quota exceeded
michielbdejong commented 3 years ago

Ah but there is no quota, so probably it's because of:

tosback3@dragon:/var/www/ToSBack3/public$ df -i
Filesystem     Inodes  IUsed IFree IUse% Mounted on
/dev/simfs     200000 200000     0  100% /
devtmpfs        65536     53 65483    1% /dev
tmpfs           65536      1 65535    1% /dev/shm
tmpfs           65536    187 65349    1% /run
tmpfs           65536      3 65533    1% /run/lock
tmpfs           65536     10 65526    1% /sys/fs/cgroup
none            65536      1 65535    1% /run/shm
michielbdejong commented 3 years ago

I removed /home/michiel/.nvm/ to free up a few inodes, commented out /home/tosback3/tosback-crawler/run.sh, and restarted apache (sudo service apache2 restart). So the website is back but the crawler is stopped now. I guess the best way forward is to move this service to a server with more than 200k iNodes. But then we have to ask EFF to do the DNS change.

CC @JustinBack thanks for reporting!

michielbdejong commented 3 years ago

Maybe just see if we can attach a new disk or something

JustinBack commented 3 years ago

Maybe Rsyncing it to a different server and check if the documents exist there? If the crawler uploaded its files, remove it locally.

michielbdejong commented 3 years ago

No the issue is not a missing file (Passenger is supposed to insert itself as an Apache module and run the Ruby code to generate the main page). The issue is that the disk of this server has very few inodes, and keeps running out whenever we leave the crawler running. There was no way to attach a second disk but I requested an upgrade through https://securedragon.net/clients/clientarea.php to a bigger server that costs the same - let's see what happens.

michielbdejong commented 3 years ago

At the same time, we're trying to get the tosback.org DNR transferred from EFF to us, so that we can move servers.