CuriosAI / sai

SAI: a fork of Leela Zero with variable komi.
GNU General Public License v3.0
106 stars 11 forks source link

removing the old networks #132

Open maxinjapan opened 3 years ago

maxinjapan commented 3 years ago

Hi. I just realised that after 62k matches I had over 230Gb of old networks saved locally on my PC. I was wondering if you could implement a function to automatically remove the unused networks after a while to avoid taking up so much space.

Vandertic commented 3 years ago

You are right that this problem should be addressed, but currently it seems not an easy upgrade. Maybe @amato-gianluca can have a look into this?

l1t1 commented 3 years ago

you can write a bash script to remove old files

Vandertic commented 3 years ago

This bash line, launched from inside the networks/ directory should leave only the 32 most recent networks. n=$(( $(ls -1 *.gz | wc -l) -32 )) ; if [[ ! $n -lt 0 ]] ; then rm $(ls -1rt *.gz | head -n$n) ; fi

I suggest to try before a "dry run" with n=$(( $(ls -1 *.gz | wc -l) -32 )) ; if [[ ! $n -lt 0 ]] ; then echo rm $(ls -1rt *.gz | head -n$n) ; fi just to check that nothing wrong is going to happen.

When the networks are 32 or less, nothing should happen, so this script can be launched automatically every few days to clean the directory.

maxinjapan commented 3 years ago

nice. thanks

l1t1 commented 3 years ago

windows bat is also needed

maxinjapan commented 3 years ago

FWIW it works in windows too if you install Ubuntu from the Miscosoft Store