search

RIVeR-Lab / numl_val_workspace

repository for vcstool workspaces
0 stars 0 forks source link

Retire/Delete old logs on NASA NAS #7

Closed allspawj closed 7 years ago

allspawj commented 8 years ago

Really needs to be done. Once old logs are processed we can enforce regular processing of new logs.

I am planning on just copying essentially random logs off of the NAS to vis storage raid, I think 1/10th of current logs should be good. Then enforcing that people process their logs the day they are created, along with doing the online report.

@velinddimitrov

velinddimitrov commented 8 years ago

I started moving some over to the RAID on Odin when NASA was here. Someone cancelled it midway through, so we need to check what has been moved over and delete it from the NAS.

allspawj commented 8 years ago

The NAS is down to 2 TB of storage, a select range of old logs are being compressed and put in a legacylog directory since they are all unsorted. I am setting up an email alert for when the NAS gets to 8tb again just in case. See new procedure for correctly handling new logs.

allspawj commented 8 years ago

Also, compressing NASAS logs tends to reduce them to 1/5th of their size, which is a pretty significant footprint change. I recommend everyone compress the logs before archiving.

velinddimitrov commented 8 years ago

We may be able to automatically zip the nasa logs at the end of every day or something similar with a script. I will try this next time I am in Lowell: https://forum.qnap.com/viewtopic.php?t=57611.

murphywonsick commented 7 years ago

See issue #9 for related discussion.