search

aodn / content

Tracks AODN Portal content and configuration issues
0 stars 0 forks source link

SOOP_TRV is writing files to /tmp and leaving them there #464

Closed mhidas closed 3 years ago

mhidas commented 3 years ago

See https://github.com/aodn/issues/issues/819#issuecomment-675771273

It looks like SOOP_TRV is writing files to /tmp and leaving them there. There are nearly 5000 files at >5MB, so that accounts for the space!

what is writing them and why are these being written to /tmp (because this should never happen from a pipeline or anything else) why is it leaving them behind... any temp file related stuff should always use a try/finally type pattern (or a bash trap) to make sure they are always cleaned up at the end of execution

ghost commented 3 years ago

Ok traced it back a little further, since the files are still being created this morning.

This is being run by cron: https://github.com/aodn/data-services/blob/master/SOOP/SOOP_TRV/soop_trv.py

It's called this function to create the temp files: https://github.com/aodn/data-services/blob/master/lib/python/aims_realtime_util.py#L401

The simplest solution is probably to ensure that the code here which is responsible for causing the file to be created, should have a finally block which guarantees the file is deleted, whether it succeeds or fails: https://github.com/aodn/data-services/blob/master/SOOP/SOOP_TRV/soop_trv.py#L149

mhidas commented 3 years ago

:+1: Thanks @lwgordonimos