search

moov-io / watchman

AML/CTF/KYC/OFAC Search of global watchlist and sanctions
https://moov-io.github.io/watchman/
Apache License 2.0
330 stars 87 forks source link

Downloaded sanction list text/csv files are not deleted after they are not needed any more #536

Closed m29h closed 6 months ago

m29h commented 6 months ago

The downloader package here creates a new temporary directory (with random directory name) every time the sanction lists are refreshed. (Standard every twelve hours)

The file content is parsed immediately after the download and then abandoned.

This fills up the /tmp folder over time. A docker container running 24/7 will waste a couple of GB of physical storage every week of running. It is not much but noticeable over time.

Is there a reason why the temp files are not deleted immediately after they are not used any more. One could even use a suitable in memory buffer instead of a physical temporary file to avoid this in the first place!

adamdecaf commented 6 months ago

No real reason why it's not deleted yet. We can remove it after parsing.