search

mozilla-services / hindsight

Hindsight - light weight data processing skeleton
Mozilla Public License 2.0
668 stars 55 forks source link

Cleaning unused rtc files on output folder + .off files in input folder #178

Closed reynaldiwijaya closed 6 years ago

reynaldiwijaya commented 6 years ago

Is there a way for officially clean the .off files from sandbox run path and the rtc files on output folder?

We have a hindsight deployed to process data from logs. It was a dynamic log collection from kubernetes pods that we gather and process as an input. The problem is many of these pods are short lived, and therefore created many inputs that were only used briefly.

At the moment, it was littering our sandbox run path with .off files and hindsight output folder with rtc files from all log source that had not been used anymore. Does hindsight support auto cleaning for these unused rtc and off files?

trink commented 6 years ago

It is safe for you to simply delete them as they are only used by the admin UI (which cleans them up when a user deletes a plugin).