search

geminiplanetimager / gpi_pipeline

Gemini Planet Imager Data Reduction Pipeline
Other
8 stars 6 forks source link

Clear out old files from the queue? #13

Closed dsavransky closed 7 years ago

dsavransky commented 8 years ago

Currently, the queue just accumulates old processed XML recipe files, which currently number several thousand. There should probably be some housekeeping task that cleans these out after some time period?

Users can manually delete all files in the queue from the Queue Viewer, but that is both a coarse and cumbersome way to do that; it should be automated somehow.

dsavransky commented 8 years ago

Recovering rm journals for issue:

dsavransky: How about a button on the status console that deletes queue_dir/*.done.xml ?

mperrin: That button actually already exists, it's just in the Queue Viewer window rather than the Status Console.

The idea here would be to automate that in some semi-intelligent manner, to avoid making the user have to periodically remember to push that button. But, low priority.

ingraham: So having a program automatically delete things is a little disconcerting...

How about if a informational message were to appear in the IDL windows saying that there are some really old completed recipes in the queue_directory that should be deleted?

A sort of reminder for people to push the button so a pile-up doesn't occur?

Also, how long should this reminder be? I think ~6 months is probably fine (I'm sure the number of files at Gemini gets pretty large)

mperrin: I'm quite sure at Gemini the queue directory contains every recipe that has ever executed on cpogpi02.

yeah I'd vote for a "Delete recipes older than 6 months from the queue" button or menu option, and have it pop up a reminder/query to the user periodically. Maybe the first of the month, or every three months or something, so it's not too often an annoyance.

mperrin commented 8 years ago

@dsavransky @vbailey @semaphoreP @kfollette any opinions on this old issue? It seems to me that so far we have not needed any functionality to delete old recipes from the queue directory, and the accumulation of a zillion old files has not caused any trouble. So I think we should mark this "don't fix" and close and ignore it. Opinions?

semaphoreP commented 8 years ago

I feel like "rm" is sufficient for this? And that any more advanced functionality is not worth the effort of implementing? rm is also capable of removing files older than I certain date I think, so we could occasionally purge recipes as a manual housekeeping task when needed

semaphoreP commented 7 years ago

Users can rm recipes themselves through their respective OSes. Unless this pops up as a bigger issue in the future, this is closed as won't fix.