search

splunk / splunk-shuttl

Splunk app for archive management, including HDFS support.
Apache License 2.0
36 stars 19 forks source link

simultaneous csv and bucket format archiving #48

Closed borischen closed 12 years ago

borischen commented 12 years ago

we should support people being able to do both at the same time

petterik commented 12 years ago

Errors: If both doesn't succeed, we want to re-archive the failed one later.

How to do it: Check if a bucket has already been archived by looking at the archive path. If it's in the archive path, that means that it was successfully transferred. Only remove the bucket to be archived from local disk when all formats has been successfully archived.