search

jjneely / buckytools

Go implementation of useful tools for dealing with Graphite's Whisper DBs and Carbon hashing
Other
87 stars 21 forks source link

delete empty folder after delete #24

Open ctrlok opened 7 years ago

ctrlok commented 7 years ago

Hi. I receive a lot of empty folders after removing metrics and it is bad for me, because I receive a lot of non exist fields in /find request. Now I fix it by find /mnt/xvdx/ -type d -empty, but maybe we should remove it after "delete" operation?

I can create PR if you agree with it.

azhiltsov commented 7 years ago

There was already one #15 and I would also prefer to have it. Might be an extra parameter at invocation will be a good compromise solution?

ctrlok commented 7 years ago

@jjneely, @azhiltsov there is two way how we can do that - with an additional flag to delete command, but I'm not sure because there is a lot of undefined behaviors:

I had a metrics:

/root/host1/metric1/name1.wsp
/root/host1/metric2/name1.wsp

If I will remove every name1, so should I remove metric1, host1 if that didn't contain any metrics?

So... Maybe it will be better to add some command like delete-empty-folders which will remove every empty folder on graphite? What do you think?

jjneely commented 7 years ago

Basically, to integrate the find command invocation into buckyd. I'd be up for that.

But yes, there are a bunch of corner cases and race conditions possible which is why I've been hesitant to add this. I've lost data because of deleting "empty" directories in my graphite cluster before.

The full incantation I suggest is

/usr/bin/find ${prefix}/storage/whisper -type d -empty -mtime +1 -delete

The -mtime setting there does a lot to sanitize possible race conditions, although its not race free.