search

basho / cuttlefish

never lose your childlike sense of wonder baby cuttlefish, promise me?
Apache License 2.0
205 stars 124 forks source link

Generating infinite config files. #91

Closed joedevivo closed 10 years ago

joedevivo commented 10 years ago

we should probably only keep the last X files

joedevivo commented 10 years ago

Some cool manual test output

➜  cuttlefish git:(bugfix/jd/infinite-files) ✗ ./cuttlefish -e ~/dev/basho/riak_ee/rel/riak/etc -s ~/dev/basho/riak_ee/rel/riak/lib -d ~/dev/basho/riak_ee/rel/riak/data/generated.configs -c ~/dev/basho/riak_ee/rel/riak/etc/riak.conf generate
 -config /Users/joe/dev/basho/riak_ee/rel/riak/data/generated.configs/app.2014.04.01.11.12.23.config -args_file /Users/joe/dev/basho/riak_ee/rel/riak/data/generated.configs/vm.2014.04.01.11.12.23.args -vm_args /Users/joe/dev/basho/riak_ee/rel/riak/data/generated.configs/vm.2014.04.01.11.12.23.args 
➜  cuttlefish git:(bugfix/jd/infinite-files) ✗ ls ~/dev/basho/riak_ee/rel/riak/data/generated.configs                                                                                                                                            
total 72
drwxr-xr-x  8 joe  staff   272 Apr  1 11:12 .
drwxr-xr-x  5 joe  staff   170 Mar 27 09:58 ..
-rw-r--r--  1 joe  staff  7528 Apr  1 10:56 app.2014.04.01.10.56.34.config
-rw-r--r--  1 joe  staff  7528 Apr  1 10:56 app.2014.04.01.10.56.47.config
-rw-r--r--  1 joe  staff  7528 Apr  1 11:12 app.2014.04.01.11.12.23.config
-rw-r--r--  1 joe  staff   177 Apr  1 10:56 vm.2014.04.01.10.56.34.args
-rw-r--r--  1 joe  staff   177 Apr  1 10:56 vm.2014.04.01.10.56.47.args
-rw-r--r--  1 joe  staff   177 Apr  1 11:12 vm.2014.04.01.11.12.23.args
➜  cuttlefish git:(bugfix/jd/infinite-files) ✗ ./cuttlefish -e ~/dev/basho/riak_ee/rel/riak/etc -s ~/dev/basho/riak_ee/rel/riak/lib -d ~/dev/basho/riak_ee/rel/riak/data/generated.configs -c ~/dev/basho/riak_ee/rel/riak/etc/riak.conf -m 5 generate
 -config /Users/joe/dev/basho/riak_ee/rel/riak/data/generated.configs/app.2014.04.01.11.13.28.config -args_file /Users/joe/dev/basho/riak_ee/rel/riak/data/generated.configs/vm.2014.04.01.11.13.28.args -vm_args /Users/joe/dev/basho/riak_ee/rel/riak/data/generated.configs/vm.2014.04.01.11.13.28.args 
➜  cuttlefish git:(bugfix/jd/infinite-files) ls ~/dev/basho/riak_ee/rel/riak/data/generated.configs                                                                                                                                                 
total 96
drwxr-xr-x  10 joe  staff   340 Apr  1 11:13 .
drwxr-xr-x   5 joe  staff   170 Mar 27 09:58 ..
-rw-r--r--   1 joe  staff  7528 Apr  1 10:56 app.2014.04.01.10.56.34.config
-rw-r--r--   1 joe  staff  7528 Apr  1 10:56 app.2014.04.01.10.56.47.config
-rw-r--r--   1 joe  staff  7528 Apr  1 11:12 app.2014.04.01.11.12.23.config
-rw-r--r--   1 joe  staff  7528 Apr  1 11:13 app.2014.04.01.11.13.28.config
-rw-r--r--   1 joe  staff   177 Apr  1 10:56 vm.2014.04.01.10.56.34.args
-rw-r--r--   1 joe  staff   177 Apr  1 10:56 vm.2014.04.01.10.56.47.args
-rw-r--r--   1 joe  staff   177 Apr  1 11:12 vm.2014.04.01.11.12.23.args
-rw-r--r--   1 joe  staff   177 Apr  1 11:13 vm.2014.04.01.11.13.28.args
➜  cuttlefish git:(bugfix/jd/infinite-files) ./cuttlefish -e ~/dev/basho/riak_ee/rel/riak/etc -s ~/dev/basho/riak_ee/rel/riak/lib -d ~/dev/basho/riak_ee/rel/riak/data/generated.configs -c ~/dev/basho/riak_ee/rel/riak/etc/riak.conf -m 5 generate
 -config /Users/joe/dev/basho/riak_ee/rel/riak/data/generated.configs/app.2014.04.01.11.13.32.config -args_file /Users/joe/dev/basho/riak_ee/rel/riak/data/generated.configs/vm.2014.04.01.11.13.32.args -vm_args /Users/joe/dev/basho/riak_ee/rel/riak/data/generated.configs/vm.2014.04.01.11.13.32.args 
➜  cuttlefish git:(bugfix/jd/infinite-files) ls ~/dev/basho/riak_ee/rel/riak/data/generated.configs                                                                                                                                                 
total 120
drwxr-xr-x  12 joe  staff   408 Apr  1 11:13 .
drwxr-xr-x   5 joe  staff   170 Mar 27 09:58 ..
-rw-r--r--   1 joe  staff  7528 Apr  1 10:56 app.2014.04.01.10.56.34.config
-rw-r--r--   1 joe  staff  7528 Apr  1 10:56 app.2014.04.01.10.56.47.config
-rw-r--r--   1 joe  staff  7528 Apr  1 11:12 app.2014.04.01.11.12.23.config
-rw-r--r--   1 joe  staff  7528 Apr  1 11:13 app.2014.04.01.11.13.28.config
-rw-r--r--   1 joe  staff  7528 Apr  1 11:13 app.2014.04.01.11.13.32.config
-rw-r--r--   1 joe  staff   177 Apr  1 10:56 vm.2014.04.01.10.56.34.args
-rw-r--r--   1 joe  staff   177 Apr  1 10:56 vm.2014.04.01.10.56.47.args
-rw-r--r--   1 joe  staff   177 Apr  1 11:12 vm.2014.04.01.11.12.23.args
-rw-r--r--   1 joe  staff   177 Apr  1 11:13 vm.2014.04.01.11.13.28.args
-rw-r--r--   1 joe  staff   177 Apr  1 11:13 vm.2014.04.01.11.13.32.args
➜  cuttlefish git:(bugfix/jd/infinite-files) ./cuttlefish -e ~/dev/basho/riak_ee/rel/riak/etc -s ~/dev/basho/riak_ee/rel/riak/lib -d ~/dev/basho/riak_ee/rel/riak/data/generated.configs -c ~/dev/basho/riak_ee/rel/riak/etc/riak.conf -m 5 generate
 -config /Users/joe/dev/basho/riak_ee/rel/riak/data/generated.configs/app.2014.04.01.11.13.37.config -args_file /Users/joe/dev/basho/riak_ee/rel/riak/data/generated.configs/vm.2014.04.01.11.13.37.args -vm_args /Users/joe/dev/basho/riak_ee/rel/riak/data/generated.configs/vm.2014.04.01.11.13.37.args 
➜  cuttlefish git:(bugfix/jd/infinite-files) ls ~/dev/basho/riak_ee/rel/riak/data/generated.configs                                                                                                                                                 
total 120
drwxr-xr-x  12 joe  staff   408 Apr  1 11:13 .
drwxr-xr-x   5 joe  staff   170 Mar 27 09:58 ..
-rw-r--r--   1 joe  staff  7528 Apr  1 10:56 app.2014.04.01.10.56.47.config
-rw-r--r--   1 joe  staff  7528 Apr  1 11:12 app.2014.04.01.11.12.23.config
-rw-r--r--   1 joe  staff  7528 Apr  1 11:13 app.2014.04.01.11.13.28.config
-rw-r--r--   1 joe  staff  7528 Apr  1 11:13 app.2014.04.01.11.13.32.config
-rw-r--r--   1 joe  staff  7528 Apr  1 11:13 app.2014.04.01.11.13.37.config
-rw-r--r--   1 joe  staff   177 Apr  1 10:56 vm.2014.04.01.10.56.47.args
-rw-r--r--   1 joe  staff   177 Apr  1 11:12 vm.2014.04.01.11.12.23.args
-rw-r--r--   1 joe  staff   177 Apr  1 11:13 vm.2014.04.01.11.13.28.args
-rw-r--r--   1 joe  staff   177 Apr  1 11:13 vm.2014.04.01.11.13.32.args
-rw-r--r--   1 joe  staff   177 Apr  1 11:13 vm.2014.04.01.11.13.37.args
jaredmorrow commented 10 years ago

Since this is more a feature than bug fix, should it wait for 2.0.1 this late in the game?

cmeiklejohn commented 10 years ago

Yeah; this doesn't seem like a bug fix and is probably a better candidate for 2.0.1.

joedevivo commented 10 years ago

I've always considered this a bug and it's the last outstanding thing I was shooting for getting into beta. I think that the risk is minimal and am happy to discuss it further tomorrow.

jonmeredith commented 10 years ago

Coming down on a bug, but for RC. Although it probably won't eat all the disk space, we've seen customer systems crash when they've run out of inodes and generating very large directory listings are antisocial for general filesystem tooling/wildcard expansion.

Although I'd really like to limit the amount of remaining work, given the PR is mostly done we should review/merge.

joedevivo commented 10 years ago

Pushed a version with @jrwest's suggestion. left the commits separate since they bolth work.

jrwest commented 10 years ago

+1 03574a2

joedevivo commented 10 years ago

@borshop merge