Reinventing the shell wheel?

[dmitrig01]

I'm not an expert in Jepsen or on Clojure, but I'm starting to think about how to use it for one of my projects, and one thing that struck me is the DSL you have for getting things set up - e.g.:

(when-not (debian/installed? "mongodb-org")
  (c/su
    (try
      (debian/install {:mongodb-org version})
      (catch RuntimeException e
        (c/exec :apt-key :adv :--keyserver "keyserver.ubuntu.com" :--recv "7F0CEB10")
        (c/exec :echo "deb http://downloads-distro.mongodb.org/repo/debian-sysvinit dist 10gen" :> "/etc/apt/sources.list.d/mongodb.list")
        (c/exec :apt-get :update)
        (debian/install {:mongodb-org version})
    (c/exec :update-rc.d :mongod :remove :-f)))))

Is there any reason it couldn't just be written as a simple shell script (not to mention one of the many config management tools out there, but that might be overkill)?

sudo apt-get install mongodb-org=$VERSION || { \
    sudo apt-key adv --keyserver "keyserver.ubuntu.com" --recv "7F0CEB10" && \
    sudo echo "deb http://downloads-distro.mongodb.org/repo/debian-sysvinit dist 10gen" > "/etc/apt/sources.list.d/mongodb.list" && \
    sudo apt-get update && \
    sudo apt-get install mongodb-org=$VERSION }
update-rc.d mongod remove -f

It just seems like one more thing to learn in an already-complex system :-)

[aphyr]

I mean, there's nothing stopping you from just running shell scripts, but the reason this thing is here is to make it easier to compose bash into higher order functions, do correct shell escaping, etc.

[aphyr]

(Also Jepsen needs to execute commands throughout the test lifecycle, so an external automation system like chef/ansible/puppet isn't really appropriate)

[dmitrig01]

That's fair. I hate to be a troll (please stop me if I get too troll-y) but: I guess it comes down to, "what is Jepsen?". It seems like it now encompasses 'everything Jepsen is' + 'bash sucks so let's write a new shell'... even something like https://github.com/aphyr/jepsen/blob/old/salticid/mongo.rb is less of a learning curve.

As for creating higher-order functions, escaping, etc - well, you can define functions, which are not fantastic, but they work. Jepsen could help by providing some kind of utility to call a bash script with a map of environment variables which get auto-escaped for you. Again - you have a much deeper understanding of what needs to go on to make the various parts happen than I do - but coming in as an outsider, it just seems like yet step i need to climb, perhaps somewhat unnecessarily, along the learning curve (and yes - even if I could write a shell script to do this, the existing examples do count for a lot in terms of guiding me and other newcomers).

[aphyr]

Jepsen really isn't designed for mass appeal; it's a research tool for me. I need these features. If you don't, that's fine. :)

[dmitrig01]

Heh, ok. I mean, it'd be great to be able to test my project with these awesome tools, but if that's not the goal of the project, that's fair.

[dmitrig01]

Hey man, I really respect and look up to the work you do. I don't know a ton about distributed systems, but everything I do know (which I hope to be a thing or two), I learned from reading your blog series and watching the Jepsen talks (and, you have the best introduction to Clojure on the internet!). I decided that because this is a really interesting area to me, I wanted to dive deeper: try it out myself, learn from it, and hopefully be able to grasp it well enough to be able to contribute something back to the project from which I learned a lot!

However – as I was trying to wrap my head around all the things Jepsen does, this thing stuck out specific to me as something interesting/peculiar, so I thought I'd ask about it. It's frustrating, then, as someone who is trying in their own way to be helpful, to hear a technical decision justified with "it's a research tool for me". It makes me feel as if there's no place for the non-distributed-systems-ninja here (which maybe there isn't, but that would make me sad! contributing to Jepsen seems like the perfect way to become better at distributed systems)

Anyway, reading back, my last comment certainly did not state my intentions. I'm definitely not here to get a free ride. I'm here from the perspective of "you are an awesome person doing awesome things, and I want to be a part of that!" (but gotta learn before I can contribute!).

So, one more time – I'm sorry about that last comment. I'm here to learn about distributed systems and about Jepsen, and to help out (both by using Jepsen and by contributing back). I really do mean well. I'm just trying to wrap my head around all that's going on.

With all that said! In the spirit of learning, it would be awesome if you could point me to an example of somewhere that bash is falling short for you. (but if you're still unhappy with me, I'd understand :-))

Thanks! (ps. "if you want to help port the old tests forward, I could really use the help!" – we are heavy users of Kafka and Zookeeper, so I'd love to help there once I get a better grip of how to actually use Jepsen/how it works)