Add Pry console and simple readline command shell

[sempervictus]

Adds a blocking command shell reading input with readline, passing to a simple query execution wrapper, and returning _raw output in plain text.

TODO: A proper REPL implementation with multi-format parsing and local data storage/manipulation is required for serious endeavors at the CLI level. Pry would likely be a good starting candidate for this.

Usage Example:

ruby 7_splunk_shell.rb 
splunk> search index=main | head 1

Thu, 4 Apr 2013 09:01:45 PDT title="Lieberman: Griner to Mavs has no downside"
link=http://espn.go.com/dallas/nba/story/_/id/9131784/nancy-lieberman-says-there-no-downside-brittney-griner-trying-make-dallas-mavericks
description="Women's basketball pioneer Nancy Lieberman is encouraging Baylor star Brittney Griner to seize the opportunity to do the same with the Dallas Mavericks, no matter how overwhelmingly remote Griner's chances of sticking actually are."
splunk> exit

[madhadron]

Very cool! I'm pushing the beta release today, which is taking all my bandwidth, so please be patient if I don't get to this immediately.

I'd actually love to do some of your todos and make it into something for our application gallery. Could you give me a little more detail on your ideas for a more full featured REPL?

[sempervictus]

Saw the 0.8 branch - figured as much, no worries. This runs in 0.8 by the way.

Regarding the REPL shell - response and input objects to be structured in the running context - fields can be classes operating on their values session cache for operations on the fly method definitions to define transforms and data aggregation REPL execution means that linked data would be transformed at every evaluation by data brought into the shell at the last iteration. Session cache upload to the server as a marshalled/structured output for later recovery of the session itself (cheap Gemstone implementation for Splunk sessions stored in an index)

I'm thinking Pry because of its great introspection and debugging tools, as well as the fact that it's already a fully functional REPL shell in itself.

There's another project called Xiki which i've looked at for more advanced shells, but its overkill IMO and does not like Rubinius at present.

[sempervictus]

SplunkShell class created from Service with appropriate runtime methods. Execute commands, run a full shell, logging of queries and output, log parsing to JSON.

Sample Output:

ruby 8_spunk_pry.rb 
Splunk::SplunkShell> splunk_shell
splunk> search index=main | head 1

Thu, 04 Apr 2013 15:57:48 EDT title="Wiidow replaces slain W. Va. sheriff"
link=http://rss.cnn.com/~r/rss/cnn_topstories/~3/X1D4OI5GHEE/index.html
description="A man suspected of killing a West Virginia sheriff remained hospitalized early Thursday, with officials describing his condition as "touch and go."<div class="feedflare">
<a href="http://rss.cnn.com/~ff/rss/cnn_topstories?a=X1D4OI5GHEE:XEIKmxRZQIw:yIl2AUoC8zA"><img src="http://feeds.feedburner.com/~ff/rss/cnn_topstories?d=yIl2AUoC8zA" border="0" /></a> <a href="http://rss.cnn.com/~ff/rss/cnn_topstories?a=X1D4OI5GHEE:XEIKmxRZQIw:7Q72WNTAKBA"><img src="http://feeds.feedburner.com/~ff/rss/cnn_topstories?d=7Q72WNTAKBA" border="0" /></a> <a href="http://rss.cnn.com/~ff/rss/cnn_topstories?a=X1D4OI5GHEE:XEIKmxRZQIw:V_sGLiPBpWU"><img src="http://feeds.feedburner.com/~ff/rss/cnn_topstories?i=X1D4OI5GHEE:XEIKmxRZQIw:V_sGLiPBpWU" border="0" /></a> <a href="http://rss.cnn.com/~ff/rss/cnn_topstories?a=X1D4OI5GHEE:XEIKmxRZQIw:qj6IDK7rITs"><img src="http://feeds.feedburner.com/~ff/rss/cnn_topstories?d=qj6IDK7rITs" border="0" /></a> <a href="http://rss.cnn.com/~ff/rss/cnn_topstories?a=X1D4OI5GHEE:XEIKmxRZQIw:gIN9vFwOqvQ"><img src="http://feeds.feedburner.com/~ff/rss/cnn_topstories?i=X1D4OI5GHEE:XEIKmxRZQIw:gIN9vFwOqvQ" border="0" /></a>
</div><img src="http://feeds.feedburner.com/~r/rss/cnn_topstories/~4/X1D4OI5GHEE" height="1" width="1" />"
splunk> exit
=> nil
Splunk::SplunkShell> splunk_log
=> {"search index=main | head 1"=>
  {"2013-04-04 16:08:02 -0400"=>
    [{"_bkt"=>"main~129~284FA551-EFC1-4A15-ACF0-9D8A0024994E",
      "_cd"=>"129:750299",
      "_indextime"=>"1365105905",
      "_raw"=>
       "Thu, 04 Apr 2013 15:57:48 EDT title=\"Wiidow replaces slain W. Va. sheriff\"\nlink=http://rss.cnn.com/~r/rss/cnn_topstories/~3/X1D4OI5GHEE/index.html\ndescription=\"A man suspected of killing a West Virginia sheriff remained hospitalized early Thursday, with officials describing his condition as \"touch and go.\"<div class=\"feedflare\">\n<a href=\"http://rss.cnn.com/~ff/rss/cnn_topstories?a=X1D4OI5GHEE:XEIKmxRZQIw:yIl2AUoC8zA\"><img src=\"http://feeds.feedburner.com/~ff/rss/cnn_topstories?d=yIl2AUoC8zA\" border=\"0\" /></a> <a href=\"http://rss.cnn.com/~ff/rss/cnn_topstories?a=X1D4OI5GHEE:XEIKmxRZQIw:7Q72WNTAKBA\"><img src=\"http://feeds.feedburner.com/~ff/rss/cnn_topstories?d=7Q72WNTAKBA\" border=\"0\" /></a> <a href=\"http://rss.cnn.com/~ff/rss/cnn_topstories?a=X1D4OI5GHEE:XEIKmxRZQIw:V_sGLiPBpWU\"><img src=\"http://feeds.feedburner.com/~ff/rss/cnn_topstories?i=X1D4OI5GHEE:XEIKmxRZQIw:V_sGLiPBpWU\" border=\"0\" /></a> <a href=\"http://rss.cnn.com/~ff/rss/cnn_topstories?a=X1D4OI5GHEE:XEIKmxRZQIw:qj6IDK7rITs\"><img src=\"http://feeds.feedburner.com/~ff/rss/cnn_topstories?d=qj6IDK7rITs\" border=\"0\" /></a> <a href=\"http://rss.cnn.com/~ff/rss/cnn_topstories?a=X1D4OI5GHEE:XEIKmxRZQIw:gIN9vFwOqvQ\"><img src=\"http://feeds.feedburner.com/~ff/rss/cnn_topstories?i=X1D4OI5GHEE:XEIKmxRZQIw:gIN9vFwOqvQ\" border=\"0\" /></a>\n</div><img src=\"http://feeds.feedburner.com/~r/rss/cnn_topstories/~4/X1D4OI5GHEE\" height=\"1\" width=\"1\" />\"",
      "_serial"=>"0",
      "_si"=>["unknown", "main"],
      "_sourcetype"=>"rss_news",
      "_time"=>"2013-04-04T15:57:48.000-04:00",
      "host"=>"unknown",
      "index"=>"main",
      "linecount"=>"5",
      "source"=>"rss_feed",
      "sourcetype"=>"rss_news",
      "splunk_server"=>"unknown"}]}}
Splunk::SplunkShell> exit

Next steps: session persistence, marshalling custom objects and methods to sessions, data transforms, then the real fun - inter-related sources, objects, fields, etc.

[madhadron]

Sorry to be slow about this. Finally have a minute.

I'd actually like to package these up separately for our application gallery (which is something we've been trying to get off the ground for a bit but didn't have anything to shove into it nor enough spare time to generate content for it). They're something that we definitely want out there, but I also have this puritanical goal of keeping the library dependencies of the SDK minimal, and to have all the example code runnable without any more libraries than the SDK itself requires. We have a lot of users who are relatively new programmers, or at least new to Ruby, and small things like needing additional libraries that seem trivial to someone steeped in this stuff have proved real stumbling blocks.

Does this sound acceptable to you?

[sempervictus]

Of course, should I just make a basic shell gem out of this?

Boris Lukashev Systems Architect Semper Victus LLC On Apr 24, 2013 1:37 PM, "Fred Ross" notifications@github.com wrote:

Sorry to be slow about this. Finally have a minute.

I'd actually like to package these up separately for our application gallery (which is something we've been trying to get off the ground for a bit but didn't have anything to shove into it nor enough spare time to generate content for it). They're something that we definitely want out there, but I also have this puritanical goal of keeping the library dependencies of the SDK minimal, and to have all the example code runnable without any more libraries than the SDK itself requires. We have a lot of users who are relatively new programmers, or at least new to Ruby, and small things like needing additional libraries that seem trivial to someone steeped in this stuff have proved real stumbling blocks.

Does this sound acceptable to you?

— Reply to this email directly or view it on GitHubhttps://github.com/splunk/splunk-sdk-ruby/pull/26#issuecomment-16951355 .

[madhadron]

A gem would be great. Then we can get you an individual contributor agreement and put it up with your name on it on our app gallery.