search

JoshCheek / seeing_is_believing

Displays the results of every line of code in your file
1.31k stars 54 forks source link

Support pp for large/complex structures #44

Closed avdi closed 7 years ago

avdi commented 10 years ago

first off, this project is super, super cool. I'm now evaluating it to see if it would be a viable replacement for xmpfilter in RubyTapas.

One feature that I use a lot in xmpfilter is the difference between annotating the end of a line and annotating on the next line. In xmpfilter, if you annotate the following line, it switches to using pp indented output instead of inspect output. This is handy for showing the contents of large or nested data structures.

Here's an example using xmpfilter:

h = {
  foo: 42,
  bar: {
    baz: 1,
    buz: 2,
    fuz: 3,
  },
  wibble: {
    magic_word: "xyzzy",
  }
}

h # => {:foo=>42, :bar=>{:baz=>1, :buz=>2, :fuz=>3}, :wibble=>{:magic_word=>"xyzzy"}}

h
# => {:foo=>42,
#     :bar=>{:baz=>1, :buz=>2, :fuz=>3},
#     :wibble=>{:magic_word=>"xyzzy"}}

Is this a possibility in SIB?

JoshCheek commented 10 years ago

Hi, Avdi. Thanks for checking out the lib!

Short Response

Composing the long response made me think through a bunch of stuff, so I'm rather tempted to see if I can't hack it in for a short-term solution, and then figure out the stuff below for a proper long-term solution. I'll probably give that a shot today or tomorrow, unless @lukeaiken would like to try (let me know and I'll walk you through what I'm thinking).

Long Response

We already display the previous line

If you set the --xmpfilter-style flag, it will already display the value on the previous line. I didn't realize that xmpfilter would pretty inspect the previous result in this case, so it does not do that at present.

{ foo: 42,
  bar: {
    baz: 1,
    buz: 2,
    fuz: 3,
  },
  wibble: {
    magic_word: "xyzzy",
  }
}
# => {:foo=>42, :bar=>{:baz=>1, :buz=>2, :fuz=>3}, :wibble=>{:magic_word=>"xyzzy"}}

Recording with pretty_inspect

Currently, when it records values, it does so with inspect. So the first change is that we would need to pass what inspect method to use (e.g. inspect or pretty_inspect). This isn't too difficult, we already pass certain arguments down, e.g. how many arguments to capture.

Displaying the result on multiple lines

The difficult portion is deciding what to do when it comes back.

A possible alternative

Given that you use emacs, you have a lot of control. The xmpfilter style of annotating in comments is not the only way to use the lib, e.g. you can get a JSON representation of the output, which you could then display however you liked (e.g. lined up in an adjacent window, but then they aren't even added back into the document).

screen shot 2014-09-06 at 9 58 44 am

Summary

I'm inclined to do this for 3 reasons:

But life is rarely that simple:

So, I'll probably try to hack something in for the short-term (ie a day or two of effort), and then wait a bit before really giving it a good implementation for the long-term.

JoshCheek commented 10 years ago

Here is where they do it. They call that for each line, setting multiline only for the style you showed

Playing with their code:

require 'pp'  # => true

h = { foo: 42,            # => 42
  bar: {
    baz: 1,               # => 1
    buz: 2,               # => 2
    fuz: 3,               # => 3
  },                      # => {:baz=>1, :buz=>2, :fuz=>3}
  wibble: {
    magic_word: "xyzzy",  # => "xyzzy"
  }                       # => {:magic_word=>"xyzzy"}
}                         # => {:foo=>42, :bar=>{:baz=>1, :buz=>2, :fuz=>3}, :wibble=>{:magic_word=>"xyzzy"}}

PP.pp(h, '', 40)                # => "{:foo=>42,\n :bar=>{:baz=>1, :buz=>2, :fuz=>3},\n :wibble=>{:magic_word=>\"xyzzy\"}}\n"
  .gsub(/\r?\n/, 'PPPROTECT');  # => "{:foo=>42,PPPROTECT :bar=>{:baz=>1, :buz=>2, :fuz=>3},PPPROTECT :wibble=>{:magic_word=>\"xyzzy\"}}PPPROTECT"

So this is harder than I thought, since, it alternates between which method it uses to record the value. So I'd need to know on a line-by-line basis which inspection method to use.

@avdi, do you find you actually use the difference? ie what if there was just a flag you could set to use pretty printed inspections? Then you could seeing_is_believing --xmpfilter-style --pretty-inspect=40 (40 = width). And it would use that for every line.

Like this constraint exists because I'm mimicking the other tool, but it might be that if we identify what the purpose of it is, that there is a better way to integrate it in SiB.

JoshCheek commented 10 years ago

Also, is xmpfilter's line number behaviour correct?

{foo: 42, bar: {baz: 1,buz: 2,fuz: 3},wibble: {magic_word: "xyzzy"}}
# => {:foo=>42,
#     :bar=>{:baz=>1, :buz=>2, :fuz=>3},
#     :wibble=>{:magic_word=>"xyzzy"}}
__LINE__ # => 1
avdi commented 10 years ago

do you find you actually use the difference?

I do, in a really organic way. One of my production values in RubyTapas is that I try to leave as much context on the screen as possible. That means I'm often slowly building up a screenful or more of code a bit at a time, annotating certain parts of it with xmpfilter comments and re-evaluating regularly. When an expression I'm evaluating will produce a short inspection, I put the xmp tag on the same line. When I know it will produce multi-line or complex output, I put the xmp tag on the next line, relying on the pretty_inspect behavior to render it readably. In the end I'm left with a mix of inline and next-line annotations. Switching back and forth between two different types of filter invocation would reformat the whole buffer and would be visually jarring in a way my current workflow isn't.

JoshCheek commented 9 years ago

Context

Hey, @avdi, getting pretty close to having this ready. I need to figure out what width to pass to PP.pp

They default the value to 79, and then pass a width of 5 less than that, so 74. Presumably these numbers are because 80 is a standard terminal, then subtract 1 for the hash to comment the line out, and the next 5 for spaces to line it up with the hash-rocket. But I can get the same results by setting it as low as 40.

Hoping you can let me know how yours gets set. Maybe you explicitly pass it in your rcodetools.el, or maybe something else somewhere else is setting it. To reduce the effort, I've included some bash scripts you can use to figure this out.

Finding the width

This one adds a line to your xmpfilter that will print out the width when running multi-line expressions:

ruby -pi -e 'puts %q(puts "\e[32mWIDTH=#{@width.inspect}\e[0m") if $. == 176' `gem which rcodetools/xmpfilter`

This one will then run your example through xmpfilter, we should see it print out the width:

printf '{foo:42,bar:{baz:1,buz:2,fuz:3},wibble:{magic_word:"xyzzy"}}\n# =>'|xmpfilter

This one will then remove the line I added, returning it to normal.

ruby -ni -e 'print unless $. == 176' `gem which rcodetools/xmpfilter`

This should all work, but if it gets totally horked, you can gem pristine rcodetools to fix it.

Let me know what the width is, I'll mimic the xmpfilter settings.

avdi commented 9 years ago

I don't think I've ever manually set anything up relating to xmpfilter width.

Here's the output:

WIDTH=79
{foo: 42, bar: { baz: 1, buz: 2, fuz: 3 }, wibble: { magic_word: %(xyzzy) }}
# => {:foo=>42,
#     :bar=>{:baz=>1, :buz=>2, :fuz=>3},
#     :wibble=>{:magic_word=>"xyzzy"}}

Thanks for your work on this!

Avdi Grimm http://avdi.org

On Wed, Sep 17, 2014 at 1:04 AM, Josh Cheek notifications@github.com wrote:

Context

Hey, @avdi https://github.com/avdi, getting pretty close to having this ready. I need to figure out what width to pass to PP.pp http://rdoc.info/stdlib/pp/PP.pp

They default the value to 79, and then pass a width of 5 less than that, so 74. Presumably these numbers are because 80 is a standard terminal, then subtract 1 for the hash to comment the line out, and the next 5 for spaces to line it up with the hash-rocket. But I can get the same results by setting it as low as 40.

Hoping you can let me know how yours gets set. Maybe you explicitly pass it in your rcodetools.el, or maybe something else somewhere else is setting it. To reduce the effort, I've included some bash scripts you can use to

figure this out.

Finding the width

This one adds a line to your xmpfilter that will print out the width when running multi-line expressions:

ruby -pi -e 'puts %q(puts "\e[32mWIDTH=#{@width.inspect}\e[0m") if $. == 176' gem which rcodetools/xmpfilter

This one will then run your example through xmpfilter, we should see it print out the width:

ruby -e 'puts "{foo: 42, bar: { baz: 1, buz: 2, fuz: 3 }, wibble: { magic_word: %(xyzzy) }}\n# =>"' | xmpfilter

This one will then remove the line I added, returning it to normal.

ruby -ni -e 'print unless $. == 176' gem which rcodetools/xmpfilter

This should all work, but if it gets totally horked, you can gem pristine xmpfilter to fix it.

Let me know what the width is, I'll mimic the xmpfilter settings.

— Reply to this email directly or view it on GitHub https://github.com/JoshCheek/seeing_is_believing/issues/44#issuecomment-55849167 .

JoshCheek commented 9 years ago

Cool. I've got it working already, but it requires a per-line decision about how to record the expression, which is a pretty big change. So I'm taking advantage of the opportunity to make a lot of big changes and release a major version.

here's an example a cuke based on your example, which is currently passing. If you'd be willing to try it out, let me know, I'll do a prerelease version.

I also moved the markers (annotations) to a place where it will be easy to choose your own, so that will probably happen, and I'll probably also get xmpfilter-style to respect the alignment strategies.

avdi commented 9 years ago

I'm going to try and use this on the next RubyTapas video record (sometime this week), and I'll report back.

Avdi Grimm http://avdi.org

On Sun, Sep 21, 2014 at 10:41 PM, Josh Cheek notifications@github.com wrote:

Cool. I've got it working already, but it requires a per-line decision about how to record the expression, which is a pretty big change. So I'm taking advantage of the opportunity to make a lot of big changes and release a major version https://github.com/JoshCheek/seeing_is_believing/issues/47.

here's https://github.com/JoshCheek/seeing_is_believing/blob/bc5663790cbef44bc48724b4d382d6a8e6600f17/features/xmpfilter-style.feature#L73-88 an example a cuke based on your example, which is currently passing. If you'd be willing to try it out, let me know, and I'll do a prerelease version.

I also moved the markers (annotations) to a place where it will be easy to choose your own https://github.com/JoshCheek/seeing_is_believing/blob/bc5663790cbef44bc48724b4d382d6a8e6600f17/lib/seeing_is_believing/binary/parse_args.rb#L15-20, so that will probably happen, and I'll probably also get xmpfilter-style to respect the alignment strategies https://github.com/JoshCheek/seeing_is_believing/blob/bc5663790cbef44bc48724b4d382d6a8e6600f17/lib/seeing_is_believing/binary/parse_args.rb#L108-112 .

— Reply to this email directly or view it on GitHub https://github.com/JoshCheek/seeing_is_believing/issues/44#issuecomment-56323496 .

JoshCheek commented 9 years ago

Okay, let me get a release out for you, then. Been a hectic day, only just saw this :P

JoshCheek commented 9 years ago

Well, not going to happen today. Wound up sauced and still have to do some things for tomorrow's class.

The git version on this branch is probably fine, was just trying to add some extra tests and more holistic functionality around it. But let me know if anything goes awry and I'll make sure to address that specifically.

JoshCheek commented 9 years ago

Hey, @avdi, just released v3.0.0.beta.1

gem install seeing_is_believing -v 3.0.0.beta.1

I'm not super confident in it, b/c I'm getting nondeterministic CI failures, but tried it out locally and so far haven't had any issues. Give it a shot and let me know if there's anything super broken, I'll hit it up quickly. Also, going to add some nice features for the real 3.0.0 release, so there's that to look forward to!

Example

Given this code

SeeingIsBelieving::VERSION  # =>
{ foo: 42,
  bar: {baz: 1, buz: 2, fuz: 3},
  wibble: {magic_word: "xyzzy"}
} # => 
# =>

When you pipe it into seeing_is_believing --xmpfilter-style, you will get this out:

SeeingIsBelieving::VERSION  # => "3.0.0.beta.1"
{ foo: 42,
  bar: {baz: 1, buz: 2, fuz: 3},
  wibble: {magic_word: "xyzzy"}
} # => {:foo=>42, :bar=>{:baz=>1, :buz=>2, :fuz=>3}, :wibble=>{:magic_word=>"xyzzy"}}
# => {:foo=>42,
#     :bar=>{:baz=>1, :buz=>2, :fuz=>3},
#     :wibble=>{:magic_word=>"xyzzy"}}
avdi commented 9 years ago

I've used this to record a couple episodes now, and it's working great! I was particularly happy to be able to show each step in a method chain.

In order to use it in emacs I simply changed the command for rcodetools.el to use SiB instead.

Minor request: could you make it recognize "#=>" as well as "# =>"? And ideally, normalize the former into the latter in its output? (xmpfilter does this). Sometimes my fingers slip, and also for some reason sometimes rcodetools.el produces "#=>" instead of "# =>" when I hit the keybinding.

Avdi Grimm http://avdi.org

On Tue, Sep 23, 2014 at 12:11 PM, Josh Cheek notifications@github.com wrote:

Hey, @avdi https://github.com/avdi, just released v3.0.0.beta.1 http://rubygems.org/gems/seeing_is_believing/versions/3.0.0.beta.1

gem install seeing_is_believing -v 3.0.0.beta.1

I'm not super confident in it, b/c I'm getting nondeterministic CI failures, but tried it out locally and so far haven't had any issues. Give it a shot and let me know if there's anything super broken, I'll hit it up quickly. Also, going to add some nice features for the real 3.0.0 release,

so there's that to look forward to!

Example

Given this code

SeeingIsBelieving::VERSION # =>{ foo: 42, bar: {baz: 1, buz: 2, fuz: 3}, wibble: {magic_word: "xyzzy"}} # => # =>

When you pipe it into seeing_is_believing --xmpfilter-style, you will get this out:

SeeingIsBelieving::VERSION # => "3.0.0.beta.1"{ foo: 42, bar: {baz: 1, buz: 2, fuz: 3}, wibble: {magic_word: "xyzzy"}} # => {:foo=>42, :bar=>{:baz=>1, :buz=>2, :fuz=>3}, :wibble=>{:magic_word=>"xyzzy"}}# => {:foo=>42,# :bar=>{:baz=>1, :buz=>2, :fuz=>3},# :wibble=>{:magic_word=>"xyzzy"}}

— Reply to this email directly or view it on GitHub https://github.com/JoshCheek/seeing_is_believing/issues/44#issuecomment-56545581 .

JoshCheek commented 9 years ago

@avdi pushed a new ver http://rubygems.org/gems/seeing_is_believing/versions/3.0.0.beta.2

Handles this use-case: https://github.com/JoshCheek/seeing_is_believing/blob/v3.0.0.beta.2/features/xmpfilter-style.feature#L27

avdi commented 9 years ago

Thanks, I'll try it out next time I record!

Avdi Grimm http://avdi.org

On Sat, Sep 27, 2014 at 8:01 PM, Josh Cheek notifications@github.com wrote:

@avdi https://github.com/avdi pushed a new ver http://rubygems.org/gems/seeing_is_believing/versions/3.0.0.beta.2

Handles this use-case: https://github.com/JoshCheek/seeing_is_believing/blob/v3.0.0.beta.2/features/xmpfilter-style.feature#L27

— Reply to this email directly or view it on GitHub https://github.com/JoshCheek/seeing_is_believing/issues/44#issuecomment-57069833 .

JoshCheek commented 9 years ago

If you're hitting annoying issues with versions of Ruby / Parser, I released beta4 which has an update that should fix this. A few other bugs will go away, too. Not all of them, though.

JoshCheek commented 9 years ago

Hi, @avdi.

beta5 is out. It should be stable enough to release as v3.0, but I'm going to wait a bit anyway, b/c I'd like to polish up some editor integrations so it's easier for people to be excited about it. If you wouldn't mind using it and letting me know if you hit any issues, I'd appreciate it.

Some things to look forward to

No Broken Windows

If you uncover any on this version, please let me know. I think it is in a good spot right now and would like to keep it bug-free.

Deprecations:

These all still work, so-as not to break the interface. But since it's a major version change, I took the opportunity to choose more descriptive names for these flags.

Here are some things I'd probably do if you or anyone implied they wanted them

avdi commented 9 years ago

This sounds spectacular! I've installed it on my RubyTapas VM so I'll be putting it through its paces. I may just have to bind that --clean version to a new keybinding.

Thank you!!

Avdi Grimm http://avdi.org

On Sun, Jan 18, 2015 at 3:57 PM, Josh Cheek notifications@github.com wrote:

Hi, Avdi.

beta5 https://rubygems.org/gems/seeing_is_believing/versions/3.0.0.beta.5 is out. It should be stable enough to release as v3.0, but I'm going to wait a bit anyway, b/c I'd like to polish up some editor integrations so it's easier for people to be excited about it. If you wouldn't mind using it and letting me know if you hit any issues, I'd appreciate it. Some things to look forward to

  • It handles all the Ruby 2.2 syntax.
  • It now has only one dependency: Parser
  • There is only one known parsing bug, which should probably be a syntax error, anyway.
  • Handles ENCODING macro
  • Handles exit!
  • Intelligent identification of DATA segment

  • Help examples ar updated

    If you clean the document with the xmpfilter flag set, it will clean the values, but leave the annotations:

    $ cat f.rb {foo: 42, bar: {baz: 1, buz: 2, fuz: 3}, wibble: {magic_word: "xyzzy"}} # => {:foo=>42, :bar=>{:baz=>1, :buz=>2, :fuz=>3}, :wibble=>{:magic_word=>"xyzzy"}}

    => {:foo=>42,

    :bar=>{:baz=>1, :buz=>2, :fuz=>3},

    :wibble=>{:magic_word=>"xyzzy"}}

    >> stdout

    !> stderr

    ~> exception

    $ seeing_is_believing --xmpfilter-style --clean f.rb {foo: 42, bar: {baz: 1, buz: 2, fuz: 3}, wibble: {magic_word: "xyzzy"}} # =>

    =>

    Handles multiple invocations of multi-line output. The format is taken from xmpfilter, but if you wind up using this and have a preference for an alternate one, let me know, b/c I'm not super in love with it. I was thinking maybe another value marker instead of the comma.

    2.times do {foo: 42, bar: {baz: 1, buz: 2, fuz: 3}, wibble: {magic_word: "xyzzy"}}

    => {:foo=>42,

    :bar=>{:baz=>1, :buz=>2, :fuz=>3},

    :wibble=>{:magic_word=>"xyzzy"}}

    ,{:foo=>42,

    :bar=>{:baz=>1, :buz=>2, :fuz=>3},

    :wibble=>{:magic_word=>"xyzzy"}}end

    • It can handle repeated invocations of multiline inspections with leading whitespace. It does this by replacing the first space with a nonbreaking space. Which is a little questionable, but seemed to be the least invasive of my ideas for how to deal with it. ruby bst = Object.new def bst.inspect " 4\n"\ " 2 6\n"\ "1 3 5 7\n" end bst # => 4 # 2 6 # 1 3 5 7

    It handles BEGIN and END

    p 1 # => 1END { p 2 } # => 2 p 3 # => 3BEGIN { p 4 # => 4 BEGIN { p 5 } # => 5 }

    >> 5# >> 4# >> 1# >> 3# >> 2

    It handles exec

    puts "PARENT PID: #$$" # => nilENV["AT_THE_ZOO"] = "hampsters journal frequently" # => "hampsters journal frequently"

    exec 'bash', '-c', <<BASH env | grep ZOO echo CHILD PID $$BASH

    puts "we'll never get here"

    >> PARENT PID: 26457# >> AT_THE_ZOO=hampsters journal frequently# >> CHILD PID 26457

No Broken Windows

If you uncover any on this version, please let me know. I think it is in a good spot right now and would like to keep it bug-free. Deprecations:

These all still work, so-as not to break the interface. But since it's a major version change, I took the opportunity to choose more descriptive names for these flags.

  • --number-of-captures is now --max-line-captures
  • --timeout is now --timeout-seconds
  • --shebang is gone altogether (turns out you can figure out the correct Ruby with RbConfig.ruby)

Here are some things I'd probably do if you or anyone implied they wanted them

  • Custom alignment strategies (e.g. align at 40 characters, but if you can't, fall back to chunk alignment)
  • Alignments for xmpfilter (currently, it stays with whatever you've set)
  • A specific format for multiple multiline outputs with xmpfilter
  • A better solution to errors on lines with value markers (xmpfilter style), I just don't even know what would be correct here. The current solution is at least not-broken.
  • Ability to use custom markers
  • Ability to customize pp output. e.g. Internally, I'm using a data structure which is basically a conjunction of a hash and a struct. Its inspect was terrible, so I played with pp for a while until I got it to look like this https://cloud.githubusercontent.com/assets/77495/5771610/db8d5fa2-9cfd-11e4-81cf-dd02ea591aad.png There's no reason we can't have things like 1.9 style hash keys, smarter word breaking, etc. If you like that, or generally want to play with the representation of anything, let me know. (note that you can use -r flag to require a file, so you could play with this yourself if you were interested)

— Reply to this email directly or view it on GitHub https://github.com/JoshCheek/seeing_is_believing/issues/44#issuecomment-70425548 .

JoshCheek commented 7 years ago

Closing this since v3 is released.