Making TomDoc play nicely with code folding

[RichMorin]

Like many documentation formats, TomDoc does not play nicely with the code-folding support found in editors such as TextMate. Specifically, although the editors allow code folding, they don't fold the TomDoc blocks (which may well be longer than the code itself).

Although this problem could be addressed by reworking each editor, this seems infeasible. So, I suggest a slight opening up of TomDoc's syntax.

Consider your example code (multiplex), as listed on

http://tom.preston-werner.com/2010/05/11/tomdoc-reasonable-ruby-documentation.html

Using code folding, I can turn the method's three lines of code into a single line. Unfortunately, this leaves me with 11 lines of unfoldable commentary. A typical code file might have dozens of methods, each of which might have dozens of lines of header comments. How am I supposed to view only the lines of interest?

In my own code, I've developed two workarounds for this problem: comment methods and internal comment blocks:

• Comment methods handle the problem of documenting classes and methods. They are simply "do-nothing" methods with internal comment blocks, as described below. I use the name header_comment for these, but any mnemonic name would serve.
• Internal comment blocks differ from current TomDoc format only slightly. Because they are located inside the method being discussed, they get folded along with the code.

  class A_Class

    def header_comment
      #
      # ... TomDoc comment text ...
    end

    def a_method
      #
      # ... TomDoc comment text ...

      ... code ...
    end

  end

In TextMate, this folds down to:

  class A_Class

    def header_comment  ooo

    def a_method  ooo

  end

Would you accept an extension to TomDoc to support this syntax? Alternatively, is there a cleaner solution?

[holman]

Really not a fan of this change (at least in terms of changing the TomDoc standard to comply). It makes glancing for comment declarations much harder. If anything, I'd rather fold code blocks down and keep the comments unfolded. I'd rather delegate to each user to modify their editor to do what they expect.

[mtodd]

I also much prefer to just fold code and leave comments unfolded. The documentation is far more interesting and important than the body of the method. Plus, with syntax highlighting, it's not that distracting, either.

[RichMorin]

I think these comments miss several critical points:

• This isn't a syntax change, just an alternate format. Any user who likes the current format is not affected (though TomDoc implementers would have to make a small change to their code).
• Delegating editor modifications to users is largely infeasible, even when the editors are Open Source.
• I have no argument with anyone who likes keeping their comments unfolded. However, the current format is a show-stopper for my workflow, because it doesn't let me concentrate on the code. So, why not allow this as a variant syntax?

[holman]

Having multiple perspectives on syntax seems kind of detrimental to the idea of TomDoc as a standard in the first place.

Overall, though, part of the spirit of TomDoc is not about conforming to spec; it's mostly about finding what works for you and going for it. I think rearranging comments inside of method declarations could make sense for you, and by all means go for it. I just think adding more official versions to the spec would muddy the waters.

[RichMorin]

As I posted initially, "Would you accept an extension to TomDoc to support this syntax? Alternatively, is there a cleaner solution?". If TomDoc is (in principle) open to this sort of extension, I'd be happy to look into offering a revision. If not, or there's a cleaner solution, I'd rather avoid wasting the effort.

My take on changes to the standard is that they can and should follow a working implementation, usage, feedback, etc. In short, I'm in no hurry to suggest changes to the standard.

[leejarvis]

I guess there's not a cleaner solution if this is something that fits your workflow. Personally I wouldn't use this, and I think a massive amount of people are unlikely to find this a useful extension. Both those writing documentation and those reading it when parsing source code are accustomed to reading documentation above the method declaration, on a personal level, do you really want to work against that, @RichMorin?

I'm plus one on not touching the standard in regards to this extension, but I don't see why it couldn't be an option.

[RichMorin]

Actually, I don't mind swimming against the flow on matters that affect my efficiency. I use pretty conventional Ruby style, but feel free to define my own conventions if I think they make the code more readable. (That freedom is one of the reasons I love Ruby.) Putting comment blocks inside of methods and classes is the least of it (and, FWIW, has a pedigree that goes back several decades). However, some of the comments I've seen call for a bit of justification, so...

I really can't speak for how anyone else works, but most of the time, I only want to look at the "working set" of code I'm actively working on or interacting with. I mean, isn't that one of the main arguments for modularity? So, I want everything else to be largely invisible, but easy to find.

In practice, this means that I have several TextMate (TM) projects, each covering a particular directory tree. I use code folding to manage methods and classes, TM tabs (etc) to manage files, and WindowShade X to manage TM project sessions and other long-running application contexts (eg, Adium, Colloquy, Eudora, Firefox, iTunes, Safari, Terminal). Transient apps (eg, Numbers, OmniGraffle, Preview, SketchUp) simply reside in the background (eg, DragThing, the Dock, the Finder) until needed.

My primary development environment is a Mac Pro with around 9.75 MP of screen real estate: 2560_1600 + 1200_1920_2 + 1366_768 (for outbound screen-sharing). This gives me the ability to keep all of the relevant code and apps available and visible or at least close to hand. This reduces my context switching and short-term memory needs, more than justifying (IMHO) the expense and hassle.

[vlbrown]

Options are, in my opinion, always a good thing. I sincerely doubt that "a massive amount of people are unlikely to find this a useful extension." How do they know? How do you?

As Henry Ford said,"If I had asked people what they wanted, they would have said faster horses." I'm quite certain you can find people who think code folding is a Very Bad Idea. Or thought so until they had it.

At least one person would find this useful. That person thinks it's important and useful enough to post about. No one who doesn't find it useful would be impacted in the least (so those of you who "wouldn't use it" are entitled to your opinions but those opinions don't count for much. :-)

[leejarvis]

@vlbrown My opinions are just that, as are @RichMorin's. I don't expect them to count for much, I'm offering my 2 cents just like everyone else is.

@RichMorin Thank you for expanding on that, but I apologize if I made you feel you had to, that wasn't my intention.

Anyway, back to the point at hand. I'm not sure this is an issue for this project. This project is the base for the TomDoc specification, an official specification. I think this is a discussion which would possibly fit a project that implemented the TomDoc specification as a parser. I think it would fit nicely as a parsing extension (adding support for nesting comments inside the method body).

To reiterate, I'm -1 on having this added as an additional version to the spec, but +1 on implementing it as a parsing extension to a project implementing this specification.

[RichMorin]

Thanks, @injekt, for your clarification. Question is, what now? Obviously, I'm free to fork the code base and issue a pull request, but I'd like to hear from Tom (or whoever) that the request will be considered. Is there anything else I should do?

Incidentally, do you happen to know the etymology of "inject", as used in the method call? I tried Googling for this and even pulled out my copies of the SmallTalk-80 books, but didn't find anything. I'd like some history to provide context for the method name...

[leejarvis]

The thing is, there isn't a code base here. Only the specification belongs here. I would recommend diving into defunkt/tomdoc, though. This library implements a simple parsing of the TomDoc standard. It's a little incomplete and doesn't have much maintenance, though. I believe @drbrain has started implementing a parser into the official RDoc library, maybe he can provide more insight into that.

It's funny you mention "inject". I'm not sure of the complete context or decision for this method name. Some believe that it actually makes more sense than reduce, though. I believe the main reason for this name is for the historical link to smalltalk. I guess you could think about it like this: An object is being injected into another object which is usually what reduce is used for. ie: (1..5).inject(0, :+) #=> 15 could be explained as each object in the range is being injected into 0, in this case using the :+ method as a short hand. But that starts to make less sense when you have examples like this (1..5).inject(20, :-) :-)

[RichMorin]

It may be that the simplest plan is to create a wrapper script that munges my flavor of TomDoc (RichDoc? :-) into the standard syntax. Once I have that working to my satisfaction, I can use it with any TomDoc implementation and offer it as sample code.

[drbrain]

RDoc implements TomDoc 1.0.0-rc1. I only worry about technical issues with the spec, so inside or outside a method is up to Tom

IMO, if your editor can't fold comments outside of def you need to find a better editor. I know vim can do it, and if vim can do it then so can emacs. If you're working around the limitations of your tools you probably need better tools.

[RichMorin]

@drbrain may well be right about text editors. I'm still using TextMate, hoping that 2.x will resolve some of the 1.x problems. I've tried BBEdit 9.x, but it doesn't do this sort of code folding, either. I've downloaded MacVim and may give that a try, when I find a tuit (I first used vi in 1982 :-). Emacs seems like an interesting OS, lacking only a decent text editor...

Anyway, as I understand it:

• The spec is Tom's, as is a somewhat dated and simplistic library.
• Changing the spec is a bigger deal than changing the library, but neither may be in the cards.
• RDoc only supports the spec, and Tom hasn't commented on my proposal.

So, again, looks like I need to write a wrapper script and/or fork a TomDoc implementation.

[holman]

Options are, in my opinion, always a good thing.

That made my skin crawl. Software should be opinionated, the most of which specs should be opinionated. Adding in different formats willy-nilly is the quickest way to sink the ship.

[RichMorin]

Adding in different formats willy-nilly is the quickest way to sink the ship.

Pythonistas claim that Ruby and Perl have too many WTDI. Vi users have been saying that about Emacs for decades. Forth and Lisp have long been criticized for their lack of constraints and structure. All of these criticisms have some essential validity, but the software in question also serves many users, so it must be doing something right.

I think it's a question of balance and control. How much flexibility is too much; how do you keep the chaos under control. I was quite interested to see @wycats lightning talk at GoGaRuCo; he implemented a specialization of Ruby syntax in a few dozen lines of Rubinius code. Do we want a world where folks are free to "Fork Ruby" (as Dave Thomas put it, a few years ago, in his keynote)? I would say yes, as long as it doesn't get in the way of cooperation.

In any case, allowing folks to move comment blocks inside methods (etc) is hardly going to sink the ship...

[mojombo]

@RichMorin: These are some interesting ideas you've brought up, thanks for letting us know how what your workflow is like. In Ruby, comments above methods/classes is idiomatic. I don't think complicating the spec to accommodate a quirk in one specific tool is wise. With TomDoc I'm aiming to make the spec as simple as possible by reducing the number of arbitrary options you have to consider. I'm curious what you've been doing for code you've worked on before you started using TomDoc. How have you been coping with Ruby code that uses RDoc or YARD?

[RichMorin]

Well, it's a quirk in at least two tools (TextMate and BBEdit), but I get your point.

Obviously, I would have the same problems with RDoc or YARD. The reason I posted a note here is that I like the feel of TomDoc more than that of the others (though I am a huge fan of the deeper analysis and data collection that YARD performs). It makes me sad to see so much inflexibility over such a minor variation, but I'll live.