Improve apropos display

[mbauman]

Currently, apropos' output is quite simple — it's just the objects whose docstrings match the search text:

julia> apropos("pearson")
cov
cor

It might be nice to make this display a bit richer. We could additionally print out the first non-code documentation line to the edge of the screen, e.g.,

julia> apropos("pearson")
cov      - Compute the Pearson covariance between the vector(s) in…
cor      - Compute the Pearson correlation between the vector(s) i…

It also might be nice to make it return a Julia type that simply contains an array of objects and their documentation, so IJulia and other non-REPL programs can format it a little nicer or linkify the text for full documentation. This does have the downside of making the display non-incremental, making it feel a little slower, but it may be worth it.

The implementation is also quite simple and should be fairly simple to extend.

[anthonyclays]

I'd like to take this up.

Some questions/remarks: Currently, in 0.3, apropos("pearson") prints this:

julia> apropos("pearson")
Base.cov(v1[, v2][, vardim=1, corrected=true, mean=nothing])
Base.cor(v1[, v2][, vardim=1, mean=nothing])

while in 0.4, apropos seems to have been removed entirely.

What are the most important functions and files I will need to read / modify, and which branch should I work in?

[jiahao]

while in 0.4, apropos seems to have been removed entirely.

@anthonyclays you'll have to update. apropos was very recently reintroduced.

[anthonyclays]

@jiahao Oh, I was on a 7 days old master. Updating now - thanks!

[MichaeLeroy]

I agree that enriching the apropos by providing a good indication as to what the found objects are/do is desirable. However, after studying and playing with Docs.jl and company, I'm concerned that addressing this issue might be easier said than done, possibly leading to confusing or misleading results

The Good apropos and the new Documentation facilities strike me as extremely well designed, providing a solid foundation for development. apropos nicely sifts through (loaded) modules' __META__ information for matches to its target. It has available both the keys (objects) and values (object documentation) of these matches but, currently "merely" prints the keys. So the information needed to implement this improvement is right where it needs to be, which seems pretty good.

The Complicated There is almost an embarrassment of riches in the object documentation available to apropos (but currently unused). It is difficult to know how to consistently extract an appropriate summary from the available information.

Looking at the object documentation for cor and cov in v"0.4.0", @mbauman 's suggestion is quite reasonable. Just dig through the object documentation, select the right Markdown structure and (if needed) truncate it for brevity.

However the situation may be different in the future. Looking at a recent version of Master:

julia> VERSION
v"0.5.0-dev+991"

julia> apropos("pearson")
cor

julia> @doc cor
  cor(x)

  Return the number one.

  cor(X[, vardim=1])

  Compute the Pearson correlation matrix of the matrix X along the dimension vardim.

  cor(X, Y[, vardim=1])

  Compute the Pearson correlation between the vectors or matrices X and Y along the dimension vardim.

  cor(x, y)

  Compute the Pearson correlation between the vectors x and y.

So it looks as if the documentation for objects (generic functions in particular) may be getting rewritten to better fit with the new documentation system, with an emphasis on providing information on how to use representative methods of generic functions rather than on providing generic documentation of the function. A naive implementation of the improvement to apropos might look good in the current release but show, rather unhelpfully, in a recent Master:

julia> apropos("Pearson")
cor      - Return the number one.

I suppose that one approach would be to look through the object documentation to print the text in the vicinity of (one of) the matches to the apropos search target.

Any suggestions on how to proceed? Do we need to rethink this improvement in the light of how Julia object documentation appears to be developing? Does it make sense to leave apropos be (it is simple, robust and effective) and consider developing a different tool for more in-depth exploration and presentation of Julia objects? A feature rich apropos-like tool that (optionally) works across all installed/registered modules could be quite cool, but that might be best developed as a package.

[MichaeLeroy]

Since my last comment, I've learned a bit more about the structure of object documentation and now feel that enhancing apropos will be easier than I had thought. This change of attitude comes with the realization that generic function documentation is ordered in a reasonable way, making it easy to identify the most generic definition of a function. (Confusingly to me, the doc macro presents function definitions from more specific to less, which is what made my examination of the cor documentation disheartening.)

So I know how to easily retrieve the most generic definition of a function. So what's the hold-up to solving this issue and having a cooler apropos command? It's truncating this definition, which is typically Markdown, for presentation. I don't yet know how to do that. Does anyone have any suggestions?