x/tools/godoc: Allow shallow directory listing

[seeruk]

I've been looking into setting up a godoc server internally, but my team and I agreed that browsing godoc is not easy when you have a large $GOPATH. The issue we've experienced comes down to how sub-directories and sub-packages are listed, i.e. directories within directories, and packages within packages are shown. The result is that you end up being forced to use your browser's "Find" functionality to find what you're looking for on the extremely long and hard to follow pages.

Another problem caused by this is that it makes loads the packages homepage a lot slower, because the page can get quite long - even on pretty beefy hardware.

I took a quick look around in the godoc code and found what seems to be a sensible place to put a new option for how packages and sub-directories are displayed and made a fork with the change in over here. I'll submit a PR for that too, but from the contribution guide it looks like I should also submit an issue that it can be linked to.

All this does is allow you to add ?m=shallow to the URL, and it'll only show the top level packages and directories for whatever you're looking at.

Another change I did have in mind was that it'd be nice to specify default PageInfoMode values when godoc starts so you don't always have to add it to the URL, but I've not had chance to look into that yet.

[gopherbot]

Change https://golang.org/cl/135338 mentions this issue: godoc: add 'shallow' PageInfoMode to only show top-level directories in subdirectory listings

[agnivade]

Thanks. I think the correct approach would be to fold sub-directories beyond a certain depth (let's say 2) into a collapsible with a (+) sign which can be expanded.

As of Go 1.11, I have made a small change towards improving this, which is to divide package listings into standard library and third-party and put them under collapsibles. So atleast when you are browsing just the standard library, you can collapse the third party section and vice versa.

I wanted to do this foldable thing, but note that with 1.11, GOPATH has gone away. There is no notion of a single GOPATH anymore which the /pkg/ listing shows. Everything is specific to a module. So once the output becomes module aware, it will automatically show packages only for that module, bringing down the output a lot.

I would request you to wait until https://github.com/golang/go/issues/26827 is fixed. And then after that, if we see that the listing is still unmanageable, we can look into ways to tackle it. But for now, adding an extra flag which might not really be needed down the road seems redundant.

[seeruk]

I think the correct approach would be to fold sub-directories beyond a certain depth (let's say 2) into a collapsible with a (+) sign which can be expanded.

That would certainly be a big improvement. One thing to keep in mind is that different packages will "start" at different depths. We use BitBucket internally now, so ours are like:

bitbucket.org
-> <org name>
   -> <repo name>

So I assume that's what we'd see, but then there are others like gopkg.in users for example, or other people with vanity URLs that may be more like this fictional package:

elliotdwright.com
-> tools
   -> benchmark
   -> blog
   -> cmd
   -> (... several more)
   -> third_party

So, these pages could still end up fairly difficult to browse even collapsed to 2 levels. I know that was just a suggestion thrown out there, but I figure it's worth discussing why this isn't a trivial thing to fix overall. Without folding, and only showing a max depth of... 0, it does get around these issues.

Perhaps they could fold up to the first directory that contains Go files, or a module. So the second example above would have stopped at tools, but the first example would stop at <repo name>?

Everything is specific to a module. So once the output becomes module aware, it will automatically show packages only for that module, bringing down the output a lot.

That's a good point actually, it will be a lot shorter there. In this scenario, how would a team approach setting up a godoc server internally? Realistically I just want something that centralises our package documentation to make it easier to discover the packages that we have, and then browse their documentation.

[agnivade]

In this scenario, how would a team approach setting up a godoc server internally?

We are still working it out. As @ianthehat has pointed out here - https://github.com/golang/go/issues/24661#issuecomment-418211394.

I would prefer to wait until the godoc module story is sorted out. And then revisit this if we still end up with very long pages. Before that, it is too early to say what the right approach will be and we may end up doing something which might become moot later on.