sourcegraph / srclib

srclib is a polyglot code analysis library, built for hackability. It consists of language analysis toolchains (currently for Go and Java, with Python, JavaScript, and Ruby in beta) with a common output format, and a CLI tool for running the analysis.
https://srclib.org
Other
942 stars 62 forks source link

"Fix" docs. #206

Closed rothfels closed 9 years ago

rothfels commented 9 years ago

Many of the code references in markdown were broken when these files started being generated by protocol buffers.

It would be better to have the code generation print the comments necessary to render the code block by name (e.g. "DefKey") -- that's left as an issue.

rothfels commented 9 years ago

@beyang , so I can deploy this.

rothfels commented 9 years ago

also, if you know how to force gogoprotobuf to force a comment in certain positions...we can make this better, but I couldn't figure out how to do that in a timebox

beyang commented 9 years ago

@shurcooL or @slimsag would know better than I about how to get gogoproto to do stuff. Would this be easy, guys?

dmitshur commented 9 years ago

I'm not sure what's going on here. Can you elaborate on what command is being run, and what generates what? Also what it should generate instead.

rothfels commented 9 years ago

@shurcooL sorry for the missing context.

We have a markdown file with an embedded reference to code living in a protobuf generated file, e.g. https://github.com/sourcegraph/srclib/blob/master/graph/def.pb.go#L42

We would like to reference the code in the markdown file as [[.code "graph/def.pb.go" "DefKey"]] but this syntax requires a // START DefKey OMIT and // END DefKey OMIT comment wrapped around the DefKey function in the pb-generated file.

That file is generated from https://github.com/sourcegraph/srclib/blob/master/graph/def.proto via go generate ./... -- do you know how to get the comments above in the generated output?

slimsag commented 9 years ago

I don't think you are likely to find an easy way to have gogoproto generate wrapping // START DefKey OMIT and // END DefKey OMIT comments in the output Go file.. You could have them placed in the type definition documentation (i.e. right above type DefKey struct {, but not at the end of the type definition (after the closing }).

What reads/needs those odd START and END omit blocks? It sounds like the correct fix may be making that program aware of Go types or something.

rothfels commented 9 years ago

@slimsag : https://github.com/sourcegraph/srclib/tree/master/docs#embedding-code-segments

(it's mkdocs)

slimsag commented 9 years ago

@rothfels It sounds like the best quick solution is to use the line number format in the doc you linked:

[[.code "grapher/grapher.go" 23 30]]
rothfels commented 9 years ago

@slimsag that's brittle since the file is generated and the line numbers can change, but the person building / deploying the docs might not realize this

rothfels commented 9 years ago

My strategy (without fitting this into the code generation) is to link to lines from a specific revision, e.g. [[.code "https://raw.githubusercontent.com/sourcegraph/srclib/bf4ec15991ed05161dad3694f8729d48c5124844/graph/ref.pb.go" 14 44]]

slimsag commented 9 years ago

To conclude my thoughts here, I see these three options:

  1. Keep your approach. The docs won't be broken but may become outdated quickly.
  2. Fork gogoprotobuf to emit // START TheTypeName OMIT // END TheTypeName OMIT segments around each data type. We would have ot maintain this fork in the future.
  3. Add a preprocessing step that uses go/parser to parse the generated protobuf file, insert the OMIT comment segments, and write the files back out. This is probably the most robust solution.
beyang commented 9 years ago

Option #3 seems like the best option. We can punt on that for now, though, and just file it as an issue.