package description is redundant with package comment.

[kardianos]

The package description is redundant with package comment. Also keywords have been explicitly not added for the same reason you don't see websites doing any more.

[mattfarina]

@kardianos thanks for the feedback. I have a couple questions...

1. Package descriptions can be short and really long. For example, see the gob package comment. When you need a short tagline how do you handle it with the variability in length of package comments?
2. All of the other language package descriptions include keywords that can be used for search. This is used in support of search. Even Cargo Crates (for Rust) has this. How would suggest bubbling up important information?

This is all in support of the application information use case. That is tools like Crates, npm, Packagist, CPAN, and so forth.

What information would you see being needed for these types of applications to make it easy to search and display useful information? Why?

This is a use case I'd like to see handled well.

[kardianos]

Look to how godoc handles it. There is a convention to take the first section before an empty comment to use that as the summary. See: http://godoc.org/?q=gob

Also, are you familiar with the package import comment?

[mattfarina]

I'm familiar. Here's what I'm seeing.

If I go look at some Go tools and packages in other places I see tag lines like:

• "Manage a cluster of Linux containers as a single system to accelerate Dev and simplify Ops." (Kubernetes)
• "An open platform for distributed applications for developers and sysadmins" (Docker)
• "A high-productivity web framework for the Go language." (Revel)
• "Fast, unopinionated, minimalist web framework" (express - Node.js)
• "The PHP Unit Testing framework." (PHPUnit)

These taglines help convey what it does and will all fail golint.

When I look in godoc I see things like:

• "Package cli provides a minimal framework for creating and organizing command line Go applications." (codegangsta/cli)
• "Package websocket implements the WebSocket protocol defined in RFC 6455." (gorilla/websocket)
• "Package mgo offers a rich MongoDB driver for Go."

The taglines above have more pop than the brief package description. Every description starting with "Package" or breaking from Go conventions doesn't work great for a listing.

Thoughts?

[kardianos]

I thought this was a package meta-data file. The popping descriptions above are for projects. The godoc descriptions are for packages.

Related comment, I really like that Go doesn't need package meta-data like this. If imports for a package could be aliased then a per package meta-data file makes sense. But in go that's not the case.

The "import.version" field is honestly the only thing not currently derivable from just inspecting the package files (well, license too). Everything else can be inspected, probed, and derived. Just a thought.

[mattfarina]

@kardianos How would you derive the owners of the project is if you needed to reach them with a security issue? If you had to deal with projects in a private repo how would you do that?

The scope of the spec is a great observation. You likely would only need one per project in the sense you're using the term. Go doesn't have a notion of project but rather package and some call the internal packages to these sub-packages. I'm less interested in the name at the moment and more interested in solving for the use cases.

You've said information can be derived elsewhere but have not shown me other ways to solve the use cases. Can you point me to some specifics and share how those do it well?

[technosophos]

"Derivability" implies inspection of source, which is not really a viable thing to require of all of your package management tools (especially when you start talking about binary distribution). However, writing a tool that completes some fields of a metadata file sounds like a fine approach to me. So I would argue for including it in the metadata file and allowing tooling to generate it for projects that feel like they conform enough to expectations that it makes sense to do so. There's definitely precedent for that.

It is worthwhile to answer the question, though, about whether we're talking about package metadata or project metadata. Here's my take: Realistically, we don't fetch individual Go packages. We fetch projects. "Package management" in the industry sense (CPAN, NPM, Cargo, etc) really corresponds to the project level, not the individual Go package level. Likewise, version management actually operates at the project level, even though we sometimes pretend that it's Go package level (Godep does, at least).

[kardianos]

@technosophos revisions happen at a repository level. Practically versions can apply to multiple repositories at once, or can apply to only a portion of a repository.

In go, I work with packages. I don't work with repositories. My vendor tool works with packages, not with repositories. I don't care if you use a mono-repo or a package per repo. Go works on packages.

@mattfarina I would follow the import path or origin path and determine the correct organization and follow their policy.

In C# I fundamentally work with solutions and projects. A project is either a DLL or an executable. In java we have jars. PHP doesn't compile and requires external encapsulation, like a repo, to define it. I have experience with other languages too, but I handle languages differently. Go's dependencies are unique because it lacks a project file; everything in Go is defined in the file's "import" section, no shortcuts exist. When you move a package in Go you rewrite import paths.

---

Use cases:

• I want to use a specific version of a package: use vendor-spec, optionally copy into "vendor" folder.
• I want to use private packages: Load keys or auth into config. Use "go get", possibly setup a http meta-handler.
• I want to fork a Go repository and use my fork: rewrite the import paths with mvpkg tool.
• I want to have a single version of an import for my application and all deps: tools can ensure that today. I use kardianos/govendor to ensure this. No extra meta-data file required.
• store dependent packages in my vcs and update them: Use the "vendor" folder and a tool like "kardianos/govendor". No extra meta-data needed.
• As a team, have the same setup and retrieve the same packages: Use "vendor-spec" and optionally copy dependencies into "vendor" folder and check them in.
• I can easily scan my package tree to know the license in use: open issue on "vendor-spec" about that. I'm hesitant on this because the full license text can be important, though I admit the style is often easy to identify. Copyright holders and author list can also be important for legal. Some projects will have multiple copyright notices in same package... But maybe.
• I can find general information about a package: General information is tough. Is there a general email for the go project? CockroachDB? Maybe... Packages used? I'm not against having a format to put information in, but this is info that a human will need to read anyway, so I'm not confident it is worth putting into a machine readable format. Open to it though. I think many pieces of info could be aggregated from a tool that reads go source.
• I can contact a package owner: see above bullet.

[mattfarina]

@kardianos we obviously approach these problems in different ways and that's ok. What I'll be curious of is what the long tail of developers find useful. This is all really about making Go projects and developers successful after all. I'm really uninterested in any project winning but rather Go developers having their jobs be a little easier and more enjoyable.

When I read your use cases what I'm seeing is a description of your preferred flow. They aren't so much use cases or requirements but more often a description of how you choose to do things. I'm sure a little of that has come into mine (even outside the solution sections). Any use case or requirement should be able to be implemented in more than one system or way. They should be implementation agnostic.

I'm also skeptical of some of your methodologies. For example, you suggest rewriting the import path. That's been described as a problem in the Go space rather than something we should be doing.

[kardianos]

If you move a package and you compile with the go tool, you need to rewrite import paths.

A number of people didn't like import path rewrites to vendor packages, thus the vendor folder was born. But it wasn't a technical issue.

On Thu, Sep 3, 2015 at 11:38 AM Matt Farina notifications@github.com wrote:

@kardianos https://github.com/kardianos we obviously approach these problems in different ways and that's ok. What I'll be curious of is what the long tail of developers find useful. This is all really about making Go projects and developers successful after all. I'm really uninterested in any project winning but rather Go developers having their jobs be a little easier and more enjoyable.

When I ready your use cases what I'm seeing is a description of your preferred flow. They aren't so much use cases or requirements but more often a description of how you choose to do things. I'm sure a little of that has come into mine (even outside the solution sections). Any use case or requirement should be able to be implemented in more than one system or way. They should be implementation agnostic.

I'm also skeptical of some of your methodologies. For example, you suggest rewriting the import path. That's been described as a problem in the Go space rather than something we should be doing.

— Reply to this email directly or view it on GitHub https://github.com/mattfarina/pkg/issues/3#issuecomment-137537763.

[mattfarina]

For the record, I'm considering dropping the description and keywords. More so for keywords because of search engines doing a better job and often not using them. I'm going to look into some of the package managers that are using keywords to see how they are leveraging them.

[mattfarina]

For the time being I'm keeping the keyword and description fields. I understand that a modern search engine doesn't need keywords to be effective and that the package description is similar to the description field here. The slight difference in how these would be used provide a useful reason to have them but keep them optional.