Proposal for Search Result Filtering and Organization: Filter by Tags

[carlisia]

Overview

We propose to increase the number of filtering criteria for search beyond free-text search.

Filtering by Tags

Individual projects will be allowed to add a “TAGS” file to the root of the project; which will list the tags that the project should be associated with. The “TAGS” file is a simple parseable format. Each line in the “TAGS” file will be a tag to associate with the project.

To maintain consistency, the following transformations will be applied to the tags:

• All tags will be lower-cased and trimmed (both left and right trimming will be applied). Also, multiple space characters will be transformed to single spaces.
• The final list of tags will be deduped.
• The file format for the “TAGS” file will be utf8.

Example content of a file: database replication

The go server will maintain an approved list of tags that it will accept and display. Any tags not in this list will be ignored. This list should be stored in a single file, so that additions or deletions to the list can be made by a pull request modifying only one file. Each time Godoc.org re-retrieves the documentation for a project, it will also re-evaluate the contents of the associated “TAGS” file: if the “TAGS” file has changed or the approved list of tags have changed.

Contributors to this proposal

@carlisia @gdey @rafaeljusto

[adg]

I like the end goal, but I have some issues with this proposal.

Godoc.org works on packages, but this proposal suggests a TAGS file in the repository root. Do the tags apply to all packages inside the repository? I think that tags as a concept should apply to packages, not repositories, and the packages inside a repository should be able to differ in their tags.

Adding yet another ALLCAPSFILENAME to repositories is messy. Is there a better place to store such tags?

What makes tags a stronger signal than keywords in a package's doc comment? Why add this extra mechanism when full-text search may well do the job for us? (Google ignores meta tags on web pages, for example.)

I'm concerned that tags are easily gamed by people that wish to abuse the system.

Curating a list of tags approved is a manual job. Who is going to do that? What's the process?

[rafaeljusto]

@adg Yes, It makes more sense of tagging the package rather than the repository. The initial idea was tagging the whole repository, but I forgot that Godoc.org will look individually for packages inside the repository.

About retrieving keywords from the comments, wouldn't we catch to many false positives? The idea of using the comments is good to avoid the TAGS file, but we could make it explicity in comments, something like:

// HelloWorld is responsible for demonstrating the language.
//
// tags: beginners, tutorial
func HelloWorld() {
  println("Hello World")
}

About managing the list of know tags, the idea is that Godoc.org would retrieve the allowed tags automatically from a file committed in Godoc's repository. I'm still thinking in a way to automate the content of this file. Maybe we could have an algorithm, that if a tag appears frequently in different repositories, we could alert the "TAG file administrator" to add it. A manual step here is important to avoid tag abuses (e.g. bad language).

[adg]

we could make it explicity in comments, something like:

So far we have managed to prevent people putting special syntax or identifiers in godoc comments (the comments displayed by godoc and godoc.org). I would like to maintain the status quo here.

A better option is to use source-level comments (not documentation comments) of the form //go:doctags foo bar. That (//go:) is the convention for go-related comment directives.

But in general I'm against adding any kind of tagging mechanism until it's been shown that full text search is ineffective.

[gdey]

@adg The original end goal was to provide a browsable list of categories that packages can be subscribe to. It, also, would allow one to narrow down the type of packages one was search for. For example a search for music could potentially have lots of false positives; however one could say in the category of database.

I think the //go:doctags works, and as @rafaeljusto the we forgot that there could be more then one package in a repo.

[adg]

@gdey yes, a special comment works, but the introduction of such a mechanism will affect the entire Go ecosystem. We cannot do this lightly.

I am opposed to introducing a special comment until we have tried and failed with other approaches.

[gdey]

@adg I am fine with waiting on this too see if the other approaches prove to be insufficient. I think it would be we good to modify the this issue to change from the TAG file to the comment, for the reasons you pointed out. Mainly, that there are more then one package, per repo. The other option is to move the tag file into the package directory… I'm not too sure about that – as it just feels messy. One of the things I wanted to do was make it easy to implement. The easier something is to do; the more likely it will get done.

[adg]

Yep, if we do take this approach then I favor a comment over tags. That should be clear when we revisit the issue later. Thanks.

[gdey]

@carlisia just reminded me of https://github.com/avelino/awesome-go; which is a curated list of go projects, with categories. If I remember correctly this was one the reasoning behind this. Enable gddc to have such a browsable list of categories, that projects could be categorized by in a automated way.

Just adding this to this issues to help with the reasoning.