issues are behind, update and share the code

rakyll commented 8 years ago

I enjoy the idea of searching and being able to access the issues offline, but the master seems to be always a bit behind the current issue tracker if I depend on this repo.

Could you share the code that generates the repo and so we can update the index locally ourselves if the upstream is behind?

rakyll commented 8 years ago

Ok, I found it. The importer is already at https://github.com/bradfitz/issuemirror, but is not visible at all.

Any plans to move it under cmd?

bradfitz commented 8 years ago

issuemirror is a support library and is meant to be generic for any project.

Actually, the plan was for github.com/bradfitz/go-issue-mirror to have zero code, and only data. But then it slowly grew code.

servegoissues should really live in issuemirror, but oh well.

I haven't yet checked in the generator anywhere. I'll keep this bug open about it, or you can file one under github.com/bradfitz/issuemirror, which is where the code will go (under cmd, like you said).

dmitshur commented 8 years ago

servegoissues should really live in issuemirror, but oh well.

If it's better, why don't we move it? I'm not opposed. Or do you mean that it's not worth the effort?

Then again, if "issuemirror ... is meant to be generic for any project" is true, then servegoissues doesn't really belong there since it depends heavily on go-issue-mirror data.

bradfitz commented 8 years ago

I'd like to start mirroring a number of github project's issues so I could work with them all offline. Is there anything Go-specific about servegoissues?

dmitshur commented 8 years ago

I'd like to start mirroring a number of github project's issues so I could work with them all offline.

That's very, very easy.

Is there anything Go-specific about servegoissues?

Here's an exhaustive list of things that are Go-specific about servegoissues:

The name. I named it servegoissues because it was meant to be a specific tool for serving issues of the Go project (i.e., https://github.com/golang/go/issues), and that way it could be in your PATH without conflicting with other things.
The data source. In line with the above point, it takes no arguments and hardcodes the github.com/bradfitz/go-issue-mirror/issues package as its source of issues.
The PR #11 currently hardcodes the link to original issue as a link to https://github.com/golang/go/issues.

That's it.

If you want to turn servegoissues into something that's general for serving issues form any issuemirror.Root source, then it's very trivial, just need to rename it to be more generic and give it flags to control the source of issues.

dmitshur commented 8 years ago

A single serveissues (or whatever it's called) binary can also be made to serve issues from multiple projects at once. For that, you'll just need to change the URL scheme and have a router that parses out the prefix of the URL which determines the project name. E.g.:

~~http://localhost:8080/123~~ - current url for Go issue 123
http://localhost:8080/go/123 - potential new url that serves Go issue 123
http://localhost:8080/another-project/123 - serves another-project issue 123

~~Then supply a issuesapp.Options.RepoSpec which extracts it from the *http.Request.~~ Edit: The API has changed, it's done via RepoSpecContextKey now. That value gets passed to all issues.Service operations as the second parameter, and Get/List/etc. can return corresponding issues for the specified project.

I'm happy to help with implementing this (if the above explanation isn't enough), or implement it myself. The code is already meant to support this use case. You can see an example here.

dmitshur commented 8 years ago

@bradfitz, in https://github.com/bradfitz/go-issue-mirror/pull/6#issuecomment-242926641 you said:

I'd actually prefer that this repo be purely data and no code, and servegoissues live in what was supposed to be the generic counterpart to this repo: github.com/bradfitz/issuemirror

Maybe we can move servegoissues there (leaving a pointer in this repo) and I can give you access to that one? I'd like this repo to be only updated by a bot regularly.

Has that changed given what I told you above about what it takes to make a more general version of servegoissues that can handle other projects too?

I want to keep working on issuesapp to make its API better and add some features, so it'd be more convenient for me if you can get me push access wherever this ends up so I can make trivial API updates (like #5).

FWIW, my suggestion would be for you to to review and merge #11, then move servegoissues as is to another repo (where I can have push access and ability to not break it), and we can continue the discussion/work about a more general tool for serving issues for multiple projects in there.

bradfitz commented 8 years ago

Committed the tool to update the issues in https://github.com/bradfitz/issuemirror/commit/635c1c53868e7eb76fc78d4d27a7dce75c9b2320

Still not running regularly, though.

dmitshur commented 7 years ago

@bradfitz Is this project (as well as issuemirror) still needed? Or is it better to remove or deprecate it, given we have x/build/maintner and x/build/maintner/godata now, which as far as I can tell are superior and do everything issuemirror + go-issue-mirror did, but more efficiently.

Or is there some use case that maintner doesn't cover, making this project still helpful and worth keeping?

If you want to deprecate/remove this, I can easily redo servegoissues on top of maintner API (if you would like to have it around).

bradfitz commented 7 years ago

I would LOVE servegoissues on top of maintner.

And I would gladly delete these repos then.

I'm sure I have servegoissues around if you can't find it. Let me know?

dmitshur commented 7 years ago

I would LOVE servegoissues on top of maintner.

And I would gladly delete these repos then.

Awesome! Glad to hear that.

I'm sure I have servegoissues around if you can't find it. Let me know?

Sorry, what do you mean?

dmitshur commented 7 years ago

Maybe you misread when I said this:

(if you would like to have it around)

I meant... "if you think servegoissues is worth having and find it useful". Which you've answered as yes.

It's easy to find, it's right here in this repo. ;)

rakyll commented 7 years ago

Please do not delete this repo until something equivalent at x/build exists.

dmitshur commented 7 years ago

I'm working on the new servegoissues on top of *maintner.GitHub API now.

@bradfitz, in 2016, you said:

I'd like to start mirroring a number of github project's issues so I could work with them all offline. Is there anything Go-specific about servegoissues?

I saw that the godata corpus has multiple repositories, so I initially thought I'd implement that ability to access them:

google/acme
google/go-genproto
google/go-github
google/google-api-go-client
google/pprof
golang-tr/go-tour-tr
camlistore/go4
golang/oauth2
golang/example
letsencrypt/boulder
golang/oauth
googleapis/googleapis
go/golang
googleapis/toolkit
kubernetes/kubernetes
golang/gddo
org/go
googlecloudplatform/gcloud-golang
bradfitz/http2
golang/go
grpc/grpc-go
googlecloudplatform/google-cloud-go
golang/tour
googleapis/gax-go
golang/build
agl/xmpp-client
x/tools
microsoft/vscode-go

However, it turns out only golang/go repo has non-zero issues:

golang/build - 0 issues
golang/example - 0 issues
golang/gddo - 0 issues
golang/go - 21280 issues
golang/oauth - 0 issues
golang/oauth2 - 0 issues
googlecloudplatform/google-cloud-go - 0 issues
google/pprof - 0 issues
grpc/grpc-go - 0 issues
google/go-github - 0 issues
microsoft/vscode-go - 0 issues
letsencrypt/boulder - 0 issues
googleapis/googleapis - 0 issues
google/go-genproto - 0 issues
googleapis/gax-go - 0 issues
bradfitz/http2 - 0 issues
org/go - 0 issues
x/tools - 0 issues
go/golang - 0 issues
google/google-api-go-client - 0 issues
googlecloudplatform/gcloud-golang - 0 issues
golang-tr/go-tour-tr - 0 issues
google/acme - 0 issues
googleapis/toolkit - 0 issues
agl/xmpp-client - 0 issues
camlistore/go4 - 0 issues
golang/tour - 0 issues
kubernetes/kubernetes - 0 issues

So I guess it's not worth it.

But let me know if you'd still prefer for me to make it possible to access multiple repos in the new servegoissues. The URL scheme, as I imagine it, would be as I described in https://github.com/bradfitz/go-issue-mirror/issues/7#issuecomment-242993530, so something like:

http://localhost:8080/golang/go/123 - potential new url that serves golang/go issue 123

Otherwise, it'll continue to be http://localhost:8080/123 to access golang/go issue 123.

dmitshur commented 7 years ago

I think I'll go with the http://localhost:8080/golang/go/123 scheme anyway, just so the code is general and works with any *maintner.Corpus.

dmitshur commented 7 years ago

What font would you like the new servegoissues to use?

I was going to just go with sans-serif and such, but now tempted to use the Go font. It looks nice (to me), and that way the issue tracker has a slightly more distinct, Go-specific look.

I'll... go with the Go font, and if you don't like it, we can revert to sans-serif, etc.

dmitshur commented 7 years ago

Ok, I'm done with the initial prototype. It's at a point where most things work, and whatever minor things are left (e.g., adding support for additional event types, colors for issue labels, etc.) are best handled as individual issues later on. There's one major known TODO left (making it update the corpus in background while servegoissues app is running).

Basically, I'd like for you to play with the current version and provide feedback before I spend more time on it. So now it's just a matter of deciding on a place for the package and getting it past code review.

@bradfitz, where would you like the new servegoissues package to be?

dmitshur commented 7 years ago

Here are some screenshots of what it looks like right now. /cc @bradfitz @rakyll

bradfitz commented 7 years ago

However, it turns out only golang/go repo has non-zero issues:

That's just a maintnerd config thing. We can mirror more. Or other projects can choose different sets.

Definitely keep the ability to view multiple.

We should mirror the golang/oauth2 and golang/tour etc repos probably.

I'll... go with the Go font, and if you don't like it, we can revert to sans-serif, etc.

Let's just use sans-serif. I'm fine with the Go font personally but I always hear complaints that it looks bad on Windows or at some DPIs or zooms. I've never seen ugliness myself, but I'm not a font nerd. I'd rather just pick sans-serif neutrally and let users tell their browser what sans-serif they like.

@bradfitz, where would you like the new servegoissues package to be?

Want to put it under golang.org/x/build/maintner for now? Somebody wanted to move maintner away to its own repo, which we might do later, but we'd do it while preserving history with a mirror of the git repo filtered away.

golang.org/x/build/maintner/cmd/maintserv ?

dmitshur commented 7 years ago

Want to put it under golang.org/x/build/maintner for now? Somebody wanted to move maintner away to its own repo, which we might do later, but we'd do it while preserving history with a mirror of the git repo filtered away.

golang.org/x/build/maintner/cmd/maintserv ?

Sure. Is it fine it imports some of my dependencies from github.com/shurcooL? I'd really prefer not to have to vendor those. I promise to maintain them and not break the build (for more than 5% in a year).

Also, worth noting, it'll only serve issues for now, not CLs or other data in *maintner.Corpus. So is maintserv still an okay name for it? Or do you plan/want to have it serve/display more in the future?

Let's just use sans-serif.

All right. I'll take them out when sending a CL tomorrow. (Bummer though, I got to really like them by now and want to see them in more places lol.)

Although... Could we use it to start, and remove when someone complains? I doubt this is a tool that a lot of the public will be interested in, so maybe the target audience will appreciate it more. It'd be a chance to try it out, since that didn't work out on the Go blog. Well, I'm fine either way.

bradfitz commented 7 years ago

Sure. Is it fine it imports some of my dependencies from github.com/shurcooL?

That's fine.

Also, worth noting, it'll only serve issues for now, not CLs or other data in *maintner.Corpus. So is maintserv still an okay name for it? Or do you plan/want to have it serve/display more in the future?

maintserve is a good name even if it's only GitHub issues for now. And maintserve is still a good name if/when it does more.

(I meant to include the trailing -e in maintserve earlier, but it's hot today and my brain is kinda mush)

Although... Could we use it to start, and remove when someone complains?

Go font is fine if you care. We'll be the main users, and it makes us happy, so sure.

dmitshur commented 7 years ago

Go font is fine if you care. We'll be the main users, and it makes us happy, so sure.

Great!

One last thing/question. I've decided to place the implementation of the issues.Service interface, which maintserve will use, together with all the other implementations of that interface at https://github.com/shurcooL/issues#directories (the directories). It'll be easier to maintain it for me if it's there, since I can update all implementations in 1 commit.

So I made the PR to add the new *maintner.Corpus-backed implementation there, would you want to review it when you have a chance? It's https://github.com/shurcooL/issues/pull/5.

dmitshur commented 7 years ago

Since that PR and the maintserve are related, I'll send the CL right now. It probably helps to see them in context.

dmitshur commented 7 years ago

Sent CL 52932. Note that it depends on shurcooL/issues#5 to compile (you'll need to check out that PR's branch if you want to test it).

dmitshur commented 7 years ago

That CL is merged, thanks for reviews!

The next generation version of servegoissues is now available!

golang.org/x/build/maintner/cmd/maintserve

Anyone can go get -u it and try it out.

@rakyll, you said in https://github.com/bradfitz/go-issue-mirror/issues/7#issuecomment-319588574:

Please do not delete this repo until something equivalent at x/build exists.

Can you please try out the new command and let me know if you consider it equivalent, therefore making this repo obsolete? Or is there anything else that needs to be ported from here?

As far as I can tell, this repository (and maybe https://github.com/bradfitz/issuemirror) are now completely superseded and can be safely deprecated (or removed). Let me know if that's not the case.

bradfitz / go-issue-mirror

issues are behind, update and share the code #7