Use push notifications for repository updates

dlang / dub-registry

Online registry for dub packages

Boost Software License 1.0

82 stars 64 forks source link

Use push notifications for repository updates #10

Open s-ludwig opened 11 years ago

s-ludwig commented 11 years ago

The registry is currently living on the edge with it's GitHub API use. 60 non-cached requests per hour are allowed and jus registering two projects at once that contain some tags/branches will hit that limit.

When using OAuth2 to access the API, the limit gets increased to 2000 and it also becomes possible to use the PubSubHubbub API to avoid polling, making the registry more responsive and saving bandwith.

Bitbucket probably has something similar.

eskimor commented 11 years ago

I personally would not mind to release my software manually, via some "dub release" command, thus dub could be totally independent from github?

s-ludwig commented 11 years ago

Well I would ;) For sure I will not remove that feature, it's one of the most important ones for me. Enabling releases that are not publically hosted may be a good enhancement, though. But that will require disk space to host the uploaded .zip files somewhere.

eskimor commented 11 years ago

Hosting packages seems to be a good idea:

Released versions are completely independent from the github repo, could be deleted, renamed, ...
Support for not publicly hosted packages/ or packages hosted on a non supported platform (no github/bitbucket)
Packages can contain different files than the repository. (E.g. only a sub-set, package.json gets enriched with the version extracted from the tag name directly, or applicable version() statements, ...)

Of course the resources would need to be provided somehow, maybe I could figure something out, I'll try as soon as I have some prototype of a building/hosting server implementation.

p0nce commented 11 years ago

Do the registry absolutely need to poll github in the first place? What I miss is a manual notification button in the dub registry website to avoid to wait 30 minutes when fixing an error. My current work-around is to delete package/re-add it which doesn't sound like a good idea in retrospect :)

MartinNowak commented 10 years ago

I'll have a look at this when I find some time.

MartinNowak commented 10 years ago

OK, I played around a little with the github API. Webhooks are a nice thing, but we'd need the repo owner's OAuth access for the write:repo_hook scope to install them. I still think, it's a good idea to implement this, so if I find even more time I'll do it.

s-ludwig commented 10 years ago

Hm okay, that's a bit of a downer. Maybe there is a way to abuse something else? Maybe the "watch" functionality can be used to aggregate all changes, so that a single request gets all updates at once?

MartinNowak commented 10 years ago

We could have a pseudo user, that watches all those repos. But from what I read you can't get notifications about commits. Watching only notifies about pull requests and comments.

s-ludwig commented 10 years ago

I was hoping the the data of the "News feed" could be accessed, as that also contains "git push" notifications. But I'm not sure if those are for all watched repositories or just for the ones where the user has push access.