milahu / nixpkgs-watcher

notifications for my favorite apps
MIT License
2 stars 0 forks source link

get a push feed of all notifications in the nixpkgs repo #1

Open milahu opened 1 year ago

milahu commented 1 year ago

... including edits of title/comment, reactions (upvote, downvote)

we need to watch edits of titles, so authors can fix their post titles, so our filters can match against the correct post titles

milahu commented 1 year ago

https://docs.github.com/en/account-and-profile/managing-subscriptions-and-notifications-on-github/setting-up-notifications/configuring-notifications

You can receive notifications for activity on GitHub.com in the following locations.

  • The notifications inbox in the GitHub.com web interface
  • The notifications inbox on GitHub Mobile, which syncs with the inbox on GitHub.com
  • An email client that uses a verified email address, which can also sync with the notifications inbox on GitHub.com and GitHub Mobile

"web notifications" sounds good, but these are no push notifications, they are just stored somewhere in the webui

i get push notifications when i have issues or PRs open in my browser = i see live edits and updates without page reload. maybe we can abuse that? but then, will that scale to a million subscriptions? a billion subscriptions? i guess github will not like this

Benefits of using an email client for notifications

One benefit of using an email client is that all of your notifications can be kept indefinitely depending on your email client's storage capacity.

Email notifications also allow flexibility with the types of notifications you receive and allow you to choose different email addresses for updates. [...] For more information, about your email customization options, see "Customizing your email notifications."

About participating and watching notifications

When you watch a repository, you're subscribing to updates for activity in that repository. Similarly, when you watch a specific team's discussions, you're subscribing to all conversation updates on that team's page. For more information, see "About team discussions."

hmm. does github scale to a million teams? a billion teams?

one question is, how much do we want to rely on github? or should this also work with gitea and gitlab servers?

milahu commented 1 year ago

a million events per day

https://blog.mergify.com/handling-780k-github-events-per-day/

they use github webhooks as event source

github webhooks requires GitHub Enterprise account (?) but the nixos org already has an enterprise account (?)

Doing more triage upfront, and using a Redis stream per pull request rather than per organization, gave us the granularity needed to organize and sort the pull requests by priority. Each pull requests is now scored and processed in the right order of priority.

source code was removed from repo, but is still in old commits

https://github.com/Mergifyio/mergify/blob/3ba5c4bb655988f9e21c0e3cdc2172c4ce408644/mergify_engine/worker.py

milahu commented 1 year ago

https://docs.github.com/en/rest/activity/events

Events are optimized for polling with the "ETag" header. If no new events have been triggered, you will see a "304 Not Modified" response, and your current rate limit will be untouched. There is also an "X-Poll-Interval" header that specifies how often (in seconds) you are allowed to poll. In times of high server load, the time may increase. Please obey the header.

curl \
  -H "Authorization: Bearer $GITHUB_TOKEN" \
  https://api.github.com/repos/NixOS/nixpkgs/events
{
  "message": "Bad credentials",
  "documentation_url": "https://docs.github.com/rest"
}

so repo events are private ...

... but org events are public

curl https://api.github.com/orgs/NixOS/events >orgs-nixos-events.json
cat orgs-nixos-events.json | jq -r '.[] | .type'

PushEvent
PullRequestEvent
IssueCommentEvent
PushEvent
PullRequestEvent
PullRequestReviewEvent
PullRequestEvent
PushEvent
PullRequestEvent
PullRequestEvent
PullRequestReviewEvent
PullRequestEvent
PullRequestEvent
PullRequestReviewCommentEvent
PullRequestReviewEvent
PushEvent
IssuesEvent
PullRequestEvent
PullRequestReviewEvent
PullRequestReviewCommentEvent
PullRequestReviewCommentEvent
PullRequestReviewCommentEvent
PullRequestReviewEvent
IssueCommentEvent
PushEvent
PullRequestEvent
IssueCommentEvent
PullRequestReviewCommentEvent
PullRequestReviewCommentEvent
PullRequestReviewEvent

we need IssuesEvent and PullRequestEvent

https://docs.github.com/en/developers/webhooks-and-events/events/github-event-types#issuesevent

Key Type Description
action string The action that was performed. Can be one of opened, edited, closed, reopened, assigned, unassigned, labeled, or unlabeled.

https://docs.github.com/en/developers/webhooks-and-events/events/github-event-types#pullrequestevent

Key Type Description
action string The action that was performed. Can be one of opened, edited, closed, reopened, assigned, unassigned, review_requested, review_request_removed, labeled, unlabeled, and synchronize.
milahu commented 1 year ago

https://stackoverflow.com/questions/48527126/how-to-get-realtime-push-events-from-github-in-all-public-repositories-of-all-us

Public events API delayed by five minutes

The public events API is now delayed by five minutes.

As a result, when you query the API, the most recent event returned will have occurred at least five minutes ago.

The API is otherwise unchanged and continues to support pagination and fetching of up to 300 events. No other events APIs are affected and they will continue to deliver events in real time.

milahu commented 1 year ago

https://docs.github.com/en/developers/webhooks-and-events/webhooks/about-webhooks

Webhooks allow you to build or set up integrations, such as GitHub Apps or OAuth Apps, which subscribe to certain events on GitHub.com. When one of those events is triggered, we'll send a HTTP POST payload to the webhook's configured URL.

https://docs.github.com/en/developers/webhooks-and-events/webhooks/creating-webhooks

Creating a webhook is a two-step process. You'll first need to set up what events your webhook should listen to. After that, you'll set up your server to receive and manage the payload.

You can use the REST API to manage repository, organization, and app webhooks.

github enterprise account is *not* required

webhooks must be enabled by the repo owner

https://github.com/$owner/$repo/settings/hooks/new

Which events would you like to trigger this webhook?

Let me select individual events.

issues pull requests

yes!

instant push events for everything except upvotes/downvotes