How does App X work?

[rcebulko]

Overview

There are a handful of apps in this repository, each with different owners, deployment strategies, permissions, environments, etc. Most of this is documented, in one way or another, in the project subdirectories READMEs or code comments. The goal of this thread is to collect high-level info about each project (not full deployment instructions or READMEs). The below template contains questions which each app owner could answer to provide a useful starting point for looking at all the apps in one place and seeing where we might want to unify or standardize approaches:

Template

[App Name]

Owner(s): rcebulko, rsimha, ... Directory: /my-app Deployed on: [GCE | GAE | ???] Project ID: amp-github-app-name URL: [None | amp-app-link.appspot.com] README link: README Portability: [Can only work with amphtml | Could be modified to be used on other repos | is repository-agnostic/configurable]

Incoming requests:

• GitHub Webhooks
• Admin pages

Outgoing requests:

• GitHub API
• Travis API

Database: [Yes | No] Authentication [No | GitHub OAuth | Basic Auth] Custom lint config: [Yes | No]

Language/Framework: Probot If Probot:

• Smee channel: http://smee.io/12345
• GitHub user: ampprojectbot
• GitHub App permissions (see https://github.com/settings/apps/my-app-name/permissions):
  • Checks: read & write (update/create check-runs on PRs)
  • Contents: read-only (read commits, branches, PRs, releases, etc)
  • Pull requests: read & write (read PR info, read & write comments)
• Webhooks:
  • Pull request: update check
  • Pull request review comment: unused
  • Pull request review: update check
  • Status: monitor status changes
  • Label: unused
• Personal access token name (from https://github.com/settings/tokens): ampprojectbot (owners bot, maybe other things)
• Required permissions [no idea if these are accurate, just examples]:
  • public_repo: Needed to read PR metadata, ...
  • read: org: Needed to list members of organization teams
  • repo: status: Needed to update check-runs, leave comments

Package management: [Yarn | npm | idk] Testing framework: Jest Test command: npm run test Tests run on presubmit: [Yes | No]

Deployment workflow: [Manual | Automated] Can this be run locally?: [Yes | No] If so:

• Command to run locally: npm run dev
• Env. vars required to run locally:
  • GITHUB_ACCESS_TOKEN: Access token with at least public_repo permission
  • GITHUB_REPO_DIR: Path to local copy of ampproject/amphtml repository (ex. /tmp/amphtml)
  • GITHUB_REPO: Name of repository (ie. ampproject/amphtml or your fork)

Public environment (ie. config): [Shell script &| .env &| app.yaml &| GCE/GAE metadata &| deploy command arguments &| config file(s)] Private environment (ie. secrets, tokens, keys): [Shell script &| .env &| app.yaml &| GCE/GAE metadata &| deploy command arguments &| config file(s)] App uses a private-key.pem file for authentication: [Yes | No]

Plan

I'm going to start posting for the apps I've created/worked on, and then will start trying to do the same for the other apps with as much info as I can find in the repo.

/cc @ampproject/wg-infra let me know if you think of anything that might be missing. Ideally we can identify some pieces of this to standardize, if not with actual infrastructure then at least with more standard processes for things like keeping track of permissions, which apps use which tokens (since you can't see which token is which in the GitHub UI, making it risky to ever delete any), manage deployments, testing, etc.

[rcebulko]

Owners Bot

Owner(s): rcebulko, erwinmombay Directory: /owners Deployed on: GCE (requires local disk for local repo) Project ID: amp-owners-bot URL: No public URL README link: README Portability: Could be modified to be used on other repos

Incoming requests:

• GitHub Webhooks

Outgoing requests:

• GitHub API

Database: No Authentication: No Custom lint config: Yes

Language/Framework: Probot

• Smee channel: http://smee.io/PfdLoWYIFFhgHA
• GitHub user: ampprojectbot
• GitHub app permissions:
  • Checks: read & write (update/create check-runs on PRs)
  • Contents: read-only (read commits, branches, PRs, releases, etc)
  • Issues: read-only (read comments, assignees, labels, etc)
  • Pull requests: read & write (read PR info, read & write comments)
  • Commit statuses: read & write (update/create check runs for commits) [deprecated API]
  • Members: read-only (list members of organization teams)
• Webhooks:
  • Check suite: unused [TODO: verify and unsubscribe]
  • Pull request: update check
  • Pull request review comment: unused [TODO: verify and unsubscribe]
  • Pull request review: update check
  • Status: unused [TODO: verify and unsubscribe]
  • Label: unused [TODO: verify and unsubscribe]
  • Membership: update local team record
  • Team: update local team record
• Personal access token name (from https://github.com/settings/tokens): ampprojectbot (owners bot, maybe other things) (used for admin pages that don't have OAuth from the Probot webhook)
• Required permissions:
  • public_repo: Needed to read PR info, reviews, comments
  • read: org: Needed to list members of organization teams
  • repo: status: unknown

Package management: npm Testing framework: Jest Test command: npm run test Tests run on presubmit: Yes

Deployment workflow: Manual (push to cloud repo, reset GCE instance) Can this be run locally?: Yes (for accessing status/info pages) If so:

• Command to run locally: npm run start
• Env. vars required to run locally:
  • GITHUB_ACCESS_TOKEN: Access token with at least public_repo and read: org permissions
  • GITHUB_REPO_DIR: Path to local copy of ampproject/amphtml repository (ex. /tmp/amphtml)
  • GITHUB_REPO: Name of repository (ie. ampproject/amphtml or your fork)

Public environment (ie. config): Shell script, GCE/GAE metadata, deploy command arguments Private environment (ie. secrets, tokens, keys): Shell script, GCE/GAE metadata, deploy command arguments App uses a private-key.pem file for authentication: Yes

[rcebulko]

Project Metrics

Owner(s): rcebulko Directory: /project-metrics Deployed on: GAE Project ID: amp-project-metrics URL: https://amp-project-metrics.appspot.com README link: README Portability: is repository-agnostic/configurable

Incoming requests:

• Metrics display page
• Badge API (used by (amphtml README)[https://github.com/ampproject/amphtml/blob/master/README.md])
• Metric History API (used by (amphtml METRICS doc)[https://github.com/ampproject/amphtml/blob/master/METRICS.md])

Outgoing requests:

• GitHub API
• CodeCov API
• Travis API

Database: PostgreSQL (CloudSQL) Authentication: No Custom lint config: No

Language/Framework: Python/Flask GitHub:

• Personal access token name (from https://github.com/settings/tokens): ampproject/amp-github-apps/project-metrics
• Required permissions:
  • public_repo: Needed to read PR info, commits, releases Travis API token: required for collecting information on builds CodeCov API token: required for computing code coverage metric

Package management: pip Testing framework: unittest Test command: python -m unittest discover -s . -p '*_test.py' Tests run on presubmit: No

Deployment workflow: Manual (gcloud app deploy app.yaml) Can this be run locally?: No (depends on AppEngine-provided imports)

Public environment (ie. config): env.yaml Private environment (ie. secrets, tokens, keys): env.yaml App uses a private-key.pem file for authentication: No

[rcebulko]

Bundle-size Bot

Owner(s): danielrozenberg Directory: /bundle-size Deployed on: GAE Project ID: amp-bundle-size-bot URL: https://amp-bundle-size-bot.appspot.com README link: README Portability: can only work with amphtml

Incoming requests:

• GitHub Webhooks
• Travis API calls

Outgoing requests:

• GitHub API

Database: Yes Authentication: No Custom lint config: No

Language/Framework: Probot

• Smee channel: ???
• GitHub user: ampprojectbot
• GitHub app permissions:
  • Checks: read & write (update/create check-runs on PRs)
  • Pull requests: read & write (read PR info, read & write comments)
  • Members: read-only (list members of organization teams) [TODO: verify or remove]
• Webhooks:
  • Check suite: re-run bundle size check when requested
  • Check run: re-run bundle size check when requested
  • Pull request: re-run check on new/updated PRs
  • Pull request review: unknown [TODO: verify or unsubscribe]
• Personal access token name (from https://github.com/settings/tokens): None

Package management: npm Testing framework: Jest Test command: npm run test Tests run on presubmit: Yes

Deployment workflow: ??? Can this be run locally?: Yes

• Command to run locally: npm run dev
• Env. vars required to run locally: ???

Public environment (ie. config): app.yaml Private environment (ie. secrets, tokens, keys): db-config.js, .env App uses a private-key.pem file for authentication: Yes

@danielrozenberg Please verify/fill in the ??? blanks if possible

[rcebulko]

PR Deploy Bot

Owner(s): estherkim Directory: /pr-deploy Deployed on: GAE Project ID: amp-pr-deploy-bot URL: https://amp-pr-deploy-bot.appspot.com README link: README Portability: is repository-agnostic/configurable

Incoming requests:

• GitHub Webhooks
• Deployed instances (in Cloud Storage bucket)

Outgoing requests:

• GitHub API

Database: No Authentication: No Custom lint config: Yes

Language/Framework: TypeScript + Probot

• Smee channel: ???
• GitHub user: ampprojectbot
• GitHub App permissions:
  • Checks: read & write (update/create check-runs on PRs)
  • Pull requests: read-only (read PR info)
• Webhooks:
  • Check run: re-run deploy when requested
  • Pull request: re-run deploy on new/updated PRs
  • Personal access token name (from https://github.com/settings/tokens): None

Package management: npm Testing framework: Jest Test command: npm run test Tests run on presubmit: Yes

Deployment workflow: ??? Can this be run locally?: Yes

• Command to run locally: npm run dev
• Env. vars required to run locally: ???

Public environment (ie. config): .env Private environment (ie. secrets, tokens, keys): .env App uses a private-key.pem file for authentication: Yes

@estherkim Please verify/fill in the ??? blanks if possible

[rcebulko]

Test Status Bot

Owner(s): danielrozenberg Directory: /test-status Deployed on: GAE Project ID: amp-test-status-bot URL: https://amp-test-status-bot.appspot.com/ README link: README Portability: is repository-agnostic/configurable

Incoming requests:

• GitHub Webhooks
• Travis API calls
• Build cop rotation API calls
• Web UI (for build cop)

Outgoing requests:

• GitHub API

Database: Yes Authentication: GitHub OAuth Custom lint config: No

Language/Framework: Probot

• Smee channel: ???
• GitHub user: ampprojectbot
• GitHub App permissions:
  • Checks: read & write (update/create check-runs on PRs)
  • Pull requests: read-only (read PR info)
• Webhooks:
  • Pull request: re-run check on new/updated PRs
• Personal access token name (from https://github.com/settings/tokens): None

Package management: npm Testing framework: Jest Test command: npm run test Tests run on presubmit: ???

Deployment workflow: ??? Can this be run locally?: Yes

• Command to run locally: npm run dev
• Env. vars required to run locally: ???

Public environment (ie. config): app.yaml Private environment (ie. secrets, tokens, keys): db-config.js, .env App uses a private-key.pem file for authentication: Yes

@danielrozenberg Please verify/fill in the ??? blanks if possible

[rcebulko]

Just based on what I've gathered looking through the existing apps and READMEs and environment files and configurations, a few free-floating thoughts in no particular order:

• A couple apps have custom ESLint configs; it may be possible to cut some of them down/out so linting is consistent throughout
• Use of databases and authentication so far is pretty sparse; probably not worth unifying/creating any shared structure there
• Since most of our apps use Probot, there's a common shared set of required environment variables (mostly private keys/secrets), and then a handful of application-specific environment variables; likewise, many apps provide their own instructions on how to create a Probot app, set up a Smee channel, request permissions/webhook subscriptions, etc., and each with varying levels of detail
  • It may be worth separating out instructions for creating a GitHub app, setting up a Smee channel, requesting permissions, subscribing to events, and generating the private key into the repository-level README; this will double as a bootstrap guide for those starting new apps
• Most apps share similar npm scripts (start, test), and some have dev and deploy, which are extremely useful; it might be good to have a standard set so the build/test/deploy workflow is constant across all (which could make some automation easier)
• What is our strategy for managing private keys, tokens, secrets, etc? Where does this info live, besides local copies of .env?
• Some apps select the repository to use from an environment variable (which is great!)
  • The name of the environment variable is different from app to app; it may be worth picking one name/convention and sticking with it
  • Future apps should, if possible, be designed around the repository as an environment variable
  • Portable apps should mention that fact in the README, so other interested teams know they can be easily repurposed
• The GitHub UI does not show what uses each access token, and you can't view them once they've been created; it's probably worth creating separate access tokens on a per-app basis named after the app (like ampproject/amp-github-apps/project-metrics created by erwinmombay) rather than named after purpose/consumer (like ampproject-bot-process-github-issues or ampprojectbot (owners bot, maybe other things) which is likely shared by multiple apps and may have unnecessary permissions, but can't be safely altered because we don't know)
• We should probably try to write new apps in TypeScript, like pr-deploy; this would both make it a bit cleaner (less JSDoc and more valuable type-checking) and would also gradually get us more comfortable in the hybrid JS-TS environment, which is what we may be pushing towards in amphtml
• It would be nice for the repository-level README to list each of the apps, a quick description of what it does, and possibly a link to the README
• There are definitely some shared common GitHub operations (creating/updating check-runs, fetching PR info) that could be worth hoisting to a shared interface

[rcebulko]

Additional thought: Project Metrics doesn't really fit this repo. It's not a GitHub app, it's a separate Python web app that should probably live in its own repository. If that project were to be moved out, all other projects here are Probot GitHub apps.

[rcebulko]

Relevant to #410 /cc @ampproject/wg-infra

[rsimha]

A couple apps have custom ESLint configs; it may be possible to cut some of them down/out so linting is consistent throughout

This is definitely worth doing. Eslint lets you add an .eslintrc at the root with repo-wide settings, and separate .eslintrc files in each directory that contain rules specific to that directory.

It would be nice for the repository-level README to list each of the apps, a quick description of what it does, and possibly a link to the README

+1!

Project Metrics doesn't really fit this repo. It's not a GitHub app, it's a separate Python web app that should probably live in its own repository. If that project were to be moved out, all other projects here are Probot GitHub apps.

Any ideas for where you'd move it?

Another suggestion for standardization (if it's not already done): Pick one workflow for package management (yarn vs. npm). We use yarn for amphtml because npm would generate different package-lock.json files on different OSs, causing the file to bounce back and forth as people made changes.

[rcebulko]

@rsimha Re: last item, all of the apps are currently using npm. I think we've been lucky in that we all use Linux

[rcebulko]

Re: second item, I think project-metrics should probably just live in its own mini-repo? WDYT? Not sure of any obvious other repo that should hold it.

[rcebulko]

I think the ideal here would be if we could deploy all apps together (see https://probot.github.io/docs/deployment/#combining-apps). Might take some massaging and time to get there, but a reasonable goal. Then we could set up auto-deploys of the whole repo on a (nightly?) regular interval, get all the configuration in one place, etc.

[rcebulko]

Archiving this.