search

getsentry / eng-pipes

Automations for the Engineering organization ([get]sentry CI/CD tooling, metrics, issues/label management, ...)
Apache License 2.0
5 stars 1 forks source link

Expand to `codecov` somehow #482

Closed chadwhitacre closed 1 year ago

chadwhitacre commented 1 year ago

For https://github.com/getsentry/team-ospo/issues/79 I need to roll out Sentry's routing and triage process in the codecov org. I've set up a second GitHub App for this purpose (yay CoveCod! :), and I need to send the GH event stream from that app somewhere for processing. If we can, it seems best to send both event streams (getsentry and codecov) to our existing eng-pipes deployment, so that we don't have to maintain two separate deployments. We could maintain two deployments, but we already have a third viable org (syntaxfm; no immediate plans to roll out automations there) and should only expect more in the future as Sentry grows. I want a consistent GH engagement management process across all orgs. I would like to send everything through our single eng-pipes.

The primary change here will be in getClient (I'm expecting some changes in the GitHub brainlets as well), and the biggest thing to talk through up front (assuming we're good with running everything through a single deployment) is configuration. We currently assume a single GitHub App, and we use these envvar keys to configure it:

https://github.com/getsentry/eng-pipes/blob/af1f4cac38068c50e21f81464ad4a193f9be3ecb/.env.example#L5-L12

Supporting an arbitrary number of GitHub Apps (one per org) is a little futzy using envvars, but I think something like this might work?

# Keep this global for all orgs/apps. Easier to rotate.
GH_WEBHOOK_SECRET=""

# App for getsentry
GH_APP_1_NAME="getsentry"
GH_APP_1_IDENTIFIER="deadbeef"
GH_APP_1_SECRET_KEY='-----BEGIN RSA PRIVATE KEY-----feedbeef-----END RSA PRIVATE KEY-----'

# App for getsentry
GH_APP_2_NAME="codecov"
GH_APP_2_IDENTIFIER="ba5eba11"
GH_APP_2_SECRET_KEY='-----BEGIN RSA PRIVATE KEY-----b01dface-----END RSA PRIVATE KEY-----'

I would parse this out at startup-time (somewhere in/under buildServer.ts), but only fail at runtime (as we do today) when there is no app configured for the requested org.

Update

I ended up going with a github-orgs.yml for configuration, because a) I need to scope the repo allow-lists to orgs and those are too cumbersome for environment variables, and b) we want to move towards a YAML config file anyway to bring in some config from security-as-code.

PRs

chadwhitacre commented 1 year ago

Went around the horn in Slack about whether to have multiple apps or a single app that we would keep unlisted and filter out any unwanted org traffic from (if anyone were to bother to get it installed outside of our orgs). We decided to proceed with two apps for now to avoid having to come up with a brand that works for the general case vs. getsantry and CoveCod.

chadwhitacre commented 1 year ago

Below is an audit of getClient in 94396e5aa9994ad5266d71670a523f29d053cc79 (current HEAD of main). I guess the main take-away is that we seem to sometimes pass a dynamic value for the second arg, and sometimes we use OWNER from config. I think we probably want to always pass a dynamic value, and we probably want to be able to turn off certain brainlets (we don't want all of the GoCD integrations in codecov).

https://github.com/getsentry/eng-pipes/blob/94396e5aa9994ad5266d71670a523f29d053cc79/src/config/index.ts#L11

def
---
src/api/github/getClient.ts:37:                                   export async function getClient(type: ClientType, org: string | null) {
src/api/github/getClient.ts:57:                                   'Must pass org to `getClient` if getting an app scoped client.'

use
---
src/utils/getOssUserType.ts:32:                                   const octokit = await getClient(ClientType.User, org);

src/brain/ghaCancel/index.ts:27:                                  const octokit = await getClient(ClientType.App, owner);
src/brain/issueLabelHandler/followups.ts:143:                     const octokit = await getClient(ClientType.App, owner);
src/brain/issueLabelHandler/followups.ts:82:                      const octokit = await getClient(ClientType.App, owner);
src/brain/issueLabelHandler/route.ts:141:                         const octokit = await getClient(ClientType.App, owner);
src/brain/issueLabelHandler/route.ts:86:                          const octokit = await getClient(ClientType.App, owner);
src/brain/issueLabelHandler/triage.ts:111:                        const octokit = await getClient(ClientType.App, owner);
src/brain/issueLabelHandler/triage.ts:61:                         const octokit = await getClient(ClientType.App, owner);
src/brain/projectsHandler/project.ts:65:                          const octokit = await getClient(ClientType.App, owner);

src/api/github/getSentryPullRequestsForGetsentryRange.ts:40:      const octokit = await getClient(ClientType.App, OWNER);
src/brain/gocdSlackFeeds/deployFeed.ts:76:                        const octokit = await getClient(ClientType.App, OWNER);
src/brain/notifyOnGoCDStageEvent/index.ts:276:                    const octokit = await getClient(ClientType.App, OWNER);
src/brain/pleaseDeployNotifier/actionViewUndeployedCommits.ts:71: const octokit = await getClient(ClientType.App, OWNER);
src/brain/requiredChecks/getAnnotations.ts:85:                    const octokit = await getClient(ClientType.App, OWNER);
src/brain/typescript/getProgress.ts:21:                           const octokit = await getClient(ClientType.App, OWNER);
src/utils/db/getFailureMessages.ts:43:                            const octokit = await getClient(ClientType.App, OWNER);
src/webhooks/pubsub/index.ts:53:                                  const octokit = await getClient(ClientType.App, OWNER);

src/api/github/getChangedStack.ts:22:                             const octokit = client || (await getClient(ClientType.App, OWNER));
src/api/github/getRelevantCommit.ts:18:                           const octokit = client || (await getClient(ClientType.App, OWNER));

test def
--------
test/jest.setup.ts:3:                                             jest.mock('@api/github/getClient');
src/api/github/__mocks__/getClient.ts:163:                        export async function getClient(type: ClientType, org?: string) {
src/api/github/__mocks__/getClient.ts:172:                        throw Error('Org required for mock getClient()');

test use
--------
src/utils/getOssUserType.test.ts:16:                              octokit = await getClient(ClientType.User);

src/brain/ghaCancel/index.test.ts:23:                             octokit = await getClient(ClientType.App, 'getsentry');
src/brain/githubMetrics/index.test.ts:64:                         octokit = await getClient(ClientType.App, 'getsentry');
src/brain/notifyOnGoCDStageEvent/index.test.ts:101:               octokit = await getClient(ClientType.App, 'getsentry');
src/brain/pleaseDeployNotifier/index.test.ts:40:                  octokit = await getClient(ClientType.App, 'getsentry');
src/brain/requiredChecks/getAnnotations.test.ts:10:               octokit = await getClient(ClientType.App, 'getsentry');
src/brain/requiredChecks/index.test.ts:69:                        octokit = await getClient(ClientType.App, 'getsentry');
src/utils/db/getFailureMessages.test.ts:16:                       octokit = await getClient(ClientType.App, 'getsentry');
src/api/github/getSentryPullRequestsForGetsentryRange.test.ts:15: getsentry = await getClient(ClientType.App, 'getsentry');

src/brain/issueLabelHandler/index.test.ts:77:                     octokit = await getClient(ClientType.App, 'Enterprise');
src/webhooks/pubsub/stalebot.test.ts:18:                          ...(await getClient(ClientType.App, 'Enterprise')),

src/brain/projectsHandler/index.test.ts:44:                       octokit = await getClient(ClientType.App, 'test-org');

src/brain/gocdSlackFeeds/deployFeed.test.ts:519:                  const octokit = await getClient(ClientType.App, OWNER);
src/brain/gocdSlackFeeds/deployFeed.test.ts:650:                  const octokit = await getClient(ClientType.App, OWNER);
chadwhitacre commented 1 year ago

Thinking next about OWNER, but that leads right into SENTRY_REPO and GETSENTRY_REPO.

def
---
src/config/index.ts:12:                                           export const SENTRY_REPO = process.env.SENTRY_REPO || 'sentry';
src/config/index.ts:13:                                           export const GETSENTRY_REPO = process.env.GETSENTRY_REPO || 'getsentry';

use
---
src/api/github/getRelevantCommit.ts:22:                           repo: GETSENTRY_REPO,
src/api/github/getRelevantCommit.ts:44:                           repo: SENTRY_REPO,
src/api/github/getSentryPullRequestsForGetsentryRange.ts:103:     repo: GETSENTRY_REPO,
src/api/github/getSentryPullRequestsForGetsentryRange.ts:46:      repo: GETSENTRY_REPO,
src/api/github/getSentryPullRequestsForGetsentryRange.ts:63:      repo: isBumpCommit ? SENTRY_REPO : GETSENTRY_REPO,
src/api/github/getSentryPullRequestsForGetsentryRange.ts:72:      repo: GETSENTRY_REPO,
src/api/github/getSentryPullRequestsForGetsentryRange.ts:94:      repo: SENTRY_REPO,
src/api/github/isGetsentryRequiredCheck/index.ts:15:              if (payload.repository?.full_name !== `${OWNER}/${GETSENTRY_REPO}`) {
src/brain/gocdSlackFeeds/deployFeed.ts:135:                       if (repo !== GETSENTRY_REPO) {
src/brain/gocdSlackFeeds/deployFeed.ts:152:                       SENTRY_REPO,
src/brain/notifyOnGoCDStageEvent/index.ts:18:                     GETSENTRY_REPO,
src/brain/notifyOnGoCDStageEvent/index.ts:190:                    relevantCommit.sha === sha ? GETSENTRY_REPO : SENTRY_REPO;
src/brain/notifyOnGoCDStageEvent/index.ts:216:                    repo: GETSENTRY_REPO,
src/brain/notifyOnGoCDStageEvent/index.ts:224:                    repo: GETSENTRY_REPO,
src/brain/notifyOnGoCDStageEvent/index.ts:22:                     SENTRY_REPO,
src/brain/notifyOnGoCDStageEvent/index.ts:236:                    if (url.indexOf(`${OWNER}/${GETSENTRY_REPO}`) != -1) {
src/brain/pleaseDeployNotifier/actionViewUndeployedCommits.ts:5:  GETSENTRY_REPO,
src/brain/pleaseDeployNotifier/actionViewUndeployedCommits.ts:82: repo: GETSENTRY_REPO,
src/brain/pleaseDeployNotifier/index.ts:133:                      const commitLink = `https://github.com/${OWNER}/${GETSENTRY_REPO}/commits/${commit}`;
src/brain/pleaseDeployNotifier/index.ts:139:                      relevantCommit.sha === checkRun.head_sha ? GETSENTRY_REPO : SENTRY_REPO;
src/brain/pleaseDeployNotifier/index.ts:13:                       GETSENTRY_REPO,
src/brain/pleaseDeployNotifier/index.ts:17:                       SENTRY_REPO,
src/brain/requiredChecks/getAnnotations.ts:96:                    repo: GETSENTRY_REPO,
src/brain/requiredChecks/getTextParts.ts:10:                      const commitLink = `https://github.com/${OWNER}/${GETSENTRY_REPO}/commits/${checkRun.head_sha}`;
src/brain/requiredChecks/getTextParts.ts:15:                      `${GETSENTRY_REPO}@master`,
src/brain/typescript/getProgress.ts:11:                           repo = SENTRY_REPO,
src/brain/typescript/index.ts:40:                                 repo: GETSENTRY_REPO,
src/brain/typescript/index.ts:45:                                 repo: GETSENTRY_REPO,
src/utils/db/getFailureMessages.ts:52:                            repo: GETSENTRY_REPO,
src/webhooks/pubsub/index.ts:12:                                  const DEFAULT_REPOS = [SENTRY_REPO];

test def
--------
test/jest.setup.ts:13:                                            GETSENTRY_REPO: 'getsentry',
test/jest.setup.ts:14:                                            SENTRY_REPO: 'sentry',
chadwhitacre commented 1 year ago

I guess what we need is to adopt a strategy for ignoring codepaths that aren't relevant in codecov. Is it possible that we can avoid those by not subscribing CoveCod to certain GitHub events? Or are there events that we subscribe to for both CI/CD and engagement tracking purposes? I guess that would be the next thing to audit.

chadwhitacre commented 1 year ago

Looking good! 👍

General
=======
src/api/github/index.ts:16:                       githubEvents.onError(defaultErrorHandler);
src/brain/githubMetrics/index.ts:25:              githubEvents.onAny(ossHandler);

Engagement Tracking
===================
src/brain/issueLabelHandler/index.ts:14:          githubEvents.on('issues.opened', markUntriaged);
src/brain/issueLabelHandler/index.ts:18:          githubEvents.on('issues.opened', markUnrouted);
src/brain/issueLabelHandler/index.ts:16:          githubEvents.on('issues.labeled', markTriaged);
src/brain/issueLabelHandler/index.ts:20:          githubEvents.on('issues.labeled', markRouted);
src/brain/issueLabelHandler/index.ts:27:          githubEvents.on('issues.labeled', ensureOneWaitingForLabel);
src/brain/issueNotifier/index.ts:248:             githubEvents.on('issues.labeled', wrapHandler('issueNotifier', githubLabelHandler));
src/brain/issueLabelHandler/index.ts:25:          githubEvents.on('issue_comment.created', updateCommunityFollowups);
src/brain/projectsHandler/index.ts:10:            githubEvents.on('projects_v2_item.edited', syncLabelsWithProjectField);

test
----
src/brain/issueLabelHandler/index.test.ts:37:     githubEvents.onError(errors);
src/brain/issueLabelHandler/index.test.ts:69:     githubEvents.onError(defaultErrorHandler);
src/brain/projectsHandler/index.test.ts:26:       githubEvents.onError(errors);
src/brain/projectsHandler/index.test.ts:36:       githubEvents.onError(defaultErrorHandler);

CI/CD
=====
src/brain/githubMetrics/index.ts:18:              githubEvents.on('check_run', sentryHandler);
src/brain/pleaseDeployNotifier/index.ts:209:      githubEvents.on('check_run', handler);
src/brain/requiredChecks/index.ts:82:             githubEvents.on('check_run', handler);
chadwhitacre commented 1 year ago

I guess as long as we don't subscribe CoveCod to check_run we should be fine. 👍

chadwhitacre commented 1 year ago

And honestly it's really only the getClient calls downstream of the event handlers we care about that need to be made to work with dynamic owner vs. static global OWNER. That said, it'll probably be more hygienic to standardize on dynamic across the codebase.

chadwhitacre commented 1 year ago

I guess the risk is that we receive an event from the codecov org and try to take an action in the getsentry (hard-coded OWNER) org. It's possible we could end up with some garbage that way.

chadwhitacre commented 1 year ago

OWNER in 9bf0d35f914b8e7c700bcf5dc1fce9b2f0080ca7 (#505):

def
---
src/config/index.ts:11:                                             export const OWNER = process.env.OWNER || 'getsentry';
src/config/index.ts:179:                                             * getClient calls to use a dynamic owner/org instead of OWNER as defined above.

use
---
src/api/github/getChangedStack.ts:17:                               const octokit = await getClient(ClientType.App, OWNER);
src/api/github/getChangedStack.ts:20:                               owner: OWNER,
src/api/github/getChangedStack.ts:40:                               Commit: `https://github.com/${OWNER}/${repo}/commit/${ref}`,

src/api/github/getRelevantCommit.ts:17:                             const octokit = await getClient(ClientType.App, OWNER);
src/api/github/getRelevantCommit.ts:21:                             owner: OWNER,
src/api/github/getRelevantCommit.ts:43:                             owner: OWNER,

src/api/github/isGetsentryRequiredCheck/index.ts:15:                if (payload.repository?.full_name !== `${OWNER}/${GETSENTRY_REPO}`) {

src/brain/gocdSlackFeeds/deployFeed.ts:76:                          const octokit = await getClient(ClientType.App, OWNER);

src/brain/notifyOnGoCDStageEvent/index.ts:215:                      owner: OWNER,
src/brain/notifyOnGoCDStageEvent/index.ts:21:                       OWNER,
src/brain/notifyOnGoCDStageEvent/index.ts:223:                      owner: OWNER,
src/brain/notifyOnGoCDStageEvent/index.ts:236:                      if (url.indexOf(`${OWNER}/${GETSENTRY_REPO}`) != -1) {
src/brain/notifyOnGoCDStageEvent/index.ts:276:                      const octokit = await getClient(ClientType.App, OWNER);

src/brain/pleaseDeployNotifier/actionViewUndeployedCommits.ts:71:   const octokit = await getClient(ClientType.App, OWNER);
src/brain/pleaseDeployNotifier/actionViewUndeployedCommits.ts:81:   owner: OWNER,
src/brain/pleaseDeployNotifier/actionViewUndeployedCommits.ts:8:    OWNER,
src/brain/pleaseDeployNotifier/index.ts:133:                        const commitLink = `https://github.com/${OWNER}/${GETSENTRY_REPO}/commits/${commit}`;
src/brain/pleaseDeployNotifier/index.ts:16:                         OWNER,

src/brain/requiredChecks/getAnnotations.ts:85:                      const octokit = await getClient(ClientType.App, OWNER);
src/brain/requiredChecks/getAnnotations.ts:95:                      owner: OWNER,
src/brain/requiredChecks/getTextParts.ts:10:                        const commitLink = `https://github.com/${OWNER}/${GETSENTRY_REPO}/commits/${checkRun.head_sha}`;

src/brain/typescript/getProgress.ts:21:                             const octokit = await getClient(ClientType.App, OWNER);
src/brain/typescript/getProgress.ts:29:                             owner: OWNER,
src/brain/typescript/getProgress.ts:36:                             owner: OWNER,
src/brain/typescript/getProgress.ts:60:                             owner: OWNER,

src/utils/db/getFailureMessages.ts:43:                              const octokit = await getClient(ClientType.App, OWNER);
src/utils/db/getFailureMessages.ts:51:                              owner: OWNER,

src/webhooks/pubsub/index.ts:53:                                    const octokit = await getClient(ClientType.App, OWNER);
src/webhooks/pubsub/slackNotifications.ts:146:                      owner: OWNER,
src/webhooks/pubsub/slackNotifications.ts:427:                      owner: OWNER,
src/webhooks/pubsub/slackNotifications.ts:8:                        OWNER,
src/webhooks/pubsub/stalebot.ts:26:                                 owner: OWNER,
src/webhooks/pubsub/stalebot.ts:33:                                 owner: OWNER,
src/webhooks/pubsub/stalebot.ts:44:                                 owner: OWNER,
src/webhooks/pubsub/stalebot.ts:50:                                 owner: OWNER,
src/webhooks/pubsub/stalebot.ts:79:                                 owner: OWNER,

test def
--------
test/jest.setup.ts:12:                                              OWNER: 'getsentry',

test use
--------
test/payloads/github/issue_comment.ts:62:                           author_association: 'OWNER',
test/payloads/github/pull_request.ts:408:                           author_association: 'OWNER',
test/payloads/github/issues.ts:62:                                  author_association: 'OWNER',
src/brain/gocdSlackFeeds/deployFeed.test.ts:519:                    const octokit = await getClient(ClientType.App, OWNER);
src/brain/gocdSlackFeeds/deployFeed.test.ts:650:                    const octokit = await getClient(ClientType.App, OWNER);
chadwhitacre commented 1 year ago

Need to tackle these ones:

src/webhooks/pubsub/index.ts:53:                                    const octokit = await getClient(ClientType.App, OWNER);
src/webhooks/pubsub/slackNotifications.ts:146:                      owner: OWNER,
src/webhooks/pubsub/slackNotifications.ts:427:                      owner: OWNER,
src/webhooks/pubsub/slackNotifications.ts:8:                        OWNER,
src/webhooks/pubsub/stalebot.ts:26:                                 owner: OWNER,
src/webhooks/pubsub/stalebot.ts:33:                                 owner: OWNER,
src/webhooks/pubsub/stalebot.ts:44:                                 owner: OWNER,
src/webhooks/pubsub/stalebot.ts:50:                                 owner: OWNER,
src/webhooks/pubsub/stalebot.ts:79:                                 owner: OWNER,
chadwhitacre commented 1 year ago

Yeah okay upgrading pubsub is going to be the main piece of work before we can start testing out sending CoveCod events to eng-pipes.

chadwhitacre commented 1 year ago

And it won't impact Codecov to install CoveCod without updating pubsub, that should just continue to poke getsentry org without impacting codecov. I think that means I'm ready to install CoveCod on codecov and test it out!

chadwhitacre commented 1 year ago

I put up a slew of incremental PRs.

I need to understand pubsub better, here's the jobs in GCP. Since repos are coming in the payload I guess we should look at sending org in the payload as well, that means we have to set up one pubsub in GCP for each org vs. wiring org knowledge into eng-pipes. Tradeoffs. Wiring into org I guess would block on multi-org config changes.

chadwhitacre commented 1 year ago

Where is the payload with repos? 🤔

Screenshot 2023-07-10 at 10 42 44 AM
chadwhitacre commented 1 year ago

Looking into eng-pipes-cron topic ...

Screenshot 2023-07-10 at 10 44 01 AM
chadwhitacre commented 1 year ago

P.S. It shouldn't matter that we have logic for Status labels in eng-pipes, since those labels won't exist in codecov/feedback and we won't be sending any events with those.

We should set up a GTM team in the codecov org.

https://github.com/getsentry/eng-pipes/blob/6c1342fda3adc36bdd72bca357b8b157ef5feb62/src/utils/getOssUserType.ts#L63-L81

chadwhitacre commented 1 year ago

Any additional bots we want to exclude?

https://github.com/getsentry/eng-pipes/blob/6c1342fda3adc36bdd72bca357b8b157ef5feb62/src/utils/isFromABot.ts#L1-L7

chadwhitacre commented 1 year ago

Alright I still don't know how pubsub works. Where do we define the payload?

chadwhitacre commented 1 year ago

Also how are Slack notifications going to work? What else do we need to configure for that?

chadwhitacre commented 1 year ago

Where do we define the payload?

https://github.com/getsentry/ops/blob/master/terraform/super-big-data/super-big-consumers/schedule.tf

(h/t)

hubertdeng123 commented 1 year ago

Since repos are coming in the payload I guess we should look at sending org in the payload as well, that means we have to set up one pubsub in GCP for each org

Agreed here.

chadwhitacre commented 1 year ago

we should look at sending org in the payload as well, that means we have to set up one pubsub in GCP for each org

Alternately, we could add org info to the list of repos, maybe something like:

  pubsub_target {
    topic_name = google_pubsub_topic.eng_pipes_cron.id
    data       = base64encode(jsonencode({ "name" = "stale-triage-notifier", "repos" = ["getsentry/sentry-docs", "getsentry/sentry", "codecov/feedback"] }))
  }

Tbh though I think we probably want these definitions on the eng-pipes side, since that's the repo we spend more of our time in. I'm going to think about a way that might make sense. 🤔

chadwhitacre commented 1 year ago

Decided on a call w/ @hubertdeng123 to move repo config out of the ops repo into eng-pipes. This gives us more control to modify that config in the future.

chadwhitacre commented 1 year ago

I think I found it! Pretty sure this is where we define prod secrets:

https://github.com/getsentry/eng-pipes/settings/environments/164955041/edit

chadwhitacre commented 1 year ago

Where are "Waiting for: Support" notifications going?

There shouldn't be any coming from the bot since the feedback repo won't have routing. If someone manually adds the label it'll show up in the #discuss-support-open-source channel with a somewhat confusing message:

https://github.com/getsentry/eng-pipes/blob/6b6a0ecf7ea1c0f834fdd74a530db62c41174c02/src/brain/issueNotifier/index.ts#L38-L43

This isn't the end of the world since it requires manually sending to support. I'm pretty sure Support and Product Owner notifications should get folded together in Q3 under https://github.com/getsentry/team-ospo/issues/157.

Teach notify-for-triage about org? special office for them? or use sfo/yyz? Try to use sfo/yyz.

We're in the same boat here as with SDK repos where we don't provide Slack notifications currently. Let's solve this under https://github.com/getsentry/team-ospo/issues/157 as well.

chadwhitacre commented 1 year ago

If I'm reading this right we essentially skip notifications for repos without routing, which is what codecov/feedback is.

https://github.com/getsentry/eng-pipes/blob/6b6a0ecf7ea1c0f834fdd74a530db62c41174c02/src/webhooks/pubsub/slackNotifications.ts#L421-L423

I think we're fine.

chadwhitacre commented 1 year ago

If I'm wrong I expect an error to show up in Sentry. I'm going to set SENTRY_DSN locally and see if anything shows up.

chadwhitacre commented 1 year ago

I see this in the console:

    error: select "offices" from "label_to_channel" where "label_name" = $1 - relation "label_to_channel" does not exist

but nothing in Sentry.

chadwhitacre commented 1 year ago

Hrm, possible this is broken in prod, too. 🤔

$ yarn migrate migrate:latest                                                                                      
Requiring external module ts-node/register
Working directory changed to ~/workbench/getsentry/eng-pipes/src
ENOENT: no such file or directory, open 'github-orgs.local.yml'
Error: ENOENT: no such file or directory, open 'github-orgs.local.yml'
    at Object.openSync (fs.js:476:3)
    at NodeFS.openSync (/Users/chadwhitacre/workbench/getsentry/eng-pipes/.pnp.cjs:16719:24)
    at makeCallSync.subPath.subPath (/Users/chadwhitacre/workbench/getsentry/eng-pipes/.pnp.cjs:19405:26)
    at ZipOpenFS.makeCallSync (/Users/chadwhitacre/workbench/getsentry/eng-pipes/.pnp.cjs:20161:26)
    at ZipOpenFS.openSync (/Users/chadwhitacre/workbench/getsentry/eng-pipes/.pnp.cjs:19404:17)
    at VirtualFS.openSync (/Users/chadwhitacre/workbench/getsentry/eng-pipes/.pnp.cjs:17157:24)
    at PosixFS.openSync (/Users/chadwhitacre/workbench/getsentry/eng-pipes/.pnp.cjs:17157:24)
    at URLFS.openSync (/Users/chadwhitacre/workbench/getsentry/eng-pipes/.pnp.cjs:17157:24)
    at Object.readFileSync (fs.js:377:35)
    at NodeFS.readFileSync (/Users/chadwhitacre/workbench/getsentry/eng-pipes/.pnp.cjs:17065:24)
$ ag 'Working directory changed'
$