Cloud 2020-12-02 Tracking issue

[tsenart]

Availability

If you have planned unavailability this iteration (e.g., vacation), you can note that here.

@asdine 12/01 - 12/03 (UAE National Holiday) @ryanslade 12/09 - 12/11 PTO

Tracked issues

@artemruts: 2.50d

• [ ] GraphQL ExternalLink should expose kind: ExternalServiceKind (enum) instead of serviceType: String (#14979) 2.00d

Completed: 0.50d

• [x] (🏁 14 days ago) Remove Bitbucket cloud from list of code host when adding user added repos (~#16315~) 0.50d
• [x] (🏁 6 days ago) Add resend email verification link to emails setting page (~#16464~) :shipit:

@arussellsaw: 1.00d

Completed: 1.00d

• [x] (🏁 13 days ago) Set read and write timeouts on gitserver http server (~#16396~) :shipit:
• [x] (🏁 13 days ago) Instrument when gitserver janitor is running (~#16397~) :shipit:
• [x] (🏁 12 days ago) Allow resending verification emails (~#16310~) 1.00d
• [x] (🏁 6 days ago) Fetch affiliated repos from code hosts for repo selection screen. (~#16501~)

@asdine: 4.00d

Completed: 4.00d

• [x] (🏁 15 days ago) Support filtering repos by owner in API (~#15488~) 1.00d
• [x] (🏁 6 days ago) Add a dedicated method to fetch only a subset of repository columns (~#16219~; PRs: ~#16437~, ~#16590~)
• [x] (🏁 6 days ago) Remove Repo store OnlyRepoIDs option (~#16613~) :shipit:
• [x] (🏁 6 days ago) Change global repo type structure (~#16615~) :shipit:
• [x] (🏁 6 days ago) Delete repo-updater Repo types (~#16618~) :shipit:
• [x] (🏁 4 days ago) repo-updater: Find root cause of memory spikes (~#16679~; PRs: ~#16688~)
• [x] (🏁 2 days ago) repo store: Replace all calls to repo-updater's Insert with internal/db/repos#Create (~#14711~; PRs: ~#15530~, ~#16213~) 1.00d 🧶
• [x] (🏁 2 days ago) repo store: Replace all calls to repo-updater's Delete with internal/db/repos#Delete (~#14712~; PRs: ~#15530~, ~#16215~) 1.00d 🧶
• [x] (🏁 2 days ago) monitoring: Add container restart alerts for repo-updater (~#16653~; PRs: ~#16664~)
• [x] (🏁 2 days ago) repo store: Replace all calls to repo-updater's ListRepos with internal/db/repos#List (~#14710~; PRs: ~#15530~, ~#16719~) 1.00d 🧶
• [x] (🏁 2 days ago) repo store: Stop using the global dbconn (~#15092~) 🧶
• [x] (🏁 1 day ago) Decouple repo-updater store from the rest of the codebase (~#16753~) :shipit:
• [x] (🏁 1 day ago) Remove unused calls to repo updater store (~#16759~) :shipit:
• [x] (🏁 1 day ago) Ensure repo is always hydrated when needed (~#16713~) :shipit:

@flying-robot: 3.00d

• [ ] Add ability to schedule janitor jobs for off-hours (#10274) 👩

Completed: 3.00d

• [x] (🏁 7 days ago) gitserver: periodically run git gc (~#16241~; PRs: ~#16359~) 3.00d

@quinnkeast

• [ ] Surface sync errors to users (#16321)

@ryanslade: 2.50d

Completed: 2.50d

• [x] (🏁 13 days ago) Temporary file size error when upserting sources (~#16318~; PRs: ~#16352~, ~#16372~) 🐛
• [x] (🏁 13 days ago) repo-updater: Lazily performs deletes in UpsertRepos (~#16404~) :shipit:
• [x] (🏁 12 days ago) User added private repos are visible to all (~#16405~; PRs: ~#16435~, ~#16444~)
• [x] (🏁 9 days ago) Instrument syncer errors (~#16319~; PRs: ~#16505~) 0.50d
• [x] (🏁 9 days ago) Don’t delete repos when GitHub tokens are exhausted (~#16393~; PRs: ~#16453~) 1.00d
• [x] (🏁 9 days ago) errcode: Use anonymous interfaces (~#16522~) :shipit:
• [x] (🏁 9 days ago) Revert "errcode: Use anonymous interfaces (#16522)" (~#16542~) :shipit:
• [x] (🏁 7 days ago) Handle revoked tokens during sync (~#15739~; PRs: ~#16516~) 1.00d
• [x] (🏁 4 days ago) repo-updater: Find root cause of memory spikes (~#16679~; PRs: ~#16688~)
• [x] (🏁 1 day ago) repo-updater: Allow custom git update interval (~#16765~) :shipit:
• [x] (🏁 1 day ago) Rethink monitoring of remaining code host rate limits (~#16455~; PRs: ~#16716~, ~#16750~)

@tsenart

• [ ] Extension registry GraphQL API is slow (#10554)
• [ ] Panic during db migration (#16159)
• [ ] Timeout when accessing subdirectory with only one file ( Request to https://sourcegraph.com/.api/graphql?TreeEntries failed with 504) (#10538)
• [ ] Add ability to schedule janitor jobs for off-hours (#10274) 👩
• [ ] [Cloud] [CRITICAL] gitserver: 2s+ echo command duration test fires regularly (#15392)
• [ ] monitoring: Graph each of the different API rate limits separately (rest, graphql, search) (#16391)
• [ ] monitoring: Change gitserver alerts and dashboard to monitor golden signals (#16580; PRs: ~#16591~)

Completed

• [x] (🏁 13 days ago) vcs/git: Remove commandRetryer (~#16360~) :shipit:
• [x] (🏁 7 days ago) monitoring: Use lines for thresholds instead fill areas (~#16547~) :shipit:
• [x] (🏁 7 days ago) go.mod: Update gonduit dependency (~#16581~) :shipit:
• [x] (🏁 7 days ago) monitoring: Disable alerting for echo_command_duration (~#16591~) :shipit:
• [x] (🏁 6 days ago) grafana: Add internal Nginx dashboards (~#16622~) :shipit:
• [x] (🏁 5 days ago) Continuously capture memory profiles in repo-updater (~#16655~; PRs: ~#16656~)
• [x] (🏁 5 days ago) gitserver: Enable continuous profiling (~#16665~) :shipit:
• [x] (🏁 4 days ago) monitoring: Critical alert for 90% memory utilisation on repo-updater (~#16652~; PRs: ~#16664~)
• [x] (🏁 4 days ago) repo-updater: Skip listing conflicting repos when sourcing fails (~#16688~) :shipit:
• [x] (🏁 4 days ago) frontend: Enable continuous profiling (~#16693~) :shipit:
• [x] (🏁 4 days ago) pubsub: Use explicit credentials file (~#16695~) :shipit:
• [x] (🏁 2 days ago) Remove gitserver.Repo type (~#16707~) :shipit:
• [x] (🏁 1 day ago) Some commits not syncing with Gerrit-hosted git repository (~#16387~)
• [x] (🏁 1 day ago) monitoring: Disable alerting for syntect_server (~#16773~) :shipit:

@unknwon: 4.50d

• [ ] sourcegraph/customer (#148) 2.00d 👩

Completed: 2.50d

• [x] (🏁 14 days ago) graphqlbackend: add new scalar BigInt to fix int overflow (~#16345~) :shipit:
• [x] (🏁 13 days ago) gqltest: add a test to cover repository with space in its name (~#15905~; PRs: ~#16424~) 0.50d
• [x] (🏁 4 days ago) Write an RFC for First-class Perforce integration (~#16508~) 2.00d 👩

Legend

• 👩 Customer issue
• 🐛 Bug
• 🧶 Technical debt
• 🎩 Quality of life
• 🛠️ Roadmap
• 🕵️ Spike
• 🔒 Security issue
• :shipit: Pull Request

[github-actions[bot]]

Heads up @tsenart - the "team/cloud" label was applied to this issue.

[ryanslade]

This week

• Worked on some of the issues that came out of an ops incident: https://docs.google.com/document/d/1dtrOHs5STJYKvyjigL1kMm6u-W0mlyRSyVxPfKIOfEw/edit#heading=h.r7tahw4ry14v
• Ironed out some issues around adding private code on .com as we test with dummy repos there
• Looked into reducing how often we trigger some slow queries, the deployment with the change appeared to reduce CPU usage in our SQL cluster quite significantly (see the drop at about 6:30):

Next week

• Keep working on tickets that came out of ops incident as well as other bugs
• Work on handling revoked tokens during sync

[flying-robot]

This week

• Draft implementation of periodic git-gc repo cleanup
• Investigated an issue where some gerrit commits were absent from gitserver

Next week

• Finish periodic git-gc cleanup
• Make repo update schedule configurable
• Discuss/plan janitor scheduling in general

[unknwon]

Last week

• Added multiple test to our backend integration test suite
• Investigated on sourcegraph/customer (#148)

This week

• Finish investigating sourcegraph/customer (#148)
• Start working on overhaul aggregated errors sent to Sentry (#16109)

[unknwon]

Last week

• Finished Perforce technical PoC with permissions (#148)
• Drafted RFC 277: First-class Perforce integration and approved, created a project for it.
• Worked on a hotfix for our user code host connections validation (#16614)

This week

• Wrapping up on overhaul aggregated errors sent to Sentry (#16109)
• Expecting more polish work for the next cycle since it would be short for many of us.

[asdine]

Last week

• Merged repository types and helpers and worked on decoupling repo-updater from the rest of the codebase

This week

• Finish the decoupling and work on improving the store layer

[flying-robot]

Last Week

• Implemented periodic git-gc repo cleanup

This Week

• Discuss/plan janitor scheduling in general (e.g. which things must be routinely run and which can be set to off-hours, etc.)

[ryanslade]

Reopening this issue since the cycle only ends tomorrow and it looks like the tracking tool doesn't update it when it's closed.