🤑 Run Fund Fest 2023

chadwhitacre commented 1 year ago

← 2022 | Notion🔒 | Sheet🔒・Sheet 2🔒 | 2024 →

Greetings!

Welcome to Sentry's annual Open Source Fund Fest for 2023! This program takes a lot of work to run and this issue is the central source of truth for coordinating all of the moving parts. This year our budget is $500,000, which is almost double last year. There are two key components to this year's edition:

GitHub Sponsors—We are continuing our partnership with GitHub, and this year our goal is to become the first company to approach 100% coverage of all of our dependencies on Sponsors, across three orgs (gesentry, codecov, and syntaxfm). We are dedicating 10% of our budget to this, so these will be small amounts but lots of them and will show as coming directly from us. We expect a number of these to be the first sponsorship that people ever receive.
Thanks.dev—We were a launch partner for this new platform last year, and our pilot project went so well that we have contracted with Thanks.dev to manage the bulk of our budget this year. The size and scope of our program is quite significant, and the product Thanks.dev is building and the level of expertise they are developing in this field make them an excellent partner for us. We're inventing the future together!

Thanks.dev offers payouts through Stripe (0% additional fee) and Open Collective (10%) and is adding payouts through GitHub Sponsors for us (targeting the 3% fee tier). You will need to sign up on Thanks.dev if you want to receive more than the small amounts we're sending to everyone directly through GitHub Sponsors.

We're aiming to publish our wrap-up blog post on October 24. In the mean time, feel free to comment on this issue or reach out on 𝕏 with questions. Thank you to all of the maintainers who build the software Sentry depends on. 🙏

FOSS Funders

While it is fun and exciting to run arguably the most comprehensive and robust program in the industry for really truly funding Open Source, the only way we're going to solve Open Source sustainability once and for all is for every company to step up and participate. Sentry is part of the FOSS Funders working group to drive this change. If your company has an Open Source funding story to tell, please join us!

To Do

[x] July 15 - Confirm budget
[x] July 15 - Update AP calendar🔒, determine invoice due dates for below.
[x] August 1 - Kick off internal survey, run through August 18 (close before hackweek)
[x] Receive and pay invoices
- [x] August 11 - zeroth batch due, missed it this year, hit next year?
- [x] August 25 - first batch due, including first from GitHub
- [x] September 8 - second batch due, including final from GitHub
- [x] September 22 - third batch due - aim for all by here - now just one, from Thanks.dev! 😁
- [x] October 6 - last chance, ~try hard not to have to use it!~ - second one from TD, we decided to split into two since they're large, make sure they make it through the banking system okay
[x] Local meetups that Sentry employees frequent? - handled in dev survey
[x] Review "What's new with GitHub Sponsors"—bulk sponsorships!
[x] platforms?
- [x] GHS
- [x] ~OC~
- [x] ~stackaid~
- [x] thanks.dev!
- [x] ~polar.sh~—not ready, maybe next year
[x] foundations
- [x] pay
  - [x] Python
  - [x] JavaScript
  - [x] Rust
  - [x] PostgreSQL
  - [x] Apache
  - [x] PHP
  - [x] Ruby
  - [x] OSI
  - [x] Outreachy
- [x] logo - https://sentry.io/branding/
  - [x] Python
  - [x] JavaScript
  - [x] Rust
  - [x] ~PostgreSQL~
  - [x] Apache
  - [x] PHP
  - [x] ~Ruby~
  - [x] OSI
  - [x] Outreachy
[x] GHS
- [x] sort out sponsor linking (GH Enterprise feature?)
- [x] create sponsorships for getsentry
- [x] get balances transferred to codecov and syntaxfm
- [x] create sponsorships for codecov
- [x] create sponsorships for syntaxfm
- [x] double-check final totals
[x] TD
- [x] Don't forget (these are things Sentry employees or others have specifically mentioned to me over the past year) ...
  - [x] codecov and syntaxfm orgs
  - [x] last year's recipients - e.g.
  - [x] Fastlane/MNF
  - [x] Moka
  - [x] rrweb - vital for Replay, but we may fund them through contracting? Find out details and record for posterity
  - [x] https://github.com/sponsors/webknjaz (X)
  - [x] https://opencollective.com/json-schema (X)
  - [x] ICCF Holland? (because) - not sure they're recruitable, made a personal donation
- [x] review draft weighting
[x] publish blog post🔒
- [x] update hero🔒 - https://brand.getsentry.com/share/M7C2Nwq1Jypfe1tcT7QC

Press

GitHub Sponsorships

"A'ight, beat that. 😤😁💃" (chadwhitacre_)
"I'm sorry, I don't make the rules. 🐭" (chadwhitacre_)
"Sentry is giving $500k+ to open source it relies on this year" (zeeg)
"it’s insane that people are anything other than thankful" (jarredsumner)
"A company donates $500K/year in direct funding to OSS" (gergelyorosz)
"YOU ARE A PIECE OF SHIT FOR NOT DONATING MORE OF YOUR VC FUNDING" (ThePrimeagen)

Announcement

"We Just Gave $500,000 to Open Source Maintainers"
X
HN - 112 points and 36 comments despite being brutally modded at ~20 points - and then modded a second time! rankings
/r/opensource/
"It's the 2nd year in a row we've hit the front page only to have HN mods remove it." (bentlegen)
"The answer I got last year (and kudos dang for the tone)" (chadwhitacre_)
"Tarsnap has given 2^18 dollars to open source" (cperciva) - HN (not modded! 😁 390 / 180 / rankings) - X

chadwhitacre commented 1 year ago

Alright, slogging through the thick fog of confusion, hopefully emerging?

I've decided not to use StackAid this year, not seeing enough momentum and I need to focus my efforts. 🐭

I really like what I see at Thanks.dev. It's basically a productization of the spreadsheet I've been using and the team has been super-responsive, basically building it to spec for me. ❤️ 🙏

GitHub Sponsors is making gradual progress but is still quite cumbersome for a program at our scale, no-where near as useful as Thanks.dev.

OpenCollective is approximately as useful as GHS and the fees are higher, 10% vs. 3%.

Two years ago I wrote deps and did allocations manually. Last year I reran deps but also put a little through TD and SA as pilots. I didn't try to reconcile recipients and amounts between them, I just let the TD/SA money be free bonus extra cash dollar dough bucks. This year I would like to flip it:

~12k — Achieve ≥ 99% coverage on GHS for getsentry, codecov, and syntaxfm, but with a limited dollar amount, basically just buying logo placement.
~120k — Do some direct payments to foundations still, though a couple of these can be run through GHS or OC.
~368k — Run the rest through Thanks.dev so we get nice weighting.

Needs from TD:

Mix in sources besides package manifests (survey results, platforms, last year's recipients).
Passthrough to OC/GHS where the recipient is not on TD (I guess with extra 3% or 10% haircut).
Weighting by repo topic ([tag-production]())
Think about slew?
Concierge? 😅

Needs for TD:

Security approval to install Thanks.dev on private repos (nice to have).
Install Thanks.dev on all public repos.

For GitHub:

Write a script to use the GraphQL API to create sponsorships.

chadwhitacre commented 1 year ago

I have $50k locked up in GHS right now, I should be able to use for PSF and DSF, not sure I'll be able to transfer to other orgs though.

chadwhitacre commented 1 year ago

Engaging with Thanks.dev. Sent in email:

Thanks.dev (TD) administers Sentry's Open Source funding program for the year.

$25k fee

$475k to be allocated and disbursed via one-time or monthly recurring payouts

$50k is already located in GitHub Sponsors (GHS), remainder ($425k) will be transferred to TD, OC, GHS, or direct to project as necessary

Recipients determined via automated dependency analysis in three GitHub orgs + additional lists (survey results, SDKs, prior year).

Allocations weighted according to Sentry's priorities (orgs, repos, ecosystems, projects, recipients)

Payout mechanisms include TD (0% additional fee), OC (10%), GitHub Sponsors (3%), and direct payments (0%) to a limited number of projects (~10).

All funds disbursed (for one-time payouts) or configured (for monthly payouts) by Oct 10.

Let me know if we need to align further on details. Do you have an order form that we can start with? If not [we] can provide. I am seeing terms on your site, not sure if we can start there or what, will let [legal] drive from here.

Internal procurement ticket: https://getsentry.atlassian.net/servicedesk/customer/portal/15/PSD-294

chadwhitacre commented 1 year ago

@nehzata Can we use this ticket to coordinate the rest of our work together this year?

I believe the contract is in your hands to docusign. Once that is inked we are ready to run!

The the thanks-dev app is installed on getsentry, codecov, and syntaxfm. However, the @getsentry-funding account is not a member of the latter two orgs. I have a request in to @jeffrey-sentry to help out with that. I also seem to be having some trouble getting the account through our new SCIM workflow on the getsentry org as well, so I think we may be fully blocked on this.

I had a call with GitHub about transfering funds between the three orgs, I added a sheet to the spreadsheet ("GHS") with details, tl;dr targeting $4/maintainer/month across all three orgs. Need to investigate minimums in our set of maintainers and decide on our approach (easiest is to drop any maintainer with a minimum > $4/mo, but we have some flex if needed).

I also added contact emails for the foundations on the "Summary" tab. Feel free to start reaching out to folks and let me know if any of the target amounts do not line up with levels/expectations for this year. For Outreachy they work in 6-month intervals so the 20k will need to be 2 x 10k I think.

nehzata commented 1 year ago

Thanks @chadwhitacre! Contract is signed now. Exciting!

On our side we've nearly finished the manual inclusions process to support survey + sdk deps. We'll get the GH sponsors script sorted next and @anehzat will be starting with the foundations.

chadwhitacre commented 1 year ago

Sounds good. I am traveling today and all next week but I have my laptop, we're so async with timezones in the best of times that I doubt you'll notice much anyway, but in particular I will watch for email threads with foundations that @anehzat starts and chime in as needed. Thanks, @nehzata @anehzat! Good luck! 😁 ☘️ 🙏

chadwhitacre commented 1 year ago

lol should've checked my email first ... love you guys! 😁

nehzata commented 1 year ago

Hi @jeffrey-sentry! Hope you're well.

Following @chadwhitacre's previous message re the app installation, we can confirm access to syntaxfm on our side but not getsentry & codecov.

Please let me know if there's anything we can do on our side to assist. 🙏

jeffrey-sentry commented 1 year ago

@nehzata I do not have access to the codecov org, but I just checked the thanks.dev app permission setting for getsentry and it does have Read access to all repos. Is there anything else we need to do on our side beyond that?

Thanks!

chadwhitacre commented 1 year ago

Here are the repos I see when I login to thanks.dev using @getsentry-funding. I guess we need to do the dance to see codecov and getsentry there.

chadwhitacre commented 1 year ago

All three (plus mine and ... Isaac's!?) are associated with my personal GitHub account. 🤔

chadwhitacre commented 1 year ago

Here's what I see when I try to configure the app using @getsentry-funding from the thanks.dev UI:

chadwhitacre commented 1 year ago

When I configure using my account I see that it is installed on all repos (for all three orgs).

chadwhitacre commented 1 year ago

@nehzata Was there a manual step last year on your side to get this wired up? I forget but have a vague recollection. Any thoughts on how to proceed?

nehzata commented 1 year ago

@chadwhitacre looks like for the Sentry org it expects us to manually select the repos individually. Last year we had a script running that would populate everything without requiring the app installation. That worked as all the repos we were interested in were public. If I understood correctly, this year there are private repos as well? If not shall we revert to the previous process?

nehzata commented 1 year ago

Just to clarify it seems we have the following options atm:

Use last year's process (public repos only)
Switch to gh/chadwhitacre profile
Manually configure the repos for Sentry (by clicking on Sentry > Configure > "Only select repositories" and adding the relevant repos)

nehzata commented 1 year ago

@nehzata I do not have access to the codecov org, but I just checked the thanks.dev app permission setting for getsentry and it does have Read access to all repos. Is there anything else we need to do on our side beyond that?

Thanks!

Thanks for checking it out @jeffrey-sentry!

nehzata commented 1 year ago

@chadwhitacre We submitted a bug report with GitHub regarding the issacs entry and got the following response.

nehzata commented 1 year ago

@chadwhitacre First version of GH mass sponsorship app is ready to test here! :)

chadwhitacre commented 1 year ago

We submitted a bug report with GitHub regarding the issacs entry and got the following response.

What org am I a collaborator in that results in isaacs showing up in the list? I don't see isaacs on the list of orgs on https://github.com/apps/thanks-dev/installations/select_target, e.g. Feel free to cc me on the support ticket and I can connect with GH support directly.

First version of GH mass sponsorship app is ready to test here! :)

Dope! Looks like it's designed to run continuously, yes? Can I run it once to set up recurring monthly donations?

for the Sentry org it expects us to manually select the repos individually

Is it some config specific to the getsentry org that requires this? How about codecov? I have the app set to "all repos" in both orgs (as well as syntaxfm) so I am wondering where specifically we are tripped up here.

nehzata commented 1 year ago

What org am I a collaborator in that results in isaacs showing up in the list? I don't see isaacs on the list of orgs on https://github.com/apps/thanks-dev/installations/select_target, e.g. Feel free to cc me on the support ticket and I can connect with GH support directly.

The original ticket is closed as it's from Nov last year. I did some further digging into this and looks like the following is happening:

You're a collaborator on github.com/isaacs/github;
isaacs has previously logged into TD and has therefore installed the app on his account;
GH sees your account as a collaborator of isaacs' RepositoryOwner` entity;
Therefore your installation token also gets access to all the repos in that entity that you have read access on via the GraphQL API (which are all his public repos);

We've come across many such disconnects between token scopes in the App installation API & GraphQL API unfortunately...

curl -s -H 'authorization: bearer <TOKEN>' https://api.github.com/repos/isaacs/github/collaborators | jq '. | map(.login)'
[
  "schacon",
  "wfarr",
  "clarkbw",
  "holman",
  "isaacs",
  "haacked",
  "aspiers",
  **"chadwhitacre"**,
  "tjfontaine",
  "TPS",
  "dreww",
  "broccolini",
  "joernhees",
  "jlord",
  "cirosantilli",
  "yoannchaudet",
  "michellemerrill"
]

First version of GH mass sponsorship app is ready to test here! :)

Dope! Looks like it's designed to run continuously, yes? Can I run it once to set up recurring monthly donations?

Yes it currently runs continuously to cater for network failures + GH access token rate limiting. Recurring donations may be possible but we don't know if it works yet as the corresponding monthly/yearly setting are not present in the API. We've asked in GH community forums. Should be a one or two line change.

Let's do a quick code review + test to confirm once you're back online please?

for the Sentry org it expects us to manually select the repos individually

Is it some config specific to the getsentry org that requires this? How about codecov? I have the app set to "all repos" in both orgs (as well as syntaxfm) so I am wondering where specifically we are tripped up here.

Not really sure. Let's go over it on a call when you're back to see if anything sticks out please? In the meantime we've started working based on the @chadwhitacre account that has correct access to unblock ourselves for now.

Finally, looks like @anehzat has the foundations mostly covered. We plan to have a first draft of distributions for your review next week.

chadwhitacre commented 1 year ago

You're a collaborator on github.com/isaacs/github

Thanks for digging, I've opened a new support ticket🔒 with GitHub to be removed as a collaborator from that repo. It is now archived, and I can't find in the UI where to remove myself.

We've asked in GH community forums.

Gotcha. I've asked my contact at GitHub Sponsors in email to see if we can get an answer here.

Let's do a quick code review + test to confirm once you're back online please? Not really sure. Let's go over it on a call

Sounds good, let's aim for the first part of next week, will drop to email to schedule. 👍

nehzata commented 1 year ago

Awesome! First invoice has been emailed btw.

Hope you're having a great time!

nehzata commented 12 months ago

@chadwhitacre as discussed, these are the only installations currently present for getsentry-funding user.

curl -s -H 'authorization: bearer <TOKEN>' https://api.github.com/user/installations | jq '.installations | map(.account.login)'
[
  "getsentry-funding",
  "syntaxfm"
]

chadwhitacre commented 12 months ago

invoice - cramer to approve, ensure it gets paid promptly
wkr-gh-sponsor
- modify to consume CSV from explore to ensure 100% deps coverage per GH view
- validate that it defaults to recurring monthly ("One of our eng noted if Sentry is already billed monthly or are Sponsors invoiced, then using the GraphQL API should create a recurring monthly sponsorship. The sponsorship doesn't need to be specified as monthly as it appears it'll default to monthly. ")
thanks-dev app install - what perms would getsentry-funding need in order to install for all repos and see them show up in thanks.dev?
- How do I give @getsentry-funding explicit permission to access the thanks-dev app installations in getsentry and codecov?

chadwhitacre commented 12 months ago

Invoice is approved and should get paid tomorrow. 👍

nehzata commented 12 months ago

utils (formerly wkr-gh-sponsor) is now updated. Doesn't need the csv file. It instead scrapes all the dependencies from GH API.

To run locally (donation disabled): . bin/activate-hermit GH_CLASSIC_ACCESS_TOKEN=<TOKEN> ./scripts/wkr-gh-sponsor --config example.config.json

chadwhitacre commented 12 months ago

these are the only installations currently present for getsentry-funding user.
curl -s -H 'authorization: bearer <TOKEN>' https://api.github.com/user/installations | jq '.installations | map(.account.login)'
[
  "getsentry-funding",
  "syntaxfm"
]

@jeffrey-sentry Any ideas on this one? The user/installations endpoint returns syntaxfm for the @getsentry-funding account, but in order to use that account (rather than, e.g., my own account) we need it to also return getsentry and codecov. Nothing obvious is jumping out at me when I compare:

Do both codecov and getsentry use SCIM? Something to do with that, maybe? 🤔

chadwhitacre commented 12 months ago

A clue! There is a "Member privileges" settings page, and I find a "Base permissions" knob at the top:

☝️ That's for syntaxfm. For getsentry and codecov this option is set to "No permission". 😱

Does this maybe interact with the user/installations API endpoint? 🤔 I don't want to change the setting permanently, but maybe we could toggle it off temporarily in syntaxfm to see if it impacts the endpoint?

chadwhitacre commented 12 months ago

@nehzata Let's do this on a call. I've uninstalled the thanks-dev app from syntaxfm, but it's still showing up in the settings page for @getsentry-funding. My hunch is that you can do something on your end to remove that connection in Thanks.dev, then I can re-add with the base permission turned off in syntaxfm to test my hypothesis. That'll probably be most efficient in realtime, so I've sent an invite for later today/tomorrow. In the mean time I've opened a support request🔒 with GitHub, since if this is true we'll need some workaround since we shouldn't set base perms in codecov and getsentry.

nehzata commented 12 months ago

@chadwhitacre meeting confirmed.

GH is still returning syntaxfm:

curl -s -H 'authorization: bearer <TOKEN>' https://api.github.com/user | jq .login

"getsentry-funding"

curl -s -H 'authorization: bearer <TOKEN>' https://api.github.com/user/installations | jq '.installations | map({id:.id, login:.account.login})'
[
  {
    "id": 32259457,
    "login": "getsentry-funding"
  },
  {
    "id": 42286928,
    "login": "syntaxfm"
  }
]

curl -s -H 'authorization: bearer <TOKEN>' https://api.github.com/user/installations/42286928/repositories | jq '.repositories | map(.name)'
[
  ".github",
  "brand",
  "giveaway",
  "hackweek-md-multiplayer-editor",
  "meta",
  "vscode-theme",
  "website"
]

chadwhitacre commented 12 months ago

We confirmed in real time that the hypothesis is correct: we need either "Owner" role on the user account or "Read" base perms for the org in order for the user/installations API to include a given org. We also confirmed that installing the app for only some repos vs. all repos does not make a difference. Assuming GitHub does not get back with a workaround on my support request, our options seem to be:

Use @chadwhitacre (already owner on all three orgs).
Grant @getsentry-funding the owner role.
Change the base permissions to Read in getsentry and codecov.

Thoughts on how to proceed @jeffrey-sentry @mdtro? Since the thanks-dev app only has read permissions, iirc using a separate low-privilege account was really a defense-in-depth, yes? I think option (1) may be the least disruptive/risky path forward, assuming no workaround. Can we live with this?

chadwhitacre commented 12 months ago

Just saw this come through! 💃

The payment for invoice 0120 was initiated on Sep 27, 2023. We notified Thanks Dev Pty Ltd of the payment initiation. The payment should reach the vendor’s bank account in 4-5 business days from the day of this email.

nehzata commented 12 months ago

Excellent!

auto-boost added to utils as well.

nehzata commented 11 months ago

Inclusions have been uploaded to both @chadwhitacre & @getsentry-funding accounts

chadwhitacre commented 11 months ago

Okay so in my court:

[x] get answer on @chadwhitacre account
[x] test out mass-gh-sponsor
[x] ~test out auto-boost~ letting @nehzata handle this for now

I've pinged our security team internally on the first one.

chadwhitacre commented 11 months ago

if we can differentiate activities from thanks.dev and activities that's performed by you in logs, then I think it's fine to use option 1

chadwhitacre commented 11 months ago

Independently, we're in the process of rolling our GH orgs up into a GH enterprise(!). The timing is fortuitous, since it interacts with sponsor accounts. tl;dr I'll have a little dancing to do in their UI before I am ready to configure our new sponsorships.

chadwhitacre commented 11 months ago

Finding this clue:

Similarly, if the request triggers a corresponding entry in the audit logs and security logs, the logs will list the user as the actor but will state that the "programmatic_access_type" is "GitHub App user-to-server token".

Looks like we can't filter on programmatic_access_type in the audit UI. I've kicked off an export to see if this key shows up in our existing audit log at all.

chadwhitacre commented 11 months ago

With help from my Security team, I was able to inspect the audit logs. The programmatic_access_type key is present as documented, and additionally there is a "name" field that seems to list the app in question. This gives us what we need to detect any unexpected behavior from the app.

chadwhitacre commented 11 months ago

@nehzata I'm sure you've already addressed this ... Could the Thanks.dev GH App use installation auth instead of user auth?

mdtro commented 11 months ago

@nehzata , 👋 I'm a part of the Sentry security team. @chadwhitacre Sorry for all the telephone and having you play mediator. 😅

I was reading the ToS to try and get some more detail about how thanks.dev works and is possibly architected. I fully understand you'll need to pull down our code to analyze it, but is it possible to get more clarification as to what is stored long-term on your systems? Questions below.

From the ToS:

The data we collect from your during the course of providing our Service will be stored according to industry standards. We perform routine backups of our systems and data, however we do not warrant these backups and you should not rely on them for your business continuity.

What data is stored? Everything from the repository or just package manifest files?
How is the data stored? ie. is the data stored on cloud storage and is the data encrypted?
Are backups encrypted and how long are they retained?
If the Thanks.dev app is ever deauthorized from the organization, is associated data deleted including backups?

nehzata commented 11 months ago

Hi @mdtro ! Thanks for looking into this. Response to questions are below. Please let me know if any other parts need clarification.

The data we collect from your during the course of providing our Service will be stored according to industry standards. We perform routine backups of our systems and data, however we do not warrant these backups and you should not rely on them for your business continuity.

What data is stored? Everything from the repository or just package manifest files?

We only read the manifest files listed here.

How is the data stored? ie. is the data stored on cloud storage and is the data encrypted?

It is stored in our DB backed by an encrypted EBS in a private EKS cluster. The EKS cluster is configured with role based permissions via IAM roles.

Are backups encrypted and how long are they retained?

Backups are encrypted on S3. The manifest files + dependency trees are not stored in backups though. We only backup the following DBs:

users db
settings db
accounting db (donations, balances etc)

Everything else will automatically get reanimated in the case of DR. Storage costs would be prohibitive otherwise.

We retain last 30 days via S3 lifecycle policies.

If the Thanks.dev app is ever deauthorized from the organization, is associated data deleted including backups?

We're not presently deleting any data automatically. In the event of a deauthorization we just detect access has been revoked and stop trying to read the manifest files + updating the dependency tree.

However, we do have the ability to manually handle CCPA and equivalent data deletion requests should there be a need.

nehzata commented 11 months ago

@nehzata I'm sure you've already addressed this ... Could the Thanks.dev GH App use installation auth instead of user auth?

@chadwhitacre We don't currently obtain an installation token. Let us look into it and get back to you please.

If I've understood correctly, this would be to differentiate thanks.dev activity from your own activity in the audit logs right?

nehzata commented 11 months ago

@chadwhitacre Can we jump on a quick call please to confirm which repos will be included in the dep tree? We'll then be able to do the final export for onboarding. Same time as usual if it's ok with you? 🙏

chadwhitacre commented 11 months ago

If I've understood correctly, this would be to differentiate thanks.dev activity from your own activity in the audit logs right?

Yes, and in general to decouple app activity from my user account.

chadwhitacre commented 11 months ago

Can we jump on a quick call please [...] Same time as usual if it's ok with you? 🙏

Of course, sent for your Thursday AM.

to confirm which repos will be included in the dep tree? We'll then be able to do the final export for onboarding.

My intention is to include all repos for all three orgs (weighted differently, of course). Is this enough to unblock?

chadwhitacre commented 11 months ago

Ftr looks like we found time to meet today/tonight.

Also, our orgs are now consolidated into an enterprise, ready to proceed with mass-gh-sponsor. 👍

mdtro commented 11 months ago

Hi @mdtro ! Thanks for looking into this. Response to questions are below. Please let me know if any other parts need clarification.

The data we collect from your during the course of providing our Service will be stored according to industry standards. We perform routine backups of our systems and data, however we do not warrant these backups and you should not rely on them for your business continuity.

What data is stored? Everything from the repository or just package manifest files?

We only read the manifest files listed here.

How is the data stored? ie. is the data stored on cloud storage and is the data encrypted?

It is stored in our DB backed by an encrypted EBS in a private EKS cluster. The EKS cluster is configured with role based permissions via IAM roles.

Are backups encrypted and how long are they retained?

Backups are encrypted on S3. The manifest files + dependency trees are not stored in backups though. We only backup the following DBs:

users db

settings db

accounting db (donations, balances etc)

Everything else will automatically get reanimated in the case of DR. Storage costs would be prohibitive otherwise.

We retain last 30 days via S3 lifecycle policies.

If the Thanks.dev app is ever deauthorized from the organization, is associated data deleted including backups?

We're not presently deleting any data automatically. In the event of a deauthorization we just detect access has been revoked and stop trying to read the manifest files + updating the dependency tree.

However, we do have the ability to manually handle CCPA and equivalent data deletion requests should there be a need.

@nehzata Thank you for the answers! When you say "you only read manifest files", does that mean that's all that is stored? Or is the entire repo stored, just only those particular files are searched for and read?

nehzata commented 11 months ago

@mdtro Apologies for not being clear. We only download and store the manifest files. More specifically we:

Obtain list of files in repository via GH REST API
Filter list for manifest files
Download and store the filtered list of files via GH REST API

getsentry / team-ospo