API rate limit exceeded for user ID 9599

[simonw]

I'm seeing this in my deploys quite often at the moment: https://github.com/simonw/til/actions/runs/3210803803/jobs/5248557801

I tried contacting GitHub support and asking for a rate limit raise but I got back a templated response that effectively said no.

[simonw]

This is happening in the "checkout til-db" step, which uses a personal access token so I can push back to that other repo in a later step:

https://github.com/simonw/til/blob/a7a144042f0829adcdf335e42f284f0b1d659a76/.github/workflows/build.yml#L18-L23

This was my very hacky workaround from a while ago to store the TIL database in between steps even though it was too big to download from Vercel:

• 25

I'm going to store it in an S3 bucket instead, using s3-credentials.

[simonw]

I'm going to stash the file in a folder in a new bucket:

~ % s3-credentials create til.simonwillison.net -c
Created bucket: til.simonwillison.net
Created  user: 's3.read-write.til.simonwillison.net' with permissions boundary: 'arn:aws:iam::aws:policy/AmazonS3FullAccess'
Attached policy s3.read-write.til.simonwillison.net to user s3.read-write.til.simonwillison.net
Created access key for user: s3.read-write.til.simonwillison.net
{
    "UserName": "s3.read-write.til.simonwillison.net",
    "AccessKeyId": "AKIAWXFXAIOZOFEAZH56",
    "Status": "Active",
    "SecretAccessKey": "...",
    "CreateDate": "2022-10-08 15:08:04+00:00"
}

Secret access key is in my 1Password.

[simonw]

Uploading a copy of that database from the til-db repo:

til-db % s3-credentials put-objects til.simonwillison.net tils.db
Uploading 27.2 MB (1 file)  [#####-------------------------------]   15%  00:01:13

[simonw]

Realized I want the bucket to be public, so deleted and recreated:

aws s3api delete-object --bucket til.simonwillison.net --key tils.db
aws s3api delete-bucket --bucket til.simonwillison.net
s3-credentials create til.simonwillison.net -c --public
s3-credentials put-objects til.simonwillison.net tils.db

[simonw]

Confirmed: this works without authentication now:

wget https://s3.amazonaws.com/til.simonwillison.net/tils.db

[simonw]

Added those AWS secrets to this repo.

[simonw]

That fixed it.

[simonw]

I'm going to archive the til-db repo now.