Static files are getting close to the 5MB serve limit

[Alex-At-Home]

2021/22 players Low tier got that high

Options:

• Split files into multiple sub-files (I wonder if that would also render faster?!)
• Can I store the data as .gz files and then send the binary without decompressing

[Alex-At-Home]

Note that I believe every request to the player leaderboard if any of them are >5MB costs me GCS transfer monies also

Maybe add a line to the ingest pipeline to not transfer any files that are >=5MB?

[Alex-At-Home]

Couldn't immediately get the direct gz working

            const respBlob = await resp.blob();
            const respBinaryArray = await respBlob.arrayBuffer();
/**/
console.log(respBinaryArray.byteLength);            
            //const respJson = await resp.json();
            res.setHeader("Content-Encoding", "gz");
            res.setHeader("Content-Type", "application/json");
            res.setHeader("Cache-Control", "s-maxage=28800"); // requests that the CDN cache this for 12 hours or until the app redeploys
//            res.status(200).json(respJson);
            res.status(200).send(Buffer.from(respBinaryArray));

Returned and send a binary object but wasn't able to decode it as JSON

[Alex-At-Home]

Ah GCS also caches the data for an hour, which isn't ideal

Options:

• Leave
• Reduce to eg 10 mins via Cache-Control
• Figure out how to invalidate on deploy (eg is there an env variable that is deployment time? I can always insert one)

[Alex-At-Home]

For now I added cacheId=${process.Env.VERCEL_URL} which I think but haven't confirmed will update on each redeploy (but otherwise should give 1 hr)

[Alex-At-Home]

Ended up getting compression working