Closed hannahhoward closed 1 month ago
Great insights.
Also, what the heck is happening between the carpark fetches?
Random idea that might lead to a reduction in time, we might want to tweak the batch size - it's set to 16 but I think workers can only make 6 concurrent requests at a time (the public bucket is fronted by a worker for multipart byte range support but also because I figured in the future we might have some sort of UCAN auth to do...).
@alanshaw I believe the spans only cover the time it takes to call fetch, and not the amount it takes to consume the body of the request. I suspect that's the time required to actually download the data.
Goals
Gain insight into what's going on with freeway
Implementation
When in doubt, instrument. This adds opentelemtry and a default cloudflare open telemetry implementation, that at minimum tells you how long fetch requests are taking. And it's.... very interesting...
This is a trace of about a 64mb file downloading.
I dunno what to make of all it, except to say... ouch those CARPARK downloads take a WHILE. I wonder if somehow accessing them through public URLs is a lot slower?