search

cloudflare / wrangler-legacy

🤠 Home to Wrangler v1 (deprecated)
https://workers.cloudflare.com
Apache License 2.0
3.2k stars 337 forks source link

Wrangler timeout too short for bulk uploads #746

Closed gregmsanderson closed 5 years ago

gregmsanderson commented 5 years ago

🐛 Bug Report

Environment

Steps to reproduce

I've been trying the newly added Workers Sites as it looked awesome. I tried it first with a simple site with about 10 assets. Worked perfectly, first time. Dev, Prod, all fine. Yay. So I then tried it with another site with more pages and many more assets (mainly images). This time, the publish failed :(

It seems to be at the point where it uploads all the site's static files to KV. The time out seems too short ... can it be increased? I guess this is outside of wrangler but maybe that request could be passed on. Since I can see it using the /bulk operation which makes sense. However I'd think someone would only be using the /bulk API if they were uploading a lot and so would need a generous timeout, even outside of Workers Sites.

It's not clear from the error if it is a timeout with one file (like too big, and so timing out) or the entire operation (too many files within its allowed time)

I'd guess there are maybe 500 files.

What did you expect to see?

The publish to successfully complete

What did you see instead?

$ wrangler publish --env dev 🌀 Using namespace for Workers Site "__name-dev-workers_sites_assets" 💁 ignoring file .DS_Store 💁 ignoring file .DS_Store 💁 Uploading... Error: ⚠️ Error: https://api.cloudflare.com/client/v4/accounts/longidhere/storage/kv/namespaces/longidhere/bulk: timed out

ashleymichal commented 5 years ago

thanks @gregmsanderson for the bug report! we've notified the team that works on the KV API and are working out a solution with them. That team is internal to Cloudflare so I can't necessarily link you to an issue to track, but I'll update here when we have a fix.

bretthoerner commented 5 years ago

@gregmsanderson I'm curious approximately how long it's taking before it times out. If this is repeatable I wonder if you could prefix your command with time (if you're on Mac/Linux).

It would be awesome for wrangler to print the time taken and the Ray ID (if there is one).

ashleymichal commented 5 years ago

it looks like we need to increase the default timeout on the client that Wrangler uses for these requests; we use reqwest (via cloudflare-rs) with no increase to the default timeout of 30s. I'm going to update the name of this ticket and we'll triage ASAP.

gregmsanderson commented 5 years ago

Hey @ashleymichal @bretthoerner

Yes, It seemed to give up after about a minute. May well have been even less, like 30s, since as you say there is no time report on the output, and I didn't run a stopwatch or anything to see exactly when. I just assumed it had some timeout-after-X rule that it was hitting.

I wasn't aware of that time command. That would indeed have helped.

Unfortunately I'm currently unable to do that for you. Earlier on today I had an idea to bypass the problem. Since the upload was timing out, I needed a faster way to upload to your server. Since there's nothing I can do about the connection from here, I figured I would instead run wrangler publish from a server I assume is near yours, using the CircleCI service. If not, at least with a faster network. And indeed I was right: their connection to you is super-fast. Watching the output of wrangler publish in their system, the bulk upload operation sails by. Hundreds of files upload in seconds. And so the bulk operation does not time out when run from there. And the operation completes.

So if I were to now publish myself, that will confuse things. But it sounds like Ashley is on the case and so that won't be needed. And it's less of an issue for me now since I might as well stick with CircleCI. But I'm sure it will be useful for others to increase that timeout a little.

Thanks.