search

kothar / go-backblaze

A golang client for Backblaze's B2 storage
MIT License
95 stars 31 forks source link

use of closed network connection #37

Open jkowalski opened 4 years ago

jkowalski commented 4 years ago

Thanks for the provider. We're using it in https://github.com/kopia/kopia

I got a bug report from one of the users https://github.com/kopia/kopia/issues/547 reporting the provider sometimes returns errors like: 

Post "https://pod-000-1042-01.backblaze.com/b2api/v1/b2_upload_file/a80b691f7eeff328733e0e15/c001_v0001042_t0059": write tcp 10.1.2.xxx:36218->206.190.208.112:443: use of closed network connection

The calling code is in: https://github.com/kopia/kopia/blob/48f253173b7a0aa0a24ac30cdca755f730ea2ea5/repo/blob/b2/b2_storage.go#L180

I presume a single client is safe to be used concurrently by multiple goroutines. If that's indeed the case what could be the reason for the error?

kothar commented 4 years ago

Thanks for the bug report. Yes, the client should be concurrency safe - there are a couple of places where coordination is needed (credential refresh for example) and are guarded by mutexes.

I'll have a look at how this might be triggered and let you know what I find.

jkowalski commented 4 years ago

any updates here?

kothar commented 4 years ago

Sorry, nothing that I've been able to spot. There is a potential problem with rereading a buffer in the event of a retried upload request, but each upload should always use a new connection, so I'm not sure where the use of closed network connection message could come from. Maybe related to the discussion here: https://github.com/golang/go/issues/21760

I'll try to find time to write some better tests around the uploads to try to reproduce the issue, any contribution in that area would be most welcome :)