Torrent.BytesMissing can be out of sync with storage

[luodw]

Hi, we have a trouble when use torrent to distribute data with torrent version 1.15.1. We use Torrent.BytesMissing() to judge whether data is completely downloaded and return data to client.

But some times Torrent.BytesMissing() == 0 and the data returned is incomplete, the data has some content with zero, so client download failed. It is happened occasionally. I guess although Torrent.BytesMissing() == 0, but some chunks may be not written to file.

Has anyone encountered this issue before? or has any ideas?

[anacrolix]

Yeah you can get BytesMissing==0 while data is still being written storage. The value tracks the state internal to the client, when it's 0, it's not expecting to need to download any more to complete a Torrent, but it could if piece hashing fails or there's a storage issue.

It would be possible to change it to track the storage.

It would be helpful to know what you're coordinating with BytesMissing.

[luodw]

@anacrolix Thanks for your reply.

1. "It would be possible to change it to track the storage." Is there a method/function ready to use?

2. "It would be helpful to know what you're coordinating with BytesMissing." we use ByteMissing just like follow:

   pcs := torrent.SubscribePieceStateChanges()
   for torrent.BytesMissing() > 0 {
      select {
      case <-pcs.Values:
      case <- time.After(time.Second*60):
            return
   }
   // data has download completely, return to client
   _, err := io.copy(http.ResponseWriter, torrent.file)

Or what is the right practice to judge whether data is downloaded and complete?

[anacrolix]

It's always safe to read directly from the Torrent or File Readers. You probably want to be using Torrent.File.NewReader and just stream directly out of that without doing any manual prioritizations.

t := some Torrent
<-Torrent.GotInfo()
f := some Torrent.Files()
r := f.NewReader()
http.ServeContent(http.ResponseWriter, r, ...)
r.Close()

See https://github.com/anacrolix/confluence/blob/master/confluence/misc.go#L82, and follow https://github.com/anacrolix/confluence/blob/master/confluence/mux.go#L19 through to its conclusion for a full example.

[luodw]

@anacrolix thanks, I will have a try.

[luodw]

@anacrolix Thanks, this is work.

By the way, Is there a way for a peer to disable outgoing connect? we use torrent to distribute image and have a role named origin. Origin is responsible for downloading data from source and make torrent, and other peer fetch torrent from origin and start p2p download. I think origin has no need to outgoing connect, as it has all data and just wait for connect.

maybe set 'HalfOpenConnsPerTorrent' to zero work？

[anacrolix]

@luodw That's a fantastic question. A quick look suggests setting HalfOpenConnsPerTorrent to zero is the best option available to you. Also make sure that you have Seed true (I guess you do).