twitter / finatra

Fast, testable, Scala services built on TwitterServer and Finagle
https://twitter.github.io/finatra/
Apache License 2.0
2.27k stars 406 forks source link

StreamingResponse stops streaming before the Inputstream is completely read #441

Closed osolliec closed 6 years ago

osolliec commented 6 years ago

I am trying to download a file from a Finatra server, the file stops downloading but the size is incorrect.

Expected behavior

The file should actually be downloaded wholly. Its size is 443MB.

Actual behavior

Using a Java InputStream, the file will download, chunk by chunk, but will stop randomly at less than 100MB.

Steps to reproduce the behavior

Using Scala version "2.12.4" finatra-http version "17.11.0"

See the following code :

import java.io.FileInputStream

import com.twitter.concurrent.AsyncStream
import com.twitter.finagle.http.Request
import com.twitter.finatra.http.Controller
import com.twitter.finatra.http.response.StreamingResponse
import com.twitter.io.{Buf, Reader}

class TestController2 extends Controller {
  get("/hi") { request: Request =>
    /* this does not work, getting "Success writing to chunked response" while the file is not wholly downloaded  */
    val inputStream = new FileInputStream("/path_to_large_file")
    val asyncStream = AsyncStream.fromReader(Reader.fromStream(inputStream))
    val headers = Map(
      "Content-Type" -> "application/octet-stream",
      "Content-Disposition" -> s"attachment; filename=large-file.tar.gz"
    )

    StreamingResponse(Predef.identity[Buf], headers = headers)(asyncStream)

    /** This works fine and gives me the actual number of bytes in the inputstream **/

    var nRead = 0
    val data = new Array[Byte](16384)
    var totalBytes = 0

    while ( {
      nRead = inputStream.read(data, 0, data.length);
      nRead
    } != -1) {
      totalBytes += nRead
    }
  }
}
osolliec commented 6 years ago

Ok, it had nothing to do with finatra, but the reference of the inputstream provider was garbage collected, and the InputStream was thus empty. Hopefully this issue will help the poor souls who want to download files with finatra.