square / okio

A modern I/O library for Android, Java, and Kotlin Multiplatform.
https://square.github.io/okio/
Apache License 2.0
8.72k stars 1.17k forks source link

IllegalArgumentException when trying to write a file #1457

Closed vegidio closed 3 months ago

vegidio commented 3 months ago

I created the code with Ktor and Okio to stream download a file:

  suspend fun download(url: String, filePath: String) {
      val client = HttpClient()  // Ktor HttpClient https://ktor.io/docs/create-client.html#configure-client
      client.prepareGet(url).execute { response ->
          val channel = response.bodyAsChannel()
          val path = filePath.toPath()
          val sink = fileSystem.sink(path).buffer()

          while (!channel.isClosedForRead) {
              val packet = channel.readRemaining(DEFAULT_HTTP_BUFFER_SIZE.toLong())

              while (packet.isNotEmpty) {
                  val bytes = packet.readBytes()
                  sink.write(bytes)
              }
          }
      }
  }

This function works well with most files, but I noticed that sometimes I throws the following exception that seems to be coming from okio.Buffer#readByteString:

Uncaught Kotlin exception: kotlin.IllegalArgumentException: byteCount: 3279151104
    at 0   umd                                 0x1048fb01b        kfun:okio.Buffer#readByteString(kotlin.Long){}okio.ByteString + 427
    at 1   umd                                 0x104a5421b        kfun:io.vinicius.umd.$downloadMediaCOROUTINE$0.invokeSuspend#internal + 2555
    at 2   umd                                 0x1047b65db        kfun:kotlin.coroutines.native.internal.BaseContinuationImpl#resumeWith(kotlin.Result<kotlin.Any?>){} + 171
    at 3   umd                                 0x10483feb3        kfun:kotlinx.coroutines.DispatchedTask#run(){} + 663
    at 4   umd                                 0x104828def        kfun:kotlinx.coroutines.EventLoopImplBase#processNextEvent(){}kotlin.Long + 715
    at 5   umd                                 0x104847f53        kfun:kotlinx.coroutines#runBlocking(kotlin.coroutines.CoroutineContext;kotlin.coroutines.SuspendFunction1<kotlinx.coroutines.CoroutineScope,0:0>){0§<kotlin.Any?>}0:0 + 831
    at 6   umd                                 0x104a5241f        kfun:io.vinicius.umd.Cli#run(){} + 5227
    at 7   umd                                 0x104a4e1ff        kfun:com.github.ajalt.clikt.parsers.Parser.parse#internal + 22311
    at 8   umd                                 0x104a56d8f        kfun:io.vinicius.umd#main(kotlin.Array<kotlin.String>){} + 2731
    at 9   umd                                 0x104a94e0f        Init_and_run_start + 383
    at 10  umd                                 0x104a94f43        Konan_main + 15
    at 11  dyld                                0x18b14d0df        start + 2359

Do you have any idea what could be happening here? This is a KMP program running in a Mac.

Thanks!

JakeWharton commented 3 months ago

That code is not responsible for the exception, although you're missing a call to .use { ... } after .buffer().

The problem with the actual offending code is that you're passing more than Int.MAX_VALUE to Buffer.readByteString. A Buffer can hold more than Int.MAX_VALUE but a ByteString cannot.

vegidio commented 3 months ago

The lack of .use {...} in my code was indeed the source of the problem. Thank you very much!