JSON responses are being truncated

Ktor Version and Engine Used (client or server and name) 1.2.4, client, core + curl + json + mock + serialization (all for Linux native)

Describe the bug I have two nearly-identical strings (containing valid JSON) and each is used by a MockEngine as GET responses. Calling HttpClient.get<T>(String, HttpRequestBuilder.() -> Unit) with one address returns the deserialised form of the shorter string, but the second call fails because the string's ending } is truncated.

This may be related to #787.

To Reproduce Steps to reproduce the behavior:

Create simple MPP with Gradle (see below)

Run task runReleaseExecutableLinux and get this output:

The Nameless City is 4012 characters long
Uncaught Kotlin exception: kotlinx.serialization.json.JsonDecodingException: Invalid JSON at 4088: Expected '}'
    at kfun:kotlinx.serialization.json.JsonException.<init>(kotlin.String)kotlinx.serialization.json.JsonException (0x385f27)
    at kfun:kotlinx.serialization.json.JsonDecodingException.<init>(kotlin.Int;kotlin.String)kotlinx.serialization.json.JsonDecodingException (0x38605a)
    at kfun:kotlinx.serialization.json.internal.JsonReader.fail(kotlin.String;kotlin.Int)kotlin.Nothing (0x387974)
    at kfun:kotlinx.serialization.json.internal.StreamingJsonInput.endStructure(kotlinx.serialization.SerialDescriptor) (0x389490)
    at kfun:dev.yoshirulz.bug_repro.Book.$serializer.deserialize(kotlinx.serialization.Decoder)dev.yoshirulz.bug_repro.Book (0x396a41)
    at kfun:kotlinx.serialization.json.internal.decodeSerializableValuePolymorphic$kotlinx-serialization-runtime@kotlinx.serialization.json.JsonInput.(kotlinx.serialization.DeserializationStrategy<#GENERIC>)Generic (0x3880f1)
    at kfun:kotlinx.serialization.json.internal.StreamingJsonInput.decodeSerializableValue(kotlinx.serialization.DeserializationStrategy<#GENERIC>)Generic (0x388f03)
    at kfun:kotlinx.serialization.decode@kotlinx.serialization.Decoder.(kotlinx.serialization.DeserializationStrategy<#GENERIC>)Generic (0x370541)
    at kfun:kotlinx.serialization.json.Json.parse(kotlinx.serialization.DeserializationStrategy<#GENERIC>;kotlin.String)Generic (0x37fd3d)
    at kfun:io.ktor.client.features.json.serializer.KotlinxSerializer.read(io.ktor.client.call.TypeInfo;kotlinx.io.core.Input)kotlin.Any (0x395a69)
    at kfun:io.ktor.client.features.json.JsonFeature.Feature.$install$lambda-1COROUTINE$3.invokeSuspend#internal (0x36e1b1)
    at kfun:io.ktor.client.features.json.JsonFeature.Feature.$install$lambda-1COROUTINE$3.invoke#internal (0x36ef7a)
    at kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal (0x326428)
    at kfun:io.ktor.util.pipeline.SuspendFunctionGun.proceed#internal (0x325cc7)
    at kfun:io.ktor.util.pipeline.SuspendFunctionGun.proceedWith#internal (0x32609f)
    at kfun:io.ktor.client.features.HttpCallValidator.Companion.$install$lambda-1COROUTINE$25.invokeSuspend#internal (0x355c89)
    at kfun:io.ktor.client.features.HttpCallValidator.Companion.$install$lambda-1COROUTINE$25.invoke#internal (0x35623a)
    at kfun:io.ktor.util.pipeline.SuspendFunctionGun.loop#internal (0x326428)
    at kfun:io.ktor.util.pipeline.SuspendFunctionGun.proceed#internal (0x325cc7)
    at kfun:io.ktor.util.pipeline.SuspendFunctionGun.execute#internal (0x326209)
    at kfun:io.ktor.util.pipeline.Pipeline.execute(#GENERIC_kotlin.Any;#GENERIC_kotlin.Any)#GENERIC_kotlin.Any (0x3241ec)
    at kfun:io.ktor.client.call.HttpClientCall.$receiveCOROUTINE$16.invokeSuspend(kotlin.Result<kotlin.Any?>)kotlin.Any? (0x3495f6)
    at kfun:io.ktor.client.call.HttpClientCall.receive(io.ktor.client.call.TypeInfo)kotlin.Any (0x349b6b)
    at kfun:dev.yoshirulz.bug_repro.$main$lambda-3COROUTINE$1.invokeSuspend#internal (0x398a01)
    at kfun:kotlin.coroutines.native.internal.BaseContinuationImpl.resumeWith(kotlin.Result<kotlin.Any?>) (0x288393)
    at kfun:kotlinx.coroutines.DispatchedTask.run() (0x2fa1b1)
    at kfun:kotlinx.coroutines.EventLoopImplBase.processNextEvent()ValueType (0x2fe7fe)
    at Init_and_run_start (0x3ac2a3)
    at __libc_start_main (0x7f7fa02c7ee3)
    at  (0x267029)
    at  ((nil))

Reproduction project:

// build.gradle.kts

plugins {
    kotlin("multiplatform") version "1.3.50"
    id("kotlinx-serialization") version "1.3.50"
}

repositories {
    jcenter()
}

kotlin {
    linuxX64("linux") {
        binaries {
            executable {
                entryPoint = "dev.yoshirulz.bug_repro.main"
            }
        }
    }

    sourceSets {
        fun ktor(module: String) = "io.ktor:ktor-client-$module:1.2.4"

        @Suppress("UNUSED_VARIABLE") val linuxMain by getting {
            dependencies {
                api("org.jetbrains.kotlinx:kotlinx-coroutines-core-native:1.3.0")
                api(ktor("mock-native"))
                implementation(ktor("core-native"))
                implementation(ktor("curl"))
                implementation(ktor("json-native"))
                implementation(ktor("serialization-native"))
            }
        }
    }
}

// Main.kt

package dev.yoshirulz.bug_repro

import io.ktor.client.HttpClient
import io.ktor.client.engine.mock.MockEngine
import io.ktor.client.engine.mock.respond
import io.ktor.client.features.json.JsonFeature
import io.ktor.client.request.get
import io.ktor.http.ContentType
import io.ktor.http.headersOf
import kotlinx.coroutines.runBlocking
import kotlinx.serialization.Serializable
import kotlinx.serialization.UnstableDefault

@Serializable
data class Book(val author: String, val name: String, val text: String) {
    fun printLength() = println("The preview of $name is ${text.length} characters long")
    companion object {
        val kSerializer = serializer()
    }
}

@UnstableDefault
fun main() {
    val httpClient = HttpClient(MockEngine) {
        engine {
            // these differ by one char at the end
            val longJSONString =
                """{"author": "H. P. Lovecraft", "name": "The Nameless City", "text": "When I drew nigh the nameless city I knew it was accursed. I was traveling in a parched and terrible valley under the moon, and afar I saw it protruding uncannily above the sands as parts of a corpse may protrude from an ill-made grave. Fear spoke from the age-worn stones of this hoary survivor of the deluge, this great-grandfather of the eldest pyramid; and a viewless aura repelled me and bade me retreat from antique and sinister secrets that no man should see, and no man else had dared to see..\nRemote in the desert of Araby lies the nameless city, crumbling and inarticulate, its low walls nearly hidden by the sands of uncounted ages. It must have been thus before the first stones of Memphis were laid, and while the bricks of Babylon were yet unbaked. There is no legend so old as to give it a name, or to recall that it was ever alive; but it is told of in whispers around campfires and muttered about by grandams in the tents of sheiks so that all the tribes shun it without wholly knowing why. It was of this place that Abdul Alhazred the mad poet dreamed of the night before he sang his unexplained couplet:\nThat is not dead which can eternal lie, / And with strange aeons even death may die.\nI should have known that the Arabs had good reason for shunning the nameless city, the city told of in strange tales but seen by no living man, yet I defied them and went into the untrodden waste with my camel. I alone have seen it, and that is why no other face bears such hideous lines of fear as mine; why no other man shivers so horribly when the night wind rattles the windows. When I came upon it in the ghastly stillness of unending sleep it looked at me, chilly from the rays of a cold moon amidst the desert's heat. And as I returned its look I forgot my triumph at finding it, and stopped still with my camel to wait for the dawn.\nFor hours I waited, till the east grew grey and the stars faded, and the grey turned to roseate light edged with gold. I heard a moaning and saw a storm of sand stirring among the antique stones though the sky was clear and the vast reaches of desert still. Then suddenly above the desert's far rim came the blazing edge of the sun, seen through the tiny sandstorm which was passing away, and in my fevered state I fancied that from some remote depth there came a crash of musical metal to hail the fiery disc as Memnon hails it from the banks of the Nile. My ears rang and my imagination seethed as I led my camel slowly across the sand to that unvocal place; that place which I alone of living men had seen.\nIn and out amongst the shapeless foundations of houses and places I wandered, finding never a carving or inscription to tell of these men, if men they were, who built this city and dwelt therein so long ago. The antiquity of the spot was unwholesome, and I longed to encounter some sign or device to prove that the city was indeed fashioned by mankind. There were certain proportions and dimensions in the ruins which I did not like. I had with me many tools, and dug much within the walls of the obliterated edifices; but progress was slow, and nothing significant was revealed. When night and the moon returned I felt a chill wind which brought new fear, so that I did not dare to remain in the city. And as I went outside the antique walls to sleep, a small sighing sandstorm gathered behind me, blowing over the grey stones though the moon was bright and most of the desert still.\nI awakened just at dawn from a pageant of horrible dreams, my ears ringing as from some metallic peal. I saw the sun peering redly through the last gusts of a little sandstorm that hovered over the nameless city, and marked the quietness of the rest of the landscape. Once more I ventured within those brooding ruins that swelled beneath the sand like an ogre under a coverlet, and again dug vainly for relics of the forgotten race. At noon I rested, and in the afternoon I spent much time tracing the walls and bygone streets, and the outlines of the nearly vanish..."}"""
            val longerJSONString =
                """{"author": "H. P. Lovecraft", "name": "The Nameless City", "text": "When I drew nigh the nameless city I knew it was accursed. I was traveling in a parched and terrible valley under the moon, and afar I saw it protruding uncannily above the sands as parts of a corpse may protrude from an ill-made grave. Fear spoke from the age-worn stones of this hoary survivor of the deluge, this great-grandfather of the eldest pyramid; and a viewless aura repelled me and bade me retreat from antique and sinister secrets that no man should see, and no man else had dared to see..\nRemote in the desert of Araby lies the nameless city, crumbling and inarticulate, its low walls nearly hidden by the sands of uncounted ages. It must have been thus before the first stones of Memphis were laid, and while the bricks of Babylon were yet unbaked. There is no legend so old as to give it a name, or to recall that it was ever alive; but it is told of in whispers around campfires and muttered about by grandams in the tents of sheiks so that all the tribes shun it without wholly knowing why. It was of this place that Abdul Alhazred the mad poet dreamed of the night before he sang his unexplained couplet:\nThat is not dead which can eternal lie, / And with strange aeons even death may die.\nI should have known that the Arabs had good reason for shunning the nameless city, the city told of in strange tales but seen by no living man, yet I defied them and went into the untrodden waste with my camel. I alone have seen it, and that is why no other face bears such hideous lines of fear as mine; why no other man shivers so horribly when the night wind rattles the windows. When I came upon it in the ghastly stillness of unending sleep it looked at me, chilly from the rays of a cold moon amidst the desert's heat. And as I returned its look I forgot my triumph at finding it, and stopped still with my camel to wait for the dawn.\nFor hours I waited, till the east grew grey and the stars faded, and the grey turned to roseate light edged with gold. I heard a moaning and saw a storm of sand stirring among the antique stones though the sky was clear and the vast reaches of desert still. Then suddenly above the desert's far rim came the blazing edge of the sun, seen through the tiny sandstorm which was passing away, and in my fevered state I fancied that from some remote depth there came a crash of musical metal to hail the fiery disc as Memnon hails it from the banks of the Nile. My ears rang and my imagination seethed as I led my camel slowly across the sand to that unvocal place; that place which I alone of living men had seen.\nIn and out amongst the shapeless foundations of houses and places I wandered, finding never a carving or inscription to tell of these men, if men they were, who built this city and dwelt therein so long ago. The antiquity of the spot was unwholesome, and I longed to encounter some sign or device to prove that the city was indeed fashioned by mankind. There were certain proportions and dimensions in the ruins which I did not like. I had with me many tools, and dug much within the walls of the obliterated edifices; but progress was slow, and nothing significant was revealed. When night and the moon returned I felt a chill wind which brought new fear, so that I did not dare to remain in the city. And as I went outside the antique walls to sleep, a small sighing sandstorm gathered behind me, blowing over the grey stones though the moon was bright and most of the desert still.\nI awakened just at dawn from a pageant of horrible dreams, my ears ringing as from some metallic peal. I saw the sun peering redly through the last gusts of a little sandstorm that hovered over the nameless city, and marked the quietness of the rest of the landscape. Once more I ventured within those brooding ruins that swelled beneath the sand like an ogre under a coverlet, and again dug vainly for relics of the forgotten race. At noon I rested, and in the afternoon I spent much time tracing the walls and bygone streets, and the outlines of the nearly vanishe..."}"""
            val responseHeaders = headersOf("Content-Type" to listOf(ContentType.Application.Json.toString()))
            addHandler { request ->
                if (request.url.host == "localhost") when(request.url.encodedPath) {
                    "/long.json" -> return@addHandler respond(longJSONString, headers = responseHeaders)
                    "/longer.json" -> return@addHandler respond(longerJSONString, headers = responseHeaders)
                }
                error("${request.url} should not be requested")
            }
        }
        install(JsonFeature)
    }
    try {
        runBlocking {
            // works:
            httpClient.get<Book>("http://localhost/long.json").printLength()
            // doesn't:
            httpClient.get<Book>("http://localhost/longer.json").printLength()
        }
    } finally {
        httpClient.close()
    }
}

Expected behavior The JSON deserialiser should recieve the whole response.

I apologise if this is actually a bug in kotlinx.serialization.

ktorio / ktor

JSON responses are being truncated #1317