2020IP / TwentyTwenty.Storage

A cross-cloud storage abstraction
Other
100 stars 26 forks source link

.NET HttpClient IO Exception The response ended prematurely, with at least XXXX additional bytes expected #46

Open jlcarpenter opened 10 months ago

jlcarpenter commented 10 months ago

I am calling an internal API to get a data stream that that I will use. I occasionally receive an IOException that says The response ended prematurely, with at least XXXXX additional bytes expected. It seems that the service is closing the stream before my client has read all the data. Based on that there does not seem to be anything that can be done on the client-side to prevent this.

Is there something I can do on the server side to prevent the connection from being closed.

Client Code:

    ServicePointManager.SecurityProtocol |= SecurityProtocolType.Tls11 | SecurityProtocolType.Tls12;
    var httpClient = _httpClientFactory.CreateClient();
    httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", token);
    httpClient.Timeout = TimeSpan.FromHours(1);
    var response = await httpClient.GetAsync(request.link, HttpCompletionOption.ResponseHeadersRead, cancellationToken).ConfigureAwait(false);
    if (!response.IsSuccessStatusCode)
    {
        return new GetRawDataResponse(false, null);
    }

    var streamToReadFrom = await response.Content.ReadAsStreamAsync(cancellationToken).ConfigureAwait(false);
    return new GetRawDataResponse(true, streamToReadFrom);

Server Code:

HttpGet("{blobId}")]
  [ProducesResponseType((int)HttpStatusCode.OK)]
  public async Task<ActionResult> GetBlob(string version, string blobId)
  {
    _logger.LogInformation("GetBlob() called with blob id of {BlobId}", 
    var (blobDescriptor, stream) = await GetBlobInfo(blobId).ConfigureAwait(false);

    if (blobDescriptor == null || stream == null)
      return NotFound("No BLOB found for that ID");

    return File(stream, blobDescriptor.ContentType, blobDescriptor.Name);
  }

Edit: It does seem to happen more frequently when multiple items are being streamed from the source.