Fix OOM lakectl FS upload bug

[idanovo]

Closes #8088

[github-actions[bot]]

E2E Test Results - DynamoDB Local - Local Block Adapter

[github-actions[bot]]

E2E Test Results - Quickstart

[idanovo]

@itaiad200 Yes, follow me-

From the uploadObject/UploadPartObject we have this code:

    req, err := http.NewRequestWithContext(ctx, http.MethodPut, preSignURL, body)
        ...
    putResp, err := u.uploader.HTTPClient.Do(req)

the type of the body we pass to NewRequestWithContext is local.fileWrapper which implement seek but doesn't implement close so the http.NewRequestFromContext function get to this code:

    rc, ok := body.(io.ReadCloser)
    if !ok && body != nil {
        rc = io.NopCloser(body)
    }

So we get a request from body that implements close (NopCloser) but doesn't implement seek

Now when getting to this line- putResp, err := u.uploader.HTTPClient.Do(req) If our HTTP client is the retryable client the Do function will eventually get to this code:

    case io.ReadSeeker:
         ...
    // Read all in so we can reset
    case io.Reader:
        buf, err := io.ReadAll(body)

Now, because our body doesn't implement seek but does implement read, we will call io.ReadAll and read the entire file into the memory and crush.

[idanovo]

Thanks for the clear explanation. How was this tested?

@itaiad200 I built a docker image that runs lakectl fs upload and tries to upload a 1GB file using the --memory=512m flag. First I saw that the bug reproduced when running it using the master branch, and then tested that my code made it work.

[arielshaqed]

https://imgflip.com/i/99kzey