Question - What happens to Data onces i Read it

[abhpat]

If I open a 10 GB file on 1 GB lambda server and read bytes using io.LimitReader and send it to multipart upload I have tried it and its working but what happens when I reach the 1GB server limit? shouldn't it crash and tell no space on disk, onces the Ftp server sends me data more than 1 GB?

I am sure this is a lag in my conceptual knowledge

But assume I have a file Object Pointer var file = SftpSession.Open(Filename)

then file would be a object that points to a location in my main memory and store file name file pointer - Where I am reading the file

If i do a io.LimitReader(file,10) Means I read 10 bytes from the file so that means file.pointer would increment by 10

but the actual data would still be there in the main memory right? I just want to know what exactly is happening behind the scenes

am i Overwriting the data itself - if yes would it be safe? Like will it always overwrite the oldest data

or the data is getting deleted once io.LimitReader reads it

[puellanivis]

io.LimitReader is just a wrapper around the other reader. It does not actually buffer any data itself, it just keeps track of the buffer size input, and the amount left to read, and shortens the read so that it never reads more than N in total.

So, the actual file data is dependent upon how the underlying reader handles the reads. The totality of the code of the io.LimitReader fits on one page: https://golang.org/src/io/io.go?s=15094:15821#L445

You can see there that there is no buffer, just a pass through, and tracking of how much data is left to read.

[puellanivis]

Closing as answered. I suppose additional to add is that you would be essentially streaming the data from the SFTP server into the HTTP body, so at any one time you would only have a limited amount of the actual file located in the service.