Closed nakabonne closed 3 years ago
I would like to work on this but you talk about OOM. Wouldn't that still happen if you keep storing the streamed data in the cache? Or do you mean streaming directly to the "pasting device"? What would a perfect implementation be?
@thealamu Sorry for the confusion. It doesn't mean streaming directly to the "pasting device". It means to stream it to the pbgopy server. However, pbgopy supports end-to-end encryption, so I'm beginning to feel it's not appropriate.
iouti.ReadAll()
continues to buffer the input from the STDIN until EOF appears. If there is incorrect input, it could eat up the memory on the "copying device". I'm trying to think of something else to do 🤔
Let's change it to fail if you try to copy more data than the specified size with the --size(MB)
option. By default 500MB is given.
Okay, please assign to me.
@thealamu Yes, I'll be happy with that. Your help is essential.
Can this be closed now?
Yes, resolved by https://github.com/nakabonne/pbgopy/pull/10
Currently, it uses
ReadAll()
to read from stdin, but that's dangerous and it may cause OOM.Instead, we'd better directly stream it via HTTP. That way doesn't require to buffer at all. https://github.com/nakabonne/pbgopy/blob/9e86a266f03feaa369b22e6933687c1a1e704e1e/commands/copy.go#L49