Closed cchantep closed 3 years ago
On osx, each packet passed to WriteAt
is at most 32768
bytes.
On Linux is the same, it is the client that sets the packet size. When used as client pkg/sftp use 32768 as default too.
Then it would be nice to make available the detected packet size from the current session at hook level.
Then it would be nice to make available the detected packet size from the current session at hook level.
I don't understand sorry, you already have the size for each packet. A strange client might send packets of different sizes, and the final packet will almost always have a different size.
The last packet is another story but the max size of packet in current session is useful
There is no concept of max packet per session, is the client that decide the packet size, this library accepts 256*1024 bytes as max packet size.
Can you please point me to the relevant SSH/SFTP specs? Maybe I'm missing something, thanks
Apart from using the offset / sessionMaxPacketSize
there is no way to determine the absolute order (aka index starting from 0) of the current packet a custom WriteAt
provided by hook.
Some logic to write the received packet require that.
Every WriteAt
will have the offset
tracked properly by the calls to WriteAt()
, and the packet size of the data to write tracked by len(b)
.
I just don’t understand by what you mean by “absolute order” when… oh wait, I think I just got it. You want to be able to order the packets not just the offset
? I don’t understand why that would be necessary. And writers could write these packets in any order of sizes, they don’t always need to be the same length as the maximum packet size.
For instance, a client could write per buffered line of output, rather than waiting until it had a completely full buffer then transmitting.
I don’t think what you want is possible.
The len(b)
is the size of the current buffer, not the expected max packet size.
And yes, absolute order may be required, as where you write in WriteAt
may require it and not support the very same offset as reference.
Hi, you can convert the WriteAt
to a sequential writer using for example pipeat or reordering the packet yourself based on the offset in a similar way. The server cannot limit the packet size, it can only refuse to handle too big packets, where too big is quite arbitrary
The use case is not to pipe and the offset by itself is not sufficient to reorder. It only makes possible to determine whether the current offset is greater than the last processed one, not that the current one it's the next one as there is no guarantee another packet must be processed between.
I don't know your use case, maybe you should better explain it.
Anyway, the offset is enough to reorder the received packets if you accumulate them until you have no "holes". This is what pipeat does.
Another option is to set SftpServerWorkerCount
to 1 here: this way reads/writes will become sequential: the client will decide the packet order in this case (a client could send in parallel anyway)
I’m sorry, but there’s just no way that the SFTP protocol will allow us to obtain the feature you are looking for. A client could send any data it wants in any particular arbitrary order, with any number of offsets and packet sizes.
We have a ton of extensive code to coördinate multiple parallel requests in WriteTo
and ReadFrom
. There is no way to get you an absolute order, except by, as drakkan noted, keeping track of how much has been written in total, and delaying the writing of packets until you receive a packet at lastWrite.offset + lastWrite.length
.
The SFTP protocol itself does not provide any sequential writing/reading, it is always “WriteAtand
ReadAt`, it is, in this way, similar to a raw file-system store, and ensuring the consistency of your data is a problem left for the client and server to negotiate.
There is nothing we can do here, sorry
In RequestServer it could be useful to know the max packet size to implement custom handler (for Put operation). For know it seems that on OSX it's
32768
but would be nicer to have a clean/portable way to determine that.