search

Kinetic / kinetic-protocol

34 stars 21 forks source link

Question on GETLOG messages #28

Open abhideodhar opened 9 years ago

abhideodhar commented 9 years ago

As per the kinetic protocol, the Getlog command can be used to provide messages back to the client. But the message size is limited to 1MB. What was the rationale behind the limit?

What should the server do if there are more than 1MB messages to be sent back to the client? Can we have some kind of a continuation message in the protocol so that the client keeps reading 1MB chunks of messages until the server sends a special value to signal "No more messages"?

thanks Abhijit

abhideodhar commented 9 years ago

@jphughes @chiaming2000

Thoughts - ?

abhideodhar commented 9 years ago

@jphughes - Hey James, can you please help us understand the 1MB limit on the protobuf in the protocol?

jphughes commented 9 years ago

I assume you are talking about increasing other lengths than the normal get/put value size? Since you asked about the protobuf length I will answer asuming that you are not asking about changing the get/put value size.

There are several things that lead to this limit. I believe the following is true.

1) the current implementation of getlog messages returns the device messages inside the .proto. The recomendation of google is that .proto should be less than a megabyte. This is just a recommendation though.

2) In Kinetic case, the normal value field has a 1MB limit with one exception. Download firmware allows a value that can be many megabytes long.

3) the "getlog device" returns a something device specific and is up to 1MB in the value field.

If there is need to return more, we can move the getlog messages to the value and increase the value length. We can also increase the getlog device to be more than a megabyte. These are all possible. The libraries would have to deal with lengths greater than 1MB, but there would be no other change in the protocol other than allowing larger.

Is there a specific think where you want to return more than 1MB?? Changing the getlog device to return more would be trivial?

abhideodhar commented 9 years ago

@jphughes - James - Thanks for the response.

Yes, I was asking wrt getlog and firmware download.

I was thinking in terms of device logs, 1MB may not be enough to debug a problem on the server. So I wanted a way to return more than 1MB worth of messages from the server.

It would be nice if we move it to the value field and increase the value length. Thanks!

jphughes commented 9 years ago

Since getlog device is a vendor specified function, I don't see a problem returning more than 1MB in a value. The protocol will support it, and the performance impact (because it is a debug command) is not significant.

No changes are needed to the protocol to allow this. Maybe some libraries will have heartache.