sipcapture / HEP

HEP-EEP: Extensible Encapsulation Protocol (Specs & Technical Docs)
http://sipcapture.org
46 stars 8 forks source link

HEP Max message length #9

Open kinsu opened 1 year ago

kinsu commented 1 year ago

Assuming that we want to use HEP to encapsulate an HTTP2 message (frame), max message size defined in HEP header (currently 65535) can be restrictive. i guess header change means new HEP version ?

adubovikov commented 1 year ago

good question :-) i think yes - otherwise you have to implement fragments. But can you please describe the scenario for such big duplication ? Normay we use payload protocols for miroring, but not entire transport frame.

kinsu commented 1 year ago

thanks for the fast reply, yes thats true but payload (e.g. HTTP2 body in Captured packet payload chunck) can grow and exceed the 65K which would cause the whole HEP message to overflow too or i am missing something ?

adubovikov commented 1 year ago

honestly, and maybe I wrong, we don't expect SIGNALING protocols size over 65K. As I mentioned before, to duplicate HTTP2 Frames which size is over 65K, by using HEP is absolutely not efficient. First too much overhead (memcopy, serialization, deserialization), second too much traffic, third - too huge receiver buffer. IMHO better duplicate and store directly from network stack the packets as they arrived. But again, it's IMHO :-)

kinsu commented 1 year ago

valid point about the overhead and NW packets but the issue is that with HTTP2 comes the encryption so duplication/mirroring on this level is tricky.

would you consider to increace the max size in a followup version ?

lmangani commented 1 year ago

@kinsu This is interesting. Could you expand on your usecase/project where this would be required?

kinsu commented 1 year ago

The intention is to use HEP to encapsulate HTTP2 for traffic monitoring. And in this case as can be seen in 3GPP 29.510 NFDiscovery responce includes an Array of NFProfile objects which can grow quite big (eg MB).

adubovikov commented 1 year ago

we can start working on HEP4 and increase the length field to 32 bytes (IMHO ~4G payload will be enougth) ;-) also you can use Protobuf and custom schema to send data to heplify-server - just another way

lmangani commented 1 year ago

The beauty of HEP3 is its extensibility. We will shortly propose a backwards compatible extension for discussion.

kinsu commented 1 year ago

thank you i ll be waiting for the proposal- especially the backwards compatibility aspect

kinsu commented 1 year ago

By the way, are there any other changes that you have in mind as part of the HEP4 or just the field size that we were discussing above ? Also, do you have any (obviously rought) timeline on when the proposal will be available ?

thanks again for the help!

adubovikov commented 1 year ago

it probably take a week - we are now busy with cloud launch. Thank you!

lmangani commented 1 year ago

I personally doubt this change alone would be worth releasing HEP4....

kinsu commented 1 year ago

it probably take a week - we are now busy with cloud launch. Thank you!

thanks !

I personally doubt this change alone would be worth releasing HEP4....

are there any other items that are requested/planned for HEP4 ?

kinsu commented 1 year ago

Hi ! are there any news on this topic ? thx