ASTF limitation - Connections from multiple different client IPs with multiple profiles to single server port

[egwakim]

Hi @hhaim , We need to simulate single server IP + single server port which servicing unexpected client IPs from multiple profiles. Current trex-core cannot cover this scenarios because profile is mapped by server port. Example) Profile1 : client 1.1.1.1 -> server 2.2.2.1:80 Profile2 : client 1.1.1.2 -> server 2.2.2.1:80 Server only mode cannot resolve this issue because program address of profile can be mixed in receiver side, server need to map proper profiles based on received packets.

To be able to cover this scenarios, profile mapping need to be done by connection(client IP + server port). It's possible in general servers, but, not possible in trex now.

Do you agree to resolve this limitation? Do you have any other better ideas to resolve this limitation?

If it'll make additional overhead to match profiles when receiving packets, then, we can have separated running option to avoid performance issue in general case. If you agree to resolve this limitation, we can contribute it to trex-core to resolve this limitation.

Thanks

Best Regards

Gwangmoon

[hhaim]

@egwakim

We could split this ask to two things

1. Support this in a single profile
2. Support it in a multi profile (same server same port serves multi template profile

To support 1 to support (1) here are two levels of fix (we should associate an emulation layer based on either something on the first packet or in the first payload)

1. The range of the clients/servers in case each template has a unique range
2. In case of an overlap, we could defer the emulation layer association to the first request (L7) and choose by a filter on the input bytes (the requests L7 data are known ahead of time so we could find offset by one byte and choose from it assuming the middle-box does not change it)

To support this in multi-profile there is a need to fix https://github.com/cisco-system-traffic-generator/trex-core/issues/376 first

BTW with this fix this issue will be solved too https://github.com/cisco-system-traffic-generator/trex-core/issues/339

Elad started to work on this

[egwakim]

Thanks for comment, We started to investigate solution.

[hhaim]

Hi @egwakim, @elados93 started to work on this issue. if you want to work on that due to others reasons, I don't see an issue. We can share the planned solution. Let me know

[egwakim]

Hi, Let me summarize the current status. The issue #376 is different issue, it's about random 'source port', Kangphil will send PR about it.

Regarding solution, To be able to support it in 'server only' mode, we should not associate client IP to server. Instead, we need to 'learn client IP' and map the client IP to specific template.

If we run in IP learning mode 1) Add 'template id' in initial SYN packets 2) Map client IP to specific template id after SYN packet received : Client IP learned 3) Map template based on learned client IP 4) Remove mapping after template removed

The drawback of this approach is 1) Additional payload will be added in initial SYN packets 2) Complex to handle it in case of SYN packet loss

What is your opinion about it?

Best Regards Gwangmoon

[hhaim]

@egwakim

1. In regard to #376 see my answer there.
2. We can't add any data to the SYN. Firewall will drop the SYN.
3. The solution we have discuss internally is like that and has two parts (for one profile)

3.1 Range of either client or server ips -simple. - work if there is no overlap in server range or client range . user will provide how the decision should be done (by client - in case of LB, or server in case of NAT) 3.2 Associate the emulation layer only based on the payload.

3.2.1 TCP : wait for payload, choose a byte pattern that identify to which emulation program 3.2.2 UDP the first packet has the information

Let me know if you need more info

[egwakim]

That's good suggestion, Some user may want to send zero payload data packets. If only initial packets have such payload it'll not the issue. In UDP case, if first packet lost, it'll be issue.

[hhaim]

@egwakim the length of the payload could be added to (3.2) engine. However there could not be two servers on the same port that one waits for request and one send request. this something we should enforce. This is how real servers works, it reads the request and provide response base on that

rule: len>100 action: template 1
Rule: Len==7 actions: template 2
rule: byte[7] == "b"    action: template 7

[jsmoon]

@hhaim Regarding 3.2.1, what about using TCP Option in the SYN packet? Is it also impossible? If it has no issue, I think the implementation is more simple than emulating the real server works.

[hhaim]

@jsmoon nothing! Firewall are built to find all the holes. In STF days I've tried everything and failed -- even the ACK number in the SYN can't be used .. We should work like a normal application, without any tricks. The range of servers is information (can be used in SYN), length of the request is an information, the payload data is an information.

[jsmoon]

@hhaim (a) I cannot understand the following statement.

'even the ACK number in the SYN can't be used ..'

As far as I know, the ACK number field in the SYN packet (not SYN+ACK) is undefined. So, I don't understand how the firewall can decide to drop the SYN packet by checking the undefined field value. Could you explain the situation?

(b) Since I'm thinking a template should match between client and server emulation program, I'm considering a trick bypassing the normal server application's front behavior. If you don't mind I'll try a trick which adds template information in the SYN packet. I think ISS may be a possible option. Can I try with this trick?

[hhaim]

(a) the firewalls defined it to be zero exactly for the reason not to convey hidden Information. You can google it. Even ipv4 option are not allowed More over in case of middle-box proxy the TCP is different only the L7 is the same so you can’t use any information.

(b) This is not the right direction! I’ve been there. The only option is to use the same information normal server uses (L7 data- or range of ip)

Thanks Hanoh

On Fri, 6 Dec 2019 at 4:48 jsmoon notifications@github.com wrote:

@hhaim https://github.com/hhaim (a) I cannot understand the following statement.

'even the ACK number in the SYN can't be used ..'

As far as I know, the ACK number field in the SYN packet (not SYN+ACK) is undefined. So, I don't understand how the firewall can decide to drop the SYN packet by checking the undefined field value. Could you explain the situation?

(b) Since I'm thinking a template should match between client and server emulation program, I'm considering a trick bypassing the normal server application's front behavior. If you don't mind I'll try a trick which adds template information in the SYN packet. I think ISS may be a possible option. Can I try with this trick?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/cisco-system-traffic-generator/trex-core/issues/381?email_source=notifications&email_token=ADDOHPLHP7XU4HYPULOKBMTQXG4QFA5CNFSM4JTQWCB2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGC3AQY#issuecomment-562409539, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADDOHPNIF3RZVL5LWLPMPQLQXG4QFANCNFSM4JTQWCBQ .

-- Hanoh Sent from my iPhone

[jsmoon]

(a) Thank you for your explanation! :-) (b) O.K. I got it.

Could you share your plan? How long will it take?

[hhaim]

@elados93 can work on that but I don't know how long will it take as it will be his first task in ASTF.
if you like to own this task (for better estimate the ETA), I don't see an issue.

[egwakim]

Hi, To able to cover all possible use cases(Client/Server Only mode, TCP/UDP, TCP relay proxy case). We need to add template information on payload.

Explicitly using new API is reasonable approach because in the real general L7 application environment, L7 application also need to have specific L7 command to associate different server behaviors.

For example> If different 2 clients request different pages on the same server like below. Client request will be differ in L7 layer. GET page1.html GET page2.html

If we add new l7_map() API. l7_map(offset,size) : Map L7 payload to specific template

• offset : offset from L4(TCP/UDP) payload(Except IP, TCP, UDP headers)
• size : size of L7 map key ( 0 <= size <= 4 ) - (default : 0 to disable) ASTFAssociationRule(port=80, ip_start=None, ip_end=None, l7_map_offset=0, l7_map_size=0 )

In trex ASTF mode, It'll work as below In Template#1. buf1 = "GET page1.html" ASTFAssociationRule(port=80, ip_start=None, ip_end=None, l7_map_offset=0, l7_map_size=len(buf1) )

In Template#2. buf2 = "GET page2.html" ASTFAssociationRule(port=80, ip_start=None, ip_end=None, l7_map_offset=0, l7_map_size=len(buf2) )

How about your opinion about it?

Regards Gwangmoon

[hhaim]

@egwakim

1. This is aligned with the solution I've described. this is the reason we added the ASTFAssociationRule in the first place. but there is a monitor change
2. In case there are two pcap that already have a different L7 request I wouldn't change the L7

cap 1: GET /3384 HTTP/1.1

cap2:

Host: 22.0.0.2
Connection: Keep-Alive

There is no need to change the L7 right? We change choose byte [6] as a differentiation (offline)

3. it case it is the same, you can ask the user to offset where to put the L7 data that will make a difference

4. the rule could include a offset/value/mask/len_of_request -> template_id

[egwakim]

Closing it. Further improvement will be done separately.