edgecomllc / eupf

5G User Plane Function (UPF) based on eBPF
Apache License 2.0
108 stars 19 forks source link

Verify DSCP marking #310

Closed pirog-spb closed 11 months ago

pirog-spb commented 1 year ago
marknefedov commented 1 year ago

The transport-level marking, executed on per-QoS flow, is based on the mapping from the 5QI and optional Allocation and Retention Policy (ARP) configuration from the SMF. https://www.cisco.com/c/en/us/td/docs/wireless/ucc/upf/Ultra-Cloud-Core-5G-UPF-Config-Guide/Ultra-Cloud-Core-5G-UPF-Config-Guide_chapter_011011.html#:~:text=The%20transport%2Dlevel%20marking%2C%20executed%20on%20per%2DQoS%20flow%2C%20is%20based%20on%20the%20mapping%20from%20the%205QI%20and%20optional%20Allocation%20and%20Retention%20Policy%20(ARP)%20configuration%20from%20the%20SMF.

marknefedov commented 1 year ago

If there are no obvious way to set Transport Level Marking from 5g cores, we can try sending session modification with Far update where we will set value ourselves. Something like we did for initial PFCP testing here: https://github.com/edgecomllc/eupf/blob/main/scripts/scapy/SessionModificationRequest.py

PapaySail commented 1 year ago

Open5GS: OK, there is options for subscriber in web:

5QI/QCI=9 & ARP=8

![image](https://github.com/edgecomllc/eupf/assets/125686203/2d77fff2-3620-4182-8a34-6f7120503671)

Packets GTP capture: DSCP=0

Details

```bash bash-5.1# ping -I uesimtun0 1.1.1.1 -c1 PING 1.1.1.1 (1.1.1.1): 56 data bytes 64 bytes from 1.1.1.1: seq=0 ttl=57 time=1.915 ms --- 1.1.1.1 ping statistics --- 1 packets transmitted, 1 packets received, 0% packet loss round-trip min/avg/max = 1.915/1.915/1.915 ms bash-5.1# curl --interface uesimtun0 https://google.com 301 Moved

301 Moved

The document has moved here. bash-5.1# ``` GTP packets capture filter `port 2152`: [GTP-DSCP0.pcapng.gz](https://github.com/edgecomllc/eupf/files/11873850/GTP-DSCP0.pcapng.gz) ```ruby No. Time Source Destination Protocol Length Differentiated Services Field Info 1 0.000000 10.45.0.2 1.1.1.1 GTP 142 0x00,0x00 Echo (ping) request id=0x005a, seq=0/0, ttl=64 (reply in 2) 2 0.001061 1.1.1.1 10.45.0.2 GTP 134 0x00,0x00 Echo (ping) reply id=0x005a, seq=0/0, ttl=57 (request in 1) 3 5.440600 10.45.0.2 108.177.14.138 GTP 118 0x00,0x00 39988 → 443 [SYN] Seq=0 Win=65280 Len=0 MSS=1360 SACK_PERM TSval=2132510900 TSecr=0 WS=128 4 5.458292 108.177.14.138 10.45.0.2 GTP 110 0x00,0x00 443 → 39988 [SYN, ACK] Seq=0 Ack=1 Win=65535 Len=0 MSS=1412 SACK_PERM TSval=1008538891 TSecr=2132510900 WS=256 5 5.462853 10.45.0.2 108.177.14.138 GTP 110 0x00,0x00 39988 → 443 [ACK] Seq=1 Ack=1 Win=65280 Len=0 TSval=2132510920 TSecr=1008538891 6 5.490122 10.45.0.2 108.177.14.138 GTP 627 0x00,0x00 Client Hello 7 5.507778 108.177.14.138 10.45.0.2 GTP 102 0x00,0x00 443 → 39988 [ACK] Seq=1 Ack=518 Win=66816 Len=0 TSval=1008538941 TSecr=2132510950 8 5.555694 108.177.14.138 10.45.0.2 GTP 151 0x00,0x00 [TCP Previous segment not captured] , Ignored Unknown Record 9 5.556750 10.45.0.2 108.177.14.138 GTP 122 0x00,0x00 [TCP Dup ACK 5#1] 39988 → 443 [ACK] Seq=518 Ack=1 Win=65280 Len=0 TSval=2132511017 TSecr=1008538941 SLE=6741 SRE=6790 10 5.786793 108.177.14.138 10.45.0.2 GTP 1450 0x00,0x00 [TCP Retransmission] 443 → 39988 [ACK] Seq=1 Ack=518 Win=66816 Len=1348 TSval=1008539220 TSecr=2132511017 11 5.788388 10.45.0.2 108.177.14.138 GTP 122 0x00,0x00 39988 → 443 [ACK] Seq=518 Ack=1349 Win=64000 Len=0 TSval=2132511248 TSecr=1008539220 SLE=6741 SRE=6790 12 5.807130 108.177.14.138 10.45.0.2 GTP 1450 0x00,0x00 [TCP Retransmission] 443 → 39988 [ACK] Seq=5393 Ack=518 Win=66816 Len=1348 TSval=1008539241 TSecr=2132511248 13 5.808686 10.45.0.2 108.177.14.138 GTP 122 0x00,0x00 [TCP Dup ACK 11#1] 39988 → 443 [ACK] Seq=518 Ack=1349 Win=64000 Len=0 TSval=2132511268 TSecr=1008539220 SLE=5393 SRE=6790 14 6.034759 108.177.14.138 10.45.0.2 GTP 1450 0x00,0x00 [TCP Retransmission] 443 → 39988 [ACK] Seq=1349 Ack=518 Win=66816 Len=1348 TSval=1008539468 TSecr=2132511268 15 6.036351 10.45.0.2 108.177.14.138 GTP 122 0x00,0x00 39988 → 443 [ACK] Seq=518 Ack=2697 Win=62848 Len=0 TSval=2132511496 TSecr=1008539468 SLE=5393 SRE=6790 16 6.283046 108.177.14.138 10.45.0.2 GTP 1450 0x00,0x00 [TCP Retransmission] 443 → 39988 [ACK] Seq=2697 Ack=518 Win=66816 Len=1348 TSval=1008539716 TSecr=2132511496 17 6.284601 10.45.0.2 108.177.14.138 GTP 122 0x00,0x00 39988 → 443 [ACK] Seq=518 Ack=4045 Win=62848 Len=0 TSval=2132511744 TSecr=1008539716 SLE=5393 SRE=6790 18 6.302215 108.177.14.138 10.45.0.2 GTP 1450 0x00,0x00 [TCP Retransmission] 443 → 39988 [ACK] Seq=4045 Ack=518 Win=66816 Len=1348 TSval=1008539735 TSecr=2132511744 19 6.306285 10.45.0.2 108.177.14.138 GTP 110 0x00,0x00 39988 → 443 [ACK] Seq=518 Ack=6790 Win=63232 Len=0 TSval=2132511766 TSecr=1008539735 20 6.308657 10.45.0.2 108.177.14.138 GTP 190 0x00,0x00 Change Cipher Spec, Application Data 21 6.309133 10.45.0.2 108.177.14.138 GTP 156 0x00,0x00 Application Data 22 6.309446 10.45.0.2 108.177.14.138 GTP 159 0x00,0x00 Application Data 23 6.309603 10.45.0.2 108.177.14.138 GTP 145 0x00,0x00 Application Data 24 6.309931 10.45.0.2 108.177.14.138 GTP 168 0x00,0x00 Application Data 25 6.326447 108.177.14.138 10.45.0.2 GTP 750 0x00,0x00 Application Data, Application Data 26 6.326674 108.177.14.138 10.45.0.2 GTP 102 0x00,0x00 443 → 39988 [ACK] Seq=7438 Ack=693 Win=66816 Len=0 TSval=1008539760 TSecr=2132511769 27 6.326719 108.177.14.138 10.45.0.2 GTP 133 0x00,0x00 Application Data 28 6.327006 108.177.14.138 10.45.0.2 GTP 102 0x00,0x00 443 → 39988 [ACK] Seq=7469 Ack=786 Win=66816 Len=0 TSval=1008539760 TSecr=2132511769 29 6.328868 10.45.0.2 108.177.14.138 GTP 141 0x00,0x00 Application Data 30 6.350682 108.177.14.138 10.45.0.2 GTP 102 0x00,0x00 443 → 39988 [ACK] Seq=7469 Ack=817 Win=66816 Len=0 TSval=1008539784 TSecr=2132511789 31 6.643093 108.177.14.138 10.45.0.2 GTP 521 0x00,0x00 Application Data 32 6.643105 108.177.14.138 10.45.0.2 GTP 353 0x00,0x00 Application Data 33 6.644489 10.45.0.2 108.177.14.138 GTP 110 0x00,0x00 39988 → 443 [ACK] Seq=817 Ack=8139 Win=64256 Len=0 TSval=2132512104 TSecr=1008540076 34 6.644896 10.45.0.2 108.177.14.138 GTP 145 0x00,0x00 Application Data 35 6.645034 108.177.14.138 10.45.0.2 GTP 133 0x00,0x00 Application Data 36 6.645098 108.177.14.138 10.45.0.2 GTP 141 0x00,0x00 Application Data 37 6.645371 10.45.0.2 108.177.14.138 GTP 134 0x00,0x00 Application Data 38 6.647434 10.45.0.2 108.177.14.138 GTP 110 0x00,0x00 39988 → 443 [RST, ACK] Seq=876 Ack=8209 Win=64256 Len=0 TSval=2132512107 TSecr=1008540078 39 6.662141 108.177.14.138 10.45.0.2 GTP 102 0x00,0x00 443 → 39988 [ACK] Seq=8209 Ack=852 Win=66816 Len=0 TSval=1008540095 TSecr=2132512104 40 6.662531 108.177.14.138 10.45.0.2 GTP 102 0x00,0x00 443 → 39988 [ACK] Seq=8209 Ack=876 Win=66816 Len=0 TSval=1008540096 TSecr=2132512105 41 6.662621 108.177.14.138 10.45.0.2 GTP 102 0x00,0x00 443 → 39988 [FIN, ACK] Seq=8209 Ack=876 Win=66816 Len=0 TSval=1008540096 TSecr=2132512105 42 6.663394 10.45.0.2 108.177.14.138 GTP 98 0x00,0x00 39988 → 443 [RST] Seq=852 Win=0 Len=0 43 6.663556 10.45.0.2 108.177.14.138 GTP 98 0x00,0x00 39988 → 443 [RST] Seq=876 Win=0 Len=0 44 6.663671 10.45.0.2 108.177.14.138 GTP 98 0x00,0x00 39988 → 443 [RST] Seq=876 Win=0 Len=0 ```

PapaySail commented 1 year ago

Open5GS: 🚫 changing subscriber's 5QI/QCI=86 & ARP=15 do not lead to another value DSCP of GTP downlink pakets.

There is no transportLevelMarking options in pfcp_sessions nor pfcp_associations

```json curl -X 'GET' \ 'http://localhost:64575/api/v1/pfcp_associations/full' \ -H 'accept: application/json' { "10.233.78.174:8805": { "ID": "10.233.78.174", "Addr": "10.233.78.174:8805", "NextSessionID": 2, "Sessions": { "2": { "LocalSEID": 2, "RemoteSEID": 1, "UplinkPDRs": { "2": { "PdrInfo": { "OuterHeaderRemoval": 0, "FarId": 1, "QerId": 0 }, "Teid": 2, "Ipv4": "", "Ipv6": "" }, "3": { "PdrInfo": { "OuterHeaderRemoval": 0, "FarId": 0, "QerId": 0 }, "Teid": 3, "Ipv4": "", "Ipv6": "" } }, "DownlinkPDRs": { "1": { "PdrInfo": { "OuterHeaderRemoval": 0, "FarId": 0, "QerId": 0 }, "Teid": 0, "Ipv4": "10.45.0.2", "Ipv6": "" } }, "FARs": { "1": { "FarInfo": { "action": 2, "local_ip": "10.233.78.148", "outer_header_creation": 1, "remote_ip": "10.233.78.168", "teid": 1 }, "GlobalId": 0 }, "2": { "FarInfo": { "action": 2, "local_ip": "0.0.0.0", "outer_header_creation": 0, "remote_ip": "0.0.0.0", "teid": 0 }, "GlobalId": 1 }, "3": { "FarInfo": { "action": 2, "local_ip": "10.233.78.148", "outer_header_creation": 1, "remote_ip": "10.233.78.174", "teid": 1 }, "GlobalId": 2 } }, "QERs": { "1": { "QerInfo": { "GateStatusUL": 0, "GateStatusDL": 0, "Qfi": 1, "MaxBitrateUL": 1073741000, "MaxBitrateDL": 1073741000, "StartUL": 0, "StartDL": 0 }, "GlobalId": 0 } } } } }, "10.233.78.190:8805": { "ID": "10.233.78.190", "Addr": "10.233.78.190:8805", "NextSessionID": 3, "Sessions": { "3": { "LocalSEID": 3, "RemoteSEID": 2, "UplinkPDRs": { "2": { "PdrInfo": { "OuterHeaderRemoval": 0, "FarId": 2, "QerId": 1 }, "Teid": 6, "Ipv4": "", "Ipv6": "" }, "3": { "PdrInfo": { "OuterHeaderRemoval": 0, "FarId": 1, "QerId": 1 }, "Teid": 7, "Ipv4": "", "Ipv6": "" } }, "DownlinkPDRs": { "1": { "PdrInfo": { "OuterHeaderRemoval": 0, "FarId": 1, "QerId": 1 }, "Teid": 0, "Ipv4": "10.45.0.3", "Ipv6": "" } }, "FARs": { "1": { "FarInfo": { "action": 2, "local_ip": "10.233.78.148", "outer_header_creation": 1, "remote_ip": "10.233.78.174", "teid": 2 }, "GlobalId": 1 }, "2": { "FarInfo": { "action": 2, "local_ip": "0.0.0.0", "outer_header_creation": 0, "remote_ip": "0.0.0.0", "teid": 0 }, "GlobalId": 2 }, "3": { "FarInfo": { "action": 2, "local_ip": "10.233.78.148", "outer_header_creation": 1, "remote_ip": "10.233.78.190", "teid": 2 }, "GlobalId": 3 } }, "QERs": { "1": { "QerInfo": { "GateStatusUL": 0, "GateStatusDL": 0, "Qfi": 1, "MaxBitrateUL": 1073741000, "MaxBitrateDL": 1073741000, "StartUL": 0, "StartDL": 0 }, "GlobalId": 1 } } } } } } curl -X 'GET' \ 'http://localhost:64575/api/v1/pfcp_sessions?ip=10.45.0.3' \ -H 'accept: application/json' [ { "LocalSEID": 3, "RemoteSEID": 2, "UplinkPDRs": { "2": { "PdrInfo": { "OuterHeaderRemoval": 0, "FarId": 2, "QerId": 1 }, "Teid": 6, "Ipv4": "", "Ipv6": "" }, "3": { "PdrInfo": { "OuterHeaderRemoval": 0, "FarId": 1, "QerId": 1 }, "Teid": 7, "Ipv4": "", "Ipv6": "" } }, "DownlinkPDRs": { "1": { "PdrInfo": { "OuterHeaderRemoval": 0, "FarId": 1, "QerId": 1 }, "Teid": 0, "Ipv4": "10.45.0.3", "Ipv6": "" } }, "FARs": { "1": { "FarInfo": { "action": 2, "local_ip": "10.233.78.148", "outer_header_creation": 1, "remote_ip": "10.233.78.174", "teid": 2 }, "GlobalId": 1 }, "2": { "FarInfo": { "action": 2, "local_ip": "0.0.0.0", "outer_header_creation": 0, "remote_ip": "0.0.0.0", "teid": 0 }, "GlobalId": 2 }, "3": { "FarInfo": { "action": 2, "local_ip": "10.233.78.148", "outer_header_creation": 1, "remote_ip": "10.233.78.190", "teid": 2 }, "GlobalId": 3 } }, "QERs": { "1": { "QerInfo": { "GateStatusUL": 0, "GateStatusDL": 0, "Qfi": 1, "MaxBitrateUL": 1073741000, "MaxBitrateDL": 1073741000, "StartUL": 0, "StartDL": 0 }, "GlobalId": 1 } } } ] ```

fix: #320

PapaySail commented 1 year ago

ℹ QoS table 5QI to DSCP mapping:

5.12. Summary of Recommendations for QCI or 5QI to DSCP Mapping The table below summarizes the 3GPP QCI and 5QI to [RFC4594] DSCP marking recommendations:

QCI/5QI Resource Type Priority Level Example Services Recommended DSCP (PHB)
1 GBR 2 Conversational Voice 44 (VA)
2 GBR 4 Conversational Video 35 (N.A.)
(Live Streaming)
3 GBR 3 Real Time Gaming, 19 (N.A.)
V2X messages,
Electricity
distribution (medium
voltage) Process
automation
(monitoring)
4 GBR 5 Non-Conversational 37 (N.A.)
Video (Buffered
Streaming)
65 GBR 0.7 Mission Critical 42 (N.A.)
user plane Push To
Talk voice (e.g.,
MCPTT)
66 GBR 2 Non-Mission-Critical 43 (N.A.)
user plane Push To
Talk voice
67 GBR 1.5 Mission Critical 33 (N.A.)
Video user plane
75 GBR 2.5 V2X messages 17 (N.A.)
71 GBR 5.6 Live uplink 35 (N.A.)
streaming
72 GBR 5.6 Live uplink 35 (N.A.)
streaming
73 GBR 5.6 Live uplink 35 (N.A.)
streaming
74 GBR 5.6 Live uplink 35 (N.A.)
streaming
76 GBR 5.6 Live uplink 35 (N.A.)
streaming
82 GBR 1.9 Discrete automation 25 (N.A.)
(small packets)
83 GBR 2.2 Discrete automation 27 (N.A.)
(large packets)
84 GBR 2.4 Intelligent 31 (N.A.)
Transport Systems
85 GBR 2.1 Electricity 23 (N.A.)
Distribution - High
Voltage
86 GBR 1.8 Intelligent 29 (N.A.)
Transport Systems
5 Non-GBR 1 IMS Signalling 40 (CS5)
6 Non-GBR 6 Video (Buffered 10 (AF11)
Streaming) TCP-based
(e.g. www, email,
chat, ftp, p2p file
sharing, progressive
video)
7 Non-GBR 7 Voice, Video (live 38 (AF43)
streaming),
interactive gaming
8 Non-GBR 8 Video (buffered 12 (AF12)
streaming) TCP-based
(e.g. www, email,
chat, ftp, p2p file
sharing, progressive
video)
9 Non-GBR 9 Same as 8 14 (AF13)
69 Non-GBR 0.5 Mission Critical 41 (N.A.)
delay sensitive
signalling (e.g.,
MC-PTT signalling,
MC Video signalling)
70 Non-GBR 5.5 Mission Critical 20 (AF22)
Data (e.g. example
services are the
same as QCI 6/8/9)
79 Non-GBR 6.5 V2X messages 21 (N.A.)
80 Non-GBR 6.8 Low latency eMMB 32 (CS4)
applications
(TCP/UDP-based);
augmented reality

Free5GC: ℹ http://forum.free5gc.org/t/qos-enforcement-implemenation-in-upf-gtp/2009/2

Currently, free5GC supports the functionality of QoS in the control plane By the way, the free5GC supports these functions in version 3.3.0.

PapaySail commented 1 year ago

Free 5GC: 🚫 setting subscriber's Default 5QI=1

eUPF log: "TransportLevelMarking:0"

```prolog 2023/06/27 12:16:08 Received 331 bytes from 10.100.50.245:8805 2023/06/27 12:16:08 Handling PFCP message from 10.100.50.245:8805 2023/06/27 12:16:08 Got Session Establishment Request from: 10.100.50.245:8805. 2023/06/27 12:16:08 Session Establishment Request: CreatePDR ID: 7 Outer Header Removal: 0 FAR ID: 7 QER ID: 8 Source Interface: 0 TEID: 7 Ipv4: 10.100.50.233 Ipv6: CreatePDR ID: 8 FAR ID: 8 QER ID: 8 Source Interface: 1 UE IPv4 Address: 10.1.0.3 CreateFAR ID: 7 Apply Action: [2] Forwarding Parameters: Network Instance: internet CreateFAR ID: 8 Apply Action: [2] Forwarding Parameters: CreateQER ID: 8 Gate Status DL: 0 Gate Status UL: 0 QFI: 1 CreateQER ID: 7 Gate Status DL: 0 Gate Status UL: 0 Max Bitrate DL: 100000 Max Bitrate UL: 200000 QFI: 0 2023/06/27 12:16:08 WARN: No OuterHeaderCreation 2023/06/27 12:16:08 Saving FAR info to session: 7, {Action:2 OuterHeaderCreation:0 Teid:0 RemoteIP:0 LocalIP:0 TransportLevelMarking:0} 2023/06/27 12:16:08 EBPF: Put FAR: internalId=5, qerInfo={Action:2 OuterHeaderCreation:0 Teid:0 RemoteIP:0 LocalIP:0 TransportLevelMarking:0} 2023/06/27 12:16:08 WARN: No OuterHeaderCreation 2023/06/27 12:16:08 Saving FAR info to session: 8, {Action:2 OuterHeaderCreation:0 Teid:0 RemoteIP:0 LocalIP:0 TransportLevelMarking:0} 2023/06/27 12:16:08 EBPF: Put FAR: internalId=6, qerInfo={Action:2 OuterHeaderCreation:0 Teid:0 RemoteIP:0 LocalIP:0 TransportLevelMarking:0} 2023/06/27 12:16:08 Saving QER info to session: 8, {GateStatusUL:0 GateStatusDL:0 Qfi:1 MaxBitrateUL:0 MaxBitrateDL:0 StartUL:0 StartDL:0} 2023/06/27 12:16:08 EBPF: Put QER: internalId=5, qerInfo={GateStatusUL:0 GateStatusDL:0 Qfi:1 MaxBitrateUL:0 MaxBitrateDL:0 StartUL:0 StartDL:0} 2023/06/27 12:16:08 Saving QER info to session: 7, {GateStatusUL:0 GateStatusDL:0 Qfi:0 MaxBitrateUL:200000000 MaxBitrateDL:100000000 StartUL:0 StartDL:0} 2023/06/27 12:16:08 EBPF: Put QER: internalId=6, qerInfo={GateStatusUL:0 GateStatusDL:0 Qfi:0 MaxBitrateUL:200000000 MaxBitrateDL:100000000 StartUL:0 StartDL:0} 2023/06/27 12:16:08 EBPF: Put PDR Uplink: teid=7, pdrInfo={OuterHeaderRemoval:0 FarId:5 QerId:5} 2023/06/27 12:16:08 EBPF: Put PDR Downlink: ipv4=10.1.0.3, pdrInfo={OuterHeaderRemoval:0 FarId:6 QerId:5} 2023/06/27 12:16:08 Received 146 bytes from 10.100.50.245:8805 2023/06/27 12:16:08 Handling PFCP message from 10.100.50.245:8805 2023/06/27 12:16:08 Got Session Modification Request from: 10.100.50.245:8805. 2023/06/27 12:16:08 Finding association for 10.100.50.245:8805 2023/06/27 12:16:08 Finding session 5 2023/06/27 12:16:08 Session Modification Request: UpdatePDR ID: 8 FAR ID: 8 Source Interface: 1 UE IPv4 Address: 10.1.0.3 UpdateFAR ID: 8 Apply Action: [2] Update forwarding Parameters: Network Instance: internet Outer Header Creation: &{OuterHeaderCreationDescription:256 TEID:1 IPv4Address:10.100.50.235 IPv6Address: PortNumber:0 CTag:0 STag:0} 2023/06/27 12:16:08 Updating FAR info: 8, {FarInfo:{Action:2 OuterHeaderCreation:1 Teid:1 RemoteIP:3945948170 LocalIP:3912393738 TransportLevelMarking:0} GlobalId:6} 2023/06/27 12:16:08 EBPF: Update FAR: internalId=6, farInfo={Action:2 OuterHeaderCreation:1 Teid:1 RemoteIP:3945948170 LocalIP:3912393738 TransportLevelMarking:0} 2023/06/27 12:16:08 EBPF: Put PDR Downlink: ipv4=10.1.0.3, pdrInfo={OuterHeaderRemoval:0 FarId:6 QerId:5} Stream closed EOF for free5gc2nd/edgecomllc-eupf-universal-chart-7ffc7b6d6d-c9hps (app) ```

PapaySail commented 1 year ago
Installed scapy:

``` sergo@edgecom:~/edgecom/eupf$ sudo apt-get update sergo@edgecom:~/edgecom/eupf$ sudo apt install python3-pip sergo@edgecom:~/edgecom/eupf$ pip install scapy Defaulting to user installation because normal site-packages is not writeable Collecting scapy Downloading scapy-2.5.0.tar.gz (1.3 MB) ---------------------------------------- 1.3/1.3 MB 7.7 MB/s eta 0:00:00 Preparing metadata (setup.py) ... done Building wheels for collected packages: scapy Building wheel for scapy (setup.py) ... done Created wheel for scapy: filename=scapy-2.5.0-py2.py3-none-any.whl size=1444349 sha256=670061cd0d8df8911d9de55952aade30665ec85a9bc3479e8f18dba630d3fdd3 Stored in directory: /home/sergo/.cache/pip/wheels/82/b7/03/8344d8cf6695624746311bc0d389e9d05535ca83c35f90241d Successfully built scapy Installing collected packages: scapy WARNING: The script scapy is installed in '/home/sergo/.local/bin' which is not on PATH. Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location. Successfully installed scapy-2.5.0 sergo@edgecom:~/edgecom/eupf$ sergo@edgecom:~/edgecom/eupf$ sudo pip install scapy Collecting scapy Downloading scapy-2.5.0.tar.gz (1.3 MB) ---------------------------------------- 1.3/1.3 MB 7.8 MB/s eta 0:00:00 Preparing metadata (setup.py) ... done Building wheels for collected packages: scapy Building wheel for scapy (setup.py) ... done Created wheel for scapy: filename=scapy-2.5.0-py2.py3-none-any.whl size=1444349 sha256=14abdb79e198f3080cab3e737c661c525f7aeab259ef873be36953a56685f448 Stored in directory: /root/.cache/pip/wheels/82/b7/03/8344d8cf6695624746311bc0d389e9d05535ca83c35f90241d Successfully built scapy Installing collected packages: scapy Successfully installed scapy-2.5.0 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv sergo@edgecom:~/edgecom/eupf$ ```

@marknefedov 💁‍♂️ Next, what to modify in the SessionModificationRequest.py script to change transport_level_markinng ❓ for:

[
  {
    "LocalSEID": 2,
    "RemoteSEID": 1,
    "UplinkPDRs": {
      "2": {
        "PdrInfo": {
          "OuterHeaderRemoval": 0,
          "FarId": 1,
          "QerId": 0
        },
        "Teid": 2,
        "Ipv4": "",
        "Ipv6": ""
      },
      "3": {
        "PdrInfo": {
          "OuterHeaderRemoval": 0,
          "FarId": 0,
          "QerId": 0
        },
        "Teid": 3,
        "Ipv4": "",
        "Ipv6": ""
      }
    },
    "DownlinkPDRs": {
      "1": {
        "PdrInfo": {
          "OuterHeaderRemoval": 0,
          "FarId": 0,
          "QerId": 0
        },
        "Teid": 0,
        "Ipv4": "10.45.0.2",
        "Ipv6": ""
      }
    },
    "FARs": {
      "1": {
        "FarInfo": {
          "action": 2,
          "local_ip": "10.233.78.148",
          "outer_header_creation": 1,
          "remote_ip": "10.233.78.174",
          "teid": 1,
          "transport_level_markinng": 0
        },
        "GlobalId": 0
      },
      "2": {
        "FarInfo": {
          "action": 2,
          "local_ip": "0.0.0.0",
          "outer_header_creation": 0,
          "remote_ip": "0.0.0.0",
          "teid": 0,
          "transport_level_markinng": 0
        },
        "GlobalId": 1
      },
      "3": {
        "FarInfo": {
          "action": 2,
          "local_ip": "10.233.78.148",
          "outer_header_creation": 1,
          "remote_ip": "10.233.78.190",
          "teid": 1,
          "transport_level_markinng": 0
        },
        "GlobalId": 2
      }
    },
    "QERs": {
      "1": {
        "QerInfo": {
          "GateStatusUL": 0,
          "GateStatusDL": 0,
          "Qfi": 1,
          "MaxBitrateUL": 1073741000,
          "MaxBitrateDL": 1073741000,
          "StartUL": 0,
          "StartDL": 0
        },
        "GlobalId": 0
      }
    }
  }
]
marknefedov commented 1 year ago

Something along those lines.

#!/usr/bin/env python3

from scapy.all import *
from scapy.contrib.pfcp import *
import socket

pfcp_session_modification_no_sdf = PFCP(version=1, S=1, seq=2, seid=0, spare_oct=0) / \
     PFCPSessionModificationRequest(IE_list=[
         IE_UpdateQER(IE_list=[
             IE_QER_Id(id=2), # Change here QER_ID
             IE_TransportLevelMarking(tos=0, traffic_class=0) # Change values to desired
         ]),
         IE_FSEID(v4=1, seid=0xffde7230bf97810a, ipv4="172.18.1.1"), # Change this to corresponding remote session id.
         IE_NodeId(id_type="FQDN", id="BIG-IMPORTANT-CP") # Change this to corresponging NodeID from association.
       ])

# https://stackoverflow.com/questions/41166420/sending-a-packet-over-physical-loopback-in-scapy
conf.L3socket=L3RawSocket

# Send to UPF
target = IP(dst="127.0.0.1")/UDP(sport=33100,dport=8805)

ans = sr1(target/pfcp_session_modification_no_sdf, iface='lo') # Interface to send from
print(ans.show())
PapaySail commented 1 year ago

Something along those lines.

🤷‍♂️ Unsuccessfully tryed by script:

sergo@edgecom:~/edgecom$ sudo python3 transmark-pfcp.py
Begin emission:
Finished sending 1 packets.
.........................................................................................................................................................
.........................................................................................................................................................
.........................................................................................................................................................
.........................................................................................................................................................
.........................................................................................................................................................
.........................................................................................................................................................
.........................................................................................................................................................
..............................................................................................................^C
Received 1181 packets, got 0 answers, remaining 1 packets
Traceback (most recent call last):
  File "/home/sergo/edgecom/transmark-pfcp.py", line 24, in <module>
    print(ans.show())
AttributeError: 'NoneType' object has no attribute 'show'
sergo@edgecom:~/edgecom$
Script

```python #!/usr/bin/env python3 from scapy.all import * from scapy.contrib.pfcp import * import socket pfcp_session_modification_no_sdf = PFCP(version=1, S=1, seq=2, seid=0, spare_oct=0) / \ PFCPSessionModificationRequest(IE_list=[ IE_UpdateQER(IE_list=[ IE_QER_Id(id=1), # Change here QER_ID IE_TransportLevelMarking(tos=1, traffic_class=1) # Change values to desired ]), IE_FSEID(v4=1, seid=1, ipv4="10.233.78.174"), # Change this to corresponding remote session id. IE_NodeId(id_type="FQDN", id="10.233.78.190") # Change this to corresponging NodeID from association. ]) # https://stackoverflow.com/questions/41166420/sending-a-packet-over-physical-loopback-in-scapy conf.L3socket=L3RawSocket # Send to UPF target = IP(dst="10.233.78.148")/UDP(sport=33100,dport=8805) ans = sr1(target/pfcp_session_modification_no_sdf, iface='lo') # Interface to send from print(ans.show()) ``` Pods: ```ruby sergo@edgecom:~/edgecom$ sudo kubectl -n open5gs get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES edgecomllc-eupf-universal-chart-6d6bfb7fd-k86zr 1/1 Running 0 13h 10.233.78.148 edgecom iperf3-847df47bfb-rjq9h 1/1 Running 0 55d 10.233.78.147 edgecom open5gs-amf-fcc76dc5d-z8j8r 1/1 Running 5 (21h ago) 8d 10.233.78.132 edgecom open5gs-ausf-84958b759c-rpsnh 1/1 Running 4 (22h ago) 8d 10.233.78.162 edgecom open5gs-bsf-787bf8649-jxrpg 1/1 Running 0 8d 10.233.78.133 edgecom open5gs-mongodb-6b7cfd5c8d-8l9tx 1/1 Running 0 8d 10.233.78.173 edgecom open5gs-nrf-657b94f57b-fxzlh 1/1 Running 0 8d 10.233.78.144 edgecom open5gs-nssf-589b9f67d9-gggsj 1/1 Running 0 8d 10.233.78.145 edgecom open5gs-pcf-7d49c48d76-95dd9 1/1 Running 6 (22h ago) 8d 10.233.78.149 edgecom open5gs-populate-66d8978cc4-d2wqq 1/1 Running 0 8d 10.233.78.134 edgecom open5gs-smf-bc5db4dd6-kpqd7 1/1 Running 15 (20h ago) 8d 10.233.78.190 edgecom open5gs-udm-5db8664fd7-qw2l9 1/1 Running 4 (22h ago) 8d 10.233.78.185 edgecom open5gs-udr-7cd558cccb-zhkk5 1/1 Running 2 (8d ago) 8d 10.233.78.140 edgecom open5gs-webui-7d5b755b8f-lf6l5 1/1 Running 0 8d 10.233.78.138 edgecom rout4eupf-9m95j 0/1 Completed 0 8d 188.120.232.247 edgecom ueransim-ueransim-gnb-6b846d555d-rbncx 1/1 Running 3 (13h ago) 8d 10.233.78.174 edgecom ueransim-ueransim-gnb-ues-5b9d9c577b-chhxz 1/1 Running 3 (35h ago) 8d 10.233.78.151 edgecom ```

curl -X 'GET' \ 'http://localhost:60335/api/v1/pfcp_associations/full' \ -H 'accept: application/json'

Details

```json { "10.233.78.190:8805": { "ID": "10.233.78.190", "Addr": "10.233.78.190:8805", "NextSessionID": 2, "Sessions": { "2": { "LocalSEID": 2, "RemoteSEID": 1, "UplinkPDRs": { "2": { "PdrInfo": { "OuterHeaderRemoval": 0, "FarId": 1, "QerId": 0 }, "Teid": 2, "Ipv4": "", "Ipv6": "" }, "3": { "PdrInfo": { "OuterHeaderRemoval": 0, "FarId": 0, "QerId": 0 }, "Teid": 3, "Ipv4": "", "Ipv6": "" } }, "DownlinkPDRs": { "1": { "PdrInfo": { "OuterHeaderRemoval": 0, "FarId": 0, "QerId": 0 }, "Teid": 0, "Ipv4": "10.45.0.2", "Ipv6": "" } }, "FARs": { "1": { "FarInfo": { "action": 2, "local_ip": "10.233.78.148", "outer_header_creation": 1, "remote_ip": "10.233.78.174", "teid": 1, "transport_level_markinng": 0 }, "GlobalId": 0 }, "2": { "FarInfo": { "action": 2, "local_ip": "0.0.0.0", "outer_header_creation": 0, "remote_ip": "0.0.0.0", "teid": 0, "transport_level_markinng": 0 }, "GlobalId": 1 }, "3": { "FarInfo": { "action": 2, "local_ip": "10.233.78.148", "outer_header_creation": 1, "remote_ip": "10.233.78.190", "teid": 1, "transport_level_markinng": 0 }, "GlobalId": 2 } }, "QERs": { "1": { "QerInfo": { "GateStatusUL": 0, "GateStatusDL": 0, "Qfi": 1, "MaxBitrateUL": 1073741000, "MaxBitrateDL": 1073741000, "StartUL": 0, "StartDL": 0 }, "GlobalId": 0 } } } } } } ```

@marknefedov Any suggestions?

marknefedov commented 1 year ago

ans = sr1(target/pfcp_session_modification_no_sdf, iface='lo') Change interface, this script should be run from the same pod where SMF is running for IP to match with one in association.

PapaySail commented 1 year ago

Entering to smf container shell as root using containerID :: to install python & scapy

root@edgecom:/home/sergo/kubectl-plugins# runc --root /run/containerd/runc/k8s.io/ exec -t -u 0 0d997ebab7d78e2b9414e9a32d80a6b2d376ee852011a29d7d78e2be42fc1de6 bash
root@open5gs-smf-bc5db4dd6-kpqd7:~# apt-get update
root@open5gs-smf-bc5db4dd6-kpqd7:~# apt install python3 python3-pip
root@open5gs-smf-bc5db4dd6-kpqd7:~# apt install python3-scapy
root@open5gs-smf-bc5db4dd6-kpqd7:~# apt install nano
root@open5gs-smf-bc5db4dd6-kpqd7:~# nano transmark-pfcp.py

transmark-pfcp.py:

#!/usr/bin/env python3

from scapy.all import *
from scapy.contrib.pfcp import *
import socket

pfcp_session_modification_no_sdf = PFCP(version=1, S=1, seq=2, seid=0, spare_oct=0) / \
     PFCPSessionModificationRequest(IE_list=[
         IE_UpdateQER(IE_list=[
             IE_QER_Id(id=1), # Change here QER_ID
             IE_TransportLevelMarking(tos=1, traffic_class=1) # Change values to desired
         ]),
         IE_FSEID(v4=1, seid=1, ipv4="10.233.78.174"), # Change this to corresponding remote session id.
         IE_NodeId(id_type="FQDN", id="10.233.78.190") # Change this to corresponging NodeID from association.
       ])

# https://stackoverflow.com/questions/41166420/sending-a-packet-over-physical-loopback-in-scapy
conf.L3socket=L3RawSocket

# Send to UPF
target = IP(dst="10.233.78.148")/UDP(sport=33100,dport=8805)

ans = sr1(target/pfcp_session_modification_no_sdf, iface='eth0') # Interface to send from
print(ans.show())
✔ RUN : 🛑 But "No established Sx Association"

```powershell root@open5gs-smf-bc5db4dd6-kpqd7:~# python3 transmark-pfcp.py Begin emission: Finished sending 1 packets. * Received 1 packets, got 1 answers, remaining 0 packets ###[ IP ]### version = 4 ihl = 5 tos = 0x0 len = 49 id = 32132 flags = DF frag = 0 ttl = 63 proto = udp chksum = 0xb14 src = 10.233.78.148 dst = 10.233.78.190 \options \ ###[ UDP ]### sport = 8805 dport = 33100 len = 29 chksum = 0xb352 ###[ PFCP (v1) Header ]### version = 1 spare_b2 = 0x0 spare_b3 = 0x0 spare_b4 = 0x0 MP = 0 S = 1 message_type= session_modification_response length = 17 seid = 0x0 seq = 2 spare_oct = 0 ###[ PFCP Session Modification Response ]### \IE_list \ |###[ IE Cause ]### | ietype = Cause | length = 1 | cause = No established Sx Association None root@open5gs-smf-bc5db4dd6-kpqd7:~# ```

📝 We have to send packet from sport=8805 for UDP to match with session. But we can't listen to answer, because this port accupied to listen by smf app. SCAPY Send and receive packets (sr)

The sr() function is for sending packets and receiving answers. The function returns a couple of packet and answers, and the unanswered packets. The function sr1() is a variant that only returns one packet that answered the packet (or the packet set) sent.

So, lets change function to Sending packets only:

The send() function will send packets at layer 3.

PapaySail commented 1 year ago

From NODE shell Sending PFCPSessionModificationRequest. Modifications in script :

✔ RUN :

``` root@edgecom:/home/sergo/edgecom# python3 transmark-pfcp.py Begin emission: WARNING: No broadcast address found for iface cali02e34b11777 Finished sending 1 packets. .* Received 2 packets, got 1 answers, remaining 0 packets ###[ IP ]### version = 4 ihl = 5 tos = 0x0 len = 49 id = 43431 flags = DF frag = 0 ttl = 64 proto = udp chksum = 0xddf0 src = 10.233.78.148 dst = 10.233.78.190 \options \ ###[ UDP ]### sport = 8805 dport = 8805 len = 29 chksum = 0xb352 ###[ PFCP (v1) Header ]### version = 1 spare_b2 = 0x0 spare_b3 = 0x0 spare_b4 = 0x0 MP = 0 S = 1 message_type= session_modification_response length = 17 seid = 0x1 seq = 2 spare_oct = 0 ###[ PFCP Session Modification Response ]### \IE_list \ |###[ IE Cause ]### | ietype = Cause | length = 1 | cause = Request accepted None root@edgecom:/home/sergo/edgecom# ```

eUPF log:

2023/06/29 11:35:27 Received 60 bytes from 10.233.78.190:8805
2023/06/29 11:35:27 Handling PFCP message from 10.233.78.190:8805
2023/06/29 11:35:27 Got Session Modification Request from: 10.233.78.190:8805.
2023/06/29 11:35:27 Finding association for 10.233.78.190:8805
2023/06/29 11:35:27 Finding session 2
2023/06/29 11:35:27
Session Modification Request:
  UpdateQER ID: 1
2023/06/29 11:35:27 Updating QER ID: 1, QER Info: {QerInfo:{GateStatusUL:0 GateStatusDL:0 Qfi:1 MaxBitrateUL:1073741000 MaxBitrateDL:
1073741000 StartUL:0 StartDL:0} GlobalId:0}
2023/06/29 11:35:27 EBPF: Update QER: internalId=0, qerInfo={GateStatusUL:0 GateStatusDL:0 Qfi:1 MaxBitrateUL:1073741000 MaxBitrateDL
:1073741000 StartUL:0 StartDL:0}

🛑 But there is no modification in transport_level_markinng

"FARs": {
          "1": {
            "FarInfo": {
              "action": 2,
              "local_ip": "10.233.78.148",
              "outer_header_creation": 1,
              "remote_ip": "10.233.78.174",
              "teid": 1,
              "transport_level_markinng": 0
            },
            "GlobalId": 0
          },
marknefedov commented 1 year ago

It looks like, I've mixed up FAR and QER. It should be FAR instead of QER. image https://www.etsi.org/deliver/etsi_ts/129200_129299/129244/17.08.00_60/ts_129244v170800p.pdf

marknefedov commented 1 year ago
IE_UpdateFAR(IE_list=[
             IE_FAR_Id(id=1), # Change here FAR_ID
             IE_TransportLevelMarking(tos=1, traffic_class=1) # Change values to desired
         ]),
PapaySail commented 1 year ago

✔ Successfully run PFCPSessionModificationRequest

Script

```python #!/usr/bin/env python3 from scapy.all import * from scapy.contrib.pfcp import * import socket pfcp_session_modification_no_sdf = PFCP(version=1, S=1, seq=2, seid=2, spare_oct=0) / \ PFCPSessionModificationRequest(IE_list=[ IE_UpdateFAR(IE_list=[ IE_FAR_Id(id=1), # Change here FAR_ID IE_TransportLevelMarking(tos=11, traffic_class=1) # Change values to desired ]), IE_UpdateFAR(IE_list=[ IE_FAR_Id(id=2), # Change here FAR_ID IE_TransportLevelMarking(tos=22, traffic_class=2) # Change values to desired ]), IE_UpdateFAR(IE_list=[ IE_FAR_Id(id=3), # Change here FAR_ID IE_TransportLevelMarking(tos=33, traffic_class=3) # Change values to desired ]), IE_FSEID(v4=1, seid=1, ipv4="10.233.78.174"), # Change this to corresponding remote session id. IE_NodeId(id_type="IPv4", id="10.233.78.190") # Change this to corresponging NodeID from association. ]) # https://stackoverflow.com/questions/41166420/sending-a-packet-over-physical-loopback-in-scapy #conf.L3socket=L3RawSocket # Send to UPF target = IP(src="10.233.78.190", dst="10.233.78.148")/UDP(sport=8805,dport=8805) ans = sr1(target/pfcp_session_modification_no_sdf, iface='calia05c7827415') # Interface to send from print(ans.show()) ```

###[ PFCP Session Modification Response ]###
           \IE_list   \
            |###[ IE Cause ]###
            |  ietype    = Cause
            |  length    = 1
            |  cause     = Request accepted

🐛TransportLevelMarking:0

eUPF log

```prolog 2023/06/29 16:46:17 Got Session Modification Request from: 10.233.78.190:8805. 2023/06/29 16:46:17 Finding association for 10.233.78.190:8805 2023/06/29 16:46:17 Finding session 2 2023/06/29 16:46:17 Session Modification Request: UpdateFAR ID: 1 UpdateFAR ID: 2 UpdateFAR ID: 3 2023/06/29 16:46:17 Updating FAR info: 1, {FarInfo:{Action:2 OuterHeaderCreation:1 Teid:1 RemoteIP:2924407050 LocalIP:2488199434 TransportLevelMarking:0} GlobalId:3} 2023/06/29 16:46:17 EBPF: Update FAR: internalId=3, farInfo={Action:2 OuterHeaderCreation:1 Teid:1 RemoteIP:2924407050 LocalIP:2488199434 TransportLevelMarking:0} 2023/06/29 16:46:17 Updating FAR info: 2, {FarInfo:{Action:2 OuterHeaderCreation:0 Teid:0 RemoteIP:0 LocalIP:0 TransportLevelMarking:0} GlobalId:4} 2023/06/29 16:46:17 EBPF: Update FAR: internalId=4, farInfo={Action:2 OuterHeaderCreation:0 Teid:0 RemoteIP:0 LocalIP:0 TransportLevelMarking:0} 2023/06/29 16:46:17 Updating FAR info: 3, {FarInfo:{Action:2 OuterHeaderCreation:1 Teid:1 RemoteIP:3192842506 LocalIP:2488199434 TransportLevelMarking:0} GlobalId:5} 2023/06/29 16:46:17 EBPF: Update FAR: internalId=5, farInfo={Action:2 OuterHeaderCreation:1 Teid:1 RemoteIP:3192842506 LocalIP:2488199434 TransportLevelMarking:0} ```

http://localhost:60335/api/v1/pfcp_associations/full

```json { "10.233.78.190:8805": { "ID": "10.233.78.190", "Addr": "10.233.78.190:8805", "NextSessionID": 2, "Sessions": { "2": { "LocalSEID": 2, "RemoteSEID": 1, "UplinkPDRs": { "2": { "PdrInfo": { "OuterHeaderRemoval": 0, "FarId": 4, "QerId": 1 }, "Teid": 2, "Ipv4": "", "Ipv6": "" }, "3": { "PdrInfo": { "OuterHeaderRemoval": 0, "FarId": 3, "QerId": 1 }, "Teid": 3, "Ipv4": "", "Ipv6": "" } }, "DownlinkPDRs": { "1": { "PdrInfo": { "OuterHeaderRemoval": 0, "FarId": 3, "QerId": 1 }, "Teid": 0, "Ipv4": "10.45.0.2", "Ipv6": "" } }, "FARs": { "1": { "FarInfo": { "action": 2, "local_ip": "10.233.78.148", "outer_header_creation": 1, "remote_ip": "10.233.78.174", "teid": 1, "transport_level_markinng": 0 }, "GlobalId": 3 }, "2": { "FarInfo": { "action": 2, "local_ip": "0.0.0.0", "outer_header_creation": 0, "remote_ip": "0.0.0.0", "teid": 0, "transport_level_markinng": 0 }, "GlobalId": 4 }, "3": { "FarInfo": { "action": 2, "local_ip": "10.233.78.148", "outer_header_creation": 1, "remote_ip": "10.233.78.190", "teid": 1, "transport_level_markinng": 0 }, "GlobalId": 5 } }, "QERs": { "1": { "QerInfo": { "GateStatusUL": 0, "GateStatusDL": 0, "Qfi": 1, "MaxBitrateUL": 1073741000, "MaxBitrateDL": 1073741000, "StartUL": 0, "StartDL": 0 }, "GlobalId": 1 } } } } } } ```

PapaySail commented 1 year ago

Verify implementation in tests : result

❌ Not passed. eUPF don't take into account an FAR::TransportLevelMarking value from PFCP Session Modification Request.

📝 Additional info:

captured PFCPSessionModificationRequest packet generated by script

```ruby No. Time Source Destination Protocol Length Differentiated Services Field ToS Traffic Class Mask field Info 1751 0.036823 10.233.78.190 10.233.78.148 PFCP 144 0x00 0x0b,0x16,0x21 0x01,0x02,0x03 PFCP Session Modification Request Frame 1751: 144 bytes on wire (1152 bits), 144 bytes captured (1152 bits) on interface -, id 0 Linux cooked capture v2 Internet Protocol Version 4, Src: 10.233.78.190, Dst: 10.233.78.148 User Datagram Protocol, Src Port: 8805, Dst Port: 8805 Packet Forwarding Control Protocol Flags: 0x21, SEID (S) Message Type: PFCP Session Modification Request (52) Length: 92 SEID: 0x0000000000000002 Sequence Number: 2 Spare: 0 Update FAR : [Grouped IE]: FAR ID: Dynamic by CP 1 IE Type: Update FAR (10) IE Length: 14 FAR ID : Dynamic by CP 1 Transport Level Marking : IE Type: Transport Level Marking (30) IE Length: 2 ToS Traffic Class: 0x0b Mask field: 0x01 [DSCP: Default] Update FAR : [Grouped IE]: FAR ID: Dynamic by CP 2 IE Type: Update FAR (10) IE Length: 14 FAR ID : Dynamic by CP 2 Transport Level Marking : IE Type: Transport Level Marking (30) IE Length: 2 ToS Traffic Class: 0x16 Mask field: 0x02 [DSCP: Default] Update FAR : [Grouped IE]: FAR ID: Dynamic by CP 3 IE Type: Update FAR (10) IE Length: 14 FAR ID : Dynamic by CP 3 Transport Level Marking : IE Type: Transport Level Marking (30) IE Length: 2 ToS Traffic Class: 0x21 Mask field: 0x03 [DSCP: Default] F-SEID : SEID: 0x0000000000000001, IPv4 10.233.78.174 Node ID : IPv4 address: 0.0.0.0 [Response In: 1752] No. Time Source Destination Protocol Length Differentiated Services Field ToS Traffic Class Mask field Info 1752 0.000576 10.233.78.148 10.233.78.190 PFCP 69 0x00 PFCP Session Modification Response Frame 1752: 69 bytes on wire (552 bits), 69 bytes captured (552 bits) on interface -, id 0 Linux cooked capture v2 Internet Protocol Version 4, Src: 10.233.78.148, Dst: 10.233.78.190 User Datagram Protocol, Src Port: 8805, Dst Port: 8805 Packet Forwarding Control Protocol Flags: 0x21, SEID (S) Message Type: PFCP Session Modification Response (53) Length: 17 SEID: 0x0000000000000001 Sequence Number: 2 Spare: 0 Cause : Request accepted(success) [Response To: 1751] [Response Time: 0.000576000 seconds] ```

Next packets captured from eUPF interface after command nr-cli imsi-999700000000001 -e "deregister normal" at UE:

Open5gs SMF generated :: Message Type: PFCP Session Deletion Request (54), Message Type: PFCP Session Establishment Request (50), Message Type: PFCP Session Modification Request (52)

```ruby No. Time Source Destination Protocol Length Differentiated Services Field ToS Traffic Class Mask field Info 3743 1.553082 10.233.78.190 10.233.78.148 PFCP 64 0x00 PFCP Session Deletion Request Frame 3743: 64 bytes on wire (512 bits), 64 bytes captured (512 bits) on interface -, id 0 Linux cooked capture v2 Internet Protocol Version 4, Src: 10.233.78.190, Dst: 10.233.78.148 User Datagram Protocol, Src Port: 8805, Dst Port: 8805 Packet Forwarding Control Protocol Flags: 0x21, SEID (S) Message Type: PFCP Session Deletion Request (54) Length: 12 SEID: 0x0000000000000002 Sequence Number: 2173 Spare: 0 [Response In: 3744] No. Time Source Destination Protocol Length Differentiated Services Field ToS Traffic Class Mask field Info 3744 0.000940 10.233.78.148 10.233.78.190 PFCP 69 0x00 PFCP Session Deletion Response Frame 3744: 69 bytes on wire (552 bits), 69 bytes captured (552 bits) on interface -, id 0 Linux cooked capture v2 Internet Protocol Version 4, Src: 10.233.78.148, Dst: 10.233.78.190 User Datagram Protocol, Src Port: 8805, Dst Port: 8805 Packet Forwarding Control Protocol Flags: 0x21, SEID (S) Message Type: PFCP Session Deletion Response (55) Length: 17 SEID: 0x0000000000000001 Sequence Number: 2173 Spare: 0 Cause : Request accepted(success) [Response To: 3743] [Response Time: 0.000940000 seconds] No. Time Source Destination Protocol Length Differentiated Services Field ToS Traffic Class Mask field Info 3745 0.484025 10.233.78.190 10.233.78.148 PFCP 651 0x00 PFCP Session Establishment Request Frame 3745: 651 bytes on wire (5208 bits), 651 bytes captured (5208 bits) on interface -, id 0 Linux cooked capture v2 Internet Protocol Version 4, Src: 10.233.78.190, Dst: 10.233.78.148 User Datagram Protocol, Src Port: 8805, Dst Port: 8805 Packet Forwarding Control Protocol Flags: 0x21, SEID (S) Message Type: PFCP Session Establishment Request (50) Length: 599 SEID: 0x0000000000000000 Sequence Number: 2174 Spare: 0 Node ID : IPv4 address: 10.233.78.190 F-SEID : SEID: 0x0000000000000002, IPv4 10.233.78.190 Create PDR : [Grouped IE]: PDR ID: 1 Create PDR : [Grouped IE]: PDR ID: 2 Create PDR : [Grouped IE]: PDR ID: 3 Create PDR : [Grouped IE]: PDR ID: 4 Create FAR : [Grouped IE]: FAR ID: Dynamic by CP 1 IE Type: Create FAR (3) IE Length: 19 FAR ID : Dynamic by CP 1 Apply Action : BAR ID : 1 Create FAR : [Grouped IE]: FAR ID: Dynamic by CP 2 IE Type: Create FAR (3) IE Length: 36 FAR ID : Dynamic by CP 2 Apply Action : Forwarding Parameters : [Grouped IE] IE Type: Forwarding Parameters (4) IE Length: 18 Destination Interface : Core Network Instance : internet Create FAR : [Grouped IE]: FAR ID: Dynamic by CP 3 IE Type: Create FAR (3) IE Length: 50 FAR ID : Dynamic by CP 3 Apply Action : Forwarding Parameters : [Grouped IE] IE Type: Forwarding Parameters (4) IE Length: 32 Destination Interface : CP- Function Network Instance : internet Outer Header Creation : Create URR : [Grouped IE]: URR ID: Dynamic by CP 1 Create QER : [Grouped IE]: QER ID: Dynamic by CP 1 Create BAR : [Grouped IE]: BAR ID: 1 PDN Type : IPv4 [Response In: 3746] No. Time Source Destination Protocol Length Differentiated Services Field ToS Traffic Class Mask field Info 3746 0.001017 10.233.78.148 10.233.78.190 PFCP 95 0x00 PFCP Session Establishment Response Frame 3746: 95 bytes on wire (760 bits), 95 bytes captured (760 bits) on interface -, id 0 Linux cooked capture v2 Internet Protocol Version 4, Src: 10.233.78.148, Dst: 10.233.78.190 User Datagram Protocol, Src Port: 8805, Dst Port: 8805 Packet Forwarding Control Protocol Flags: 0x21, SEID (S) Message Type: PFCP Session Establishment Response (51) Length: 43 SEID: 0x0000000000000002 Sequence Number: 2174 Spare: 0 Node ID : IPv4 address: 10.233.78.148 Cause : Request accepted(success) F-SEID : SEID: 0x0000000000000003, IPv4 10.233.78.148 [Response To: 3745] [Response Time: 0.001017000 seconds] No. Time Source Destination Protocol Length Differentiated Services Field ToS Traffic Class Mask field Info 3747 0.013187 10.233.78.190 10.233.78.148 PFCP 118 0x00 PFCP Session Modification Request Frame 3747: 118 bytes on wire (944 bits), 118 bytes captured (944 bits) on interface -, id 0 Linux cooked capture v2 Internet Protocol Version 4, Src: 10.233.78.190, Dst: 10.233.78.148 User Datagram Protocol, Src Port: 8805, Dst Port: 8805 Packet Forwarding Control Protocol Flags: 0x21, SEID (S) Message Type: PFCP Session Modification Request (52) Length: 66 SEID: 0x0000000000000003 Sequence Number: 2175 Spare: 0 Update FAR : [Grouped IE]: FAR ID: Dynamic by CP 1 IE Type: Update FAR (10) IE Length: 50 FAR ID : Dynamic by CP 1 Apply Action : Update Forwarding Parameters : [Grouped IE] IE Type: Update Forwarding Parameters (11) IE Length: 32 Destination Interface : Access Network Instance : internet Outer Header Creation : [Response In: 3748] No. Time Source Destination Protocol Length Differentiated Services Field ToS Traffic Class Mask field Info 3748 0.000446 10.233.78.148 10.233.78.190 PFCP 69 0x00 PFCP Session Modification Response Frame 3748: 69 bytes on wire (552 bits), 69 bytes captured (552 bits) on interface -, id 0 Linux cooked capture v2 Internet Protocol Version 4, Src: 10.233.78.148, Dst: 10.233.78.190 User Datagram Protocol, Src Port: 8805, Dst Port: 8805 Packet Forwarding Control Protocol Flags: 0x21, SEID (S) Message Type: PFCP Session Modification Response (53) Length: 17 SEID: 0x0000000000000002 Sequence Number: 2175 Spare: 0 Cause : Request accepted(success) [Response To: 3747] [Response Time: 0.000446000 seconds] ```

All the packets in single file:
PFCPSessionModificationRequest-script+open5gs.pcapng.gz