fiorix / go-diameter

Diameter stack and Base Protocol (RFC 6733) for the Go programming language
Other
252 stars 143 forks source link

Unsupported mandatory AVP found in Base package #75

Closed muxalko closed 7 years ago

muxalko commented 7 years ago

Hello Fiorix!

I am trying to make GoDiameter to communicate with freeDiameter respectively. However, received CEA includes three application Ids and one unknown AVP. Despite _DIAMETERSUCCESS (2001) , freeDiameter fails to parse the response correctly.

freeDiameter Log:

18:02:54  NOTI   SND to 'cgrates-poc.opencloud.com':
18:02:54  NOTI        'Capabilities-Exchange-Request'
18:02:54  NOTI          Version: 0x01
18:02:54  NOTI          Length: 212
18:02:54  NOTI          Flags: 0x80 (R---)
18:02:54  NOTI          Command Code: 257
18:02:54  NOTI          ApplicationId: 0
18:02:54  NOTI          Hop-by-Hop Identifier: 0x1BFA6EE1
18:02:54  NOTI          End-to-End Identifier: 0x8E10F059
18:02:54  NOTI           {internal data}: src:(nil)(0) rwb:(nil) rt:0 cb:(nil),(nil)((nil)) qry:(nil) asso:0 sess:(nil)
18:02:54  NOTI           AVP: 'Origin-Host'(264) l=34 f=-M val="freediameter.opencloud.com"
18:02:54  NOTI           AVP: 'Origin-Realm'(296) l=21 f=-M val="opencloud.com"
18:02:54  NOTI           AVP: 'Origin-State-Id'(278) l=12 f=-M val=1492441313 (0x58f4d8e1)
18:02:54  NOTI           AVP: 'Host-IP-Address'(257) l=14 f=-M val=10.224.228.36
18:02:54  NOTI           AVP: 'Vendor-Id'(266) l=12 f=-M val=0 (0x0)
18:02:54  NOTI           AVP: 'Product-Name'(269) l=20 f=-- val="freeDiameter"
18:02:54  NOTI           AVP: 'Firmware-Revision'(267) l=12 f=-- val=10201 (0x27d9)
18:02:54  NOTI           AVP: 'Inband-Security-Id'(299) l=12 f=-M val='NO_INBAND_SECURITY' (0 (0x0))
18:02:54  NOTI           AVP: 'Acct-Application-Id'(259) l=12 f=-M val=3 (0x3)
18:02:54  NOTI           AVP: 'Auth-Application-Id'(258) l=12 f=-M val=4294967295 (0xffffffff)
18:02:54  NOTI           AVP: 'Supported-Vendor-Id'(265) l=12 f=-M val=5535 (0x159f)
18:02:54  NOTI           AVP: 'Supported-Vendor-Id'(265) l=12 f=-M val=10415 (0x28af)
18:02:54   DBG   'STATE_WAITCNXACK'     -> 'STATE_WAITCEA'      'cgrates-poc.opencloud.com'
18:02:54  NOTI   Unsupported mandatory AVP found
18:02:54  ERROR  Unsupported AVP: 0(not searched in dictionary) l=12 f=-M val=(not set)
18:02:54  ERROR  ERROR: in '((parsedict_do_avp(dict, ((struct avp *)(avpch->o)), mandatory, error_info)))' :    Operation not supported
18:02:54  ERROR  ERROR: in '((parsedict_do_avp(dict, ((struct avp *)(avpch->o)), mandatory, error_info)))' :    Operation not supported
18:02:54  NOTI   RCV from 'cgrates-poc.opencloud.com':
18:02:54  NOTI        'Capabilities-Exchange-Answer'
18:02:54  NOTI          Version: 0x01
18:02:54  NOTI          Length: 228
18:02:54  NOTI          Flags: 0x00 (----)
18:02:54  NOTI          Command Code: 257
18:02:54  NOTI          ApplicationId: 0
18:02:54  NOTI          Hop-by-Hop Identifier: 0x1BFA6EE1
18:02:54  NOTI          End-to-End Identifier: 0x8E10F059
18:02:54  NOTI           {internal data}: src:cgrates-poc.opencloud.com(25) rwb:0x7f8ffc0019f0 rt:0 cb:(nil),(nil)((nil)) qry:0x7f90040012f0 asso:0 sess:(nil)
18:02:54  NOTI           AVP: 'Result-Code'(268) l=12 f=-M val='DIAMETER_SUCCESS' (2001 (0x7d1))
18:02:54  NOTI           AVP: 'Origin-Host'(264) l=19 f=-M val="cgrates-poc"
18:02:54  NOTI           AVP: 'Origin-Realm'(296) l=21 f=-M val="opencloud.com"
18:02:54  NOTI           AVP: 'Host-IP-Address'(257) l=14 f=-M val=10.224.228.186
18:02:54  NOTI           AVP: 'Vendor-Id'(266) l=12 f=-M val=0 (0x0)
18:02:54  NOTI           AVP: 'Product-Name'(269) l=19 f=-- val="Golant-data"
18:02:54  NOTI           AVP: 'Origin-State-Id'(278) l=12 f=-M val=1492441313 (0x58f4d8e1)
18:02:54  NOTI           AVP: 'Acct-Application-Id'(259) l=12 f=-M val=3 (0x3)
18:02:54  NOTI           AVP: 'Auth-Application-Id'(258) l=12 f=-M val=4 (0x4)
18:02:54  NOTI           AVP: 'Auth-Application-Id'(258) l=12 f=-M val=1 (0x1)
18:02:54  NOTI           AVP: 'Supported-Vendor-Id'(265) l=12 f=-M val=10415 (0x28af)
18:02:54  NOTI           AVP: 'Vendor-Specific-Application-Id'(260) l=32 f=-M val=(grouped)
18:02:54  NOTI              AVP: 'Vendor-Id'(266) l=12 f=-M val=10415 (0x28af)
18:02:54  NOTI              AVP: 0(not searched in dictionary) l=12 f=-M val=(not set)
18:02:54  NOTI           AVP: 267(not searched in dictionary) l=12 f=-- val=(not set)
18:02:54  NOTI   Unsupported mandatory AVP found

Trace:

Diameter Protocol
    Version: 0x01
    Length: 196
    Flags: 0x00
    Command Code: 257 Capabilities-Exchange
    ApplicationId: Diameter Common Messages (0)
    Hop-by-Hop Identifier: 0x5ff41e87
    End-to-End Identifier: 0xc68a0593
    [Request In: 124]
    [Response Time: 0.000189000 seconds]
    AVP: Result-Code(268) l=12 f=-M- val=DIAMETER_SUCCESS (2001)
    AVP: Origin-Host(264) l=19 f=-M- val=cgrates-poc
    AVP: Origin-Realm(296) l=21 f=-M- val=opencloud.com
    AVP: Host-IP-Address(257) l=14 f=-M- val=10.224.228.186
    AVP: Vendor-Id(266) l=12 f=-M- val=10415
    AVP: Product-Name(269) l=19 f=--- val=Golant-data
    AVP: Origin-State-Id(278) l=12 f=-M- val=1492106344
    AVP: Acct-Application-Id(259) l=12 f=-M- val=Diameter Base Accounting (3)
    AVP: Auth-Application-Id(258) l=12 f=-M- val=Diameter Credit Control Application (4)
    AVP: Auth-Application-Id(258) l=12 f=-M- val=NASREQ Application (1)
    **AVP: Unknown(0) l=12 f=-M- val=00000004**
    AVP: Firmware-Revision(267) l=12 f=--- val=918

From the documentation, I understand that actually four dictionaries (applications) are pre-loaded in.

Embedded dictionaries:

- Base Protocol RFC 6733
- Credit Control RFC 4006
- Network Access Server RFC 7155
- 3GPP specific AVPs from TS 32.299 version 12.7.0  <=== What is that exactly and what is used for ?

I am a newbie in diameter world, but I feel, it could be related to to RFC3588 and "Vendor-Specific-Application-Id" grouped AVP. Could you kindly point me towards the reason why that particular unknown AVP appears in the answer and which dictionary could possibly make freeDiameter understand that?

Is there a way to disable those specific AVPs from being added to CEA, since I only need DCCA?

Just tried to comment out the following in the autogen.sh file, and running it, then building the project.


// Default is a Parser object with pre-loaded
// Base Protocol and Credit Control dictionaries.
var Default *Parser

func init() {
        Default, _ = NewParser()
//      Default.Load(bytes.NewReader([]byte(baseXML)))
        Default.Load(bytes.NewReader([]byte(creditcontrolXML)))
//      Default.Load(bytes.NewReader([]byte(networkaccessserverXML)))
//      Default.Load(bytes.NewReader([]byte(tgpprorfXML)))
}

The AVP is removed, but freeDiameter rejects the session without base dictionary, so, I think it is mandatory. This is more like a question, still not sure where is the exact issue.

Please assist, Thanks in advance,

jaroszan commented 7 years ago

Can you please describe what exactly are you trying to advertise from freeDiameter? Which set of vendors and applications? I am not familiar with freeDiameter, but there seems to be some error records in the log for outgoing CER.

Does wireshark decode correctly the CER that is sent to go-diameter peer?

muxalko commented 7 years ago

Hi, My goal is to get DCCA application running between GGSN and OCS for the data charging. Because of SCTP requirement, we were forced to put some translation node in between. For that purpose, we decided to go with freeDiameter, which has Linux Kernel SCTP support built-in.

The topology is as follows:

    go-diameter              freeDiameter                  GGSN
   +----------+              +---------+              +-----------+
   |          |<----TCP----->|+-------+|<----SCTP---->|    DCCA   |
   |    OCS   |              || Relay ||              |           |
   |          |              || Agent ||              |           |
   +----------+              |+-------+|              +-----------+
                             +---------+

Wireshark cannot decode the answer :

Diameter Protocol
    Version: 0x01
    Length: 228
    Flags: 0x00
    Command Code: 257 Capabilities-Exchange
    ApplicationId: Diameter Common Messages (0)
    Hop-by-Hop Identifier: 0x18e9eaeb
    End-to-End Identifier: 0x202c9cdb
    [Request In: 1541]
    [Response Time: 0.000229000 seconds]
    AVP: Result-Code(268) l=12 f=-M- val=DIAMETER_SUCCESS (2001)
    AVP: Origin-Host(264) l=19 f=-M- val=cgrates-poc
    AVP: Origin-Realm(296) l=21 f=-M- val=opencloud.com
    AVP: Host-IP-Address(257) l=14 f=-M- val=10.224.228.186
    AVP: Vendor-Id(266) l=12 f=-M- val=0
    AVP: Product-Name(269) l=19 f=--- val=Golant-data
    AVP: Origin-State-Id(278) l=12 f=-M- val=1492513282
    AVP: Acct-Application-Id(259) l=12 f=-M- val=Diameter Base Accounting (3)
    AVP: Auth-Application-Id(258) l=12 f=-M- val=Diameter Credit Control Application (4)
    AVP: Auth-Application-Id(258) l=12 f=-M- val=NASREQ Application (1)
    AVP: Supported-Vendor-Id(265) l=12 f=-M- val=10415
    AVP: Vendor-Specific-Application-Id(260) l=32 f=-M-
        AVP Code: 260 Vendor-Specific-Application-Id
        AVP Flags: 0x40
        AVP Length: 32
        Vendor-Specific-Application-Id: 0000010a4000000c000028af000000004000000c00000004
            AVP: Vendor-Id(266) l=12 f=-M- val=10415
            AVP: Unknown(0) l=12 f=-M- val=00000004
                AVP Code: 0 Unknown
                    Unknown AVP 0 (vendor=Reserved), if you know what this is you can add it to dictionary.xml
                        [Expert Info (Warning/Undecoded): Unknown AVP 0 (vendor=Reserved), if you know what this is you can add it to dictionary.xml]
                            [Unknown AVP 0 (vendor=Reserved), if you know what this is you can add it to dictionary.xml]
                            [Severity level: Warning]
                            [Group: Undecoded]
                AVP Flags: 0x40
                    0... .... = Vendor-Specific: Not set
                    .1.. .... = Mandatory: Set
                    ..0. .... = Protected: Not set
                    ...0 .... = Reserved: Not set
                    .... 0... = Reserved: Not set
                    .... .0.. = Reserved: Not set
                    .... ..0. = Reserved: Not set
                    .... ...0 = Reserved: Not set
                AVP Length: 12
                Value: 00000004
    AVP: Firmware-Revision(267) l=12 f=--- val=918

Well, after scratching my head, I just tried to eliminate freeDiameter and communicate directly with GGSN over TCP. And the result was similar. So, the issue, IMHO, in configuration/dictionaries at the OCS side.

Here is the trace :

Diameter Protocol
    Version: 0x01
    Length: 216
    Flags: 0x80, Request
    Command Code: 257 Capabilities-Exchange
    ApplicationId: Diameter Common Messages (0)
    Hop-by-Hop Identifier: 0x120f83ab
    End-to-End Identifier: 0x1c02639b
    [Answer In: 2]
    AVP: Origin-Host(264) l=53 f=-M- val=GatewayService-6-11-1.PTggsn.golantelecom.net
    AVP: Origin-Realm(296) l=24 f=-M- val=golantelecom.net
    AVP: Host-IP-Address(257) l=14 f=-M- val=10.224.234.232
    AVP: Vendor-Id(266) l=12 f=-M- val=28458
    AVP: Product-Name(269) l=16 f=--- val=FLEXI NG
    AVP: Origin-State-Id(278) l=12 f=-M- val=15
    AVP: Supported-Vendor-Id(265) l=12 f=-M- val=10415
    AVP: Supported-Vendor-Id(265) l=12 f=-M- val=28458
    AVP: Supported-Vendor-Id(265) l=12 f=-M- val=94
    AVP: Auth-Application-Id(258) l=12 f=-M- val=Diameter Credit Control Application (4)
    AVP: Firmware-Revision(267) l=12 f=--- val=1

Diameter Protocol
    Version: 0x01
    Length: 228
    Flags: 0x00
    Command Code: 257 Capabilities-Exchange
    ApplicationId: Diameter Common Messages (0)
    Hop-by-Hop Identifier: 0x120f83ab
    End-to-End Identifier: 0x1c02639b
    [Request In: 1]
    [Response Time: 0.000106000 seconds]
    AVP: Result-Code(268) l=12 f=-M- val=DIAMETER_SUCCESS (2001)
    AVP: Origin-Host(264) l=19 f=-M- val=cgrates-poc
    AVP: Origin-Realm(296) l=21 f=-M- val=opencloud.com
    AVP: Host-IP-Address(257) l=14 f=-M- val=10.224.228.36
    AVP: Vendor-Id(266) l=12 f=-M- val=0
    AVP: Product-Name(269) l=19 f=--- val=Golant-data
    AVP: Origin-State-Id(278) l=12 f=-M- val=15
    AVP: Acct-Application-Id(259) l=12 f=-M- val=Diameter Base Accounting (3)
    AVP: Auth-Application-Id(258) l=12 f=-M- val=Diameter Credit Control Application (4)
    AVP: Auth-Application-Id(258) l=12 f=-M- val=NASREQ Application (1)
    AVP: Supported-Vendor-Id(265) l=12 f=-M- val=10415
    AVP: Vendor-Specific-Application-Id(260) l=32 f=-M-
        AVP Code: 260 Vendor-Specific-Application-Id
        AVP Flags: 0x40
        AVP Length: 32
        Vendor-Specific-Application-Id: 0000010a4000000c000028af000000004000000c00000004
            AVP: Vendor-Id(266) l=12 f=-M- val=10415
                AVP Code: 266 Vendor-Id
                AVP Flags: 0x40
                AVP Length: 12
                Vendor-Id: 10415
                VendorId: 3GPP (10415)
            AVP: Unknown(0) l=12 f=-M- val=00000004
                AVP Code: 0 Unknown
                    Unknown AVP 0 (vendor=Reserved), if you know what this is you can add it to dictionary.xml
                AVP Flags: 0x40
                AVP Length: 12
                Value: 00000004
    AVP: Firmware-Revision(267) l=12 f=--- val=918

We had opened issue at our GGSN Vendor Support (Nokia Fllexi NG), and here is their answer:

1) The OCS Diameter server is including AVPs Auth-Application-Id (258) and Acct-Application-Id (259) in the CEA, while actually the Flexi NG Diameter Base Protocol Interface Description, sections 5.2 Base protocol AVPs and 5.4 Capabilities-Exchange-Answer, specifies that Flexi NG does not support those in received CEA messages, so OCS should not include them;

2) The OCS Diameter server is including AVP Unknown (0) inside AVP Vendor-Specific-Application-Id (260) (highlighted on the snapshot you sent), while the correct AVP to be included is Auth-Application-Id (258), so even though according to the above mentioned document sections this AVP is ignored by Flexi NG on received CEA, it can still cause CEA processing to failing, so either AVP Vendor-Specific-Application-Id (260) should not be included or its content should be fixed and proper AVP Auth-Application-Id (258) included instead.

I am loading only 3gpp_dcca dictionary, and didn't expect to see some additional custom applications but DCCA. Why is this grouped AVP (Vendor-Specific-Application-Id) does appear in the answer ? Perhaps, I miss something very basic here. How do I get rid of those AVPs ?

Thanks in advance, Michael

jaroszan commented 7 years ago

RFC allows diameter server to advertise all supported applications so it is ok that you receive more than you ask.

Can you please paste here the code which is responsible for capabilities exchange on the go-diameter side? Are you using server example out of the box?

jaroszan commented 7 years ago

Just checked server/client examples provided with go-diameter and it seems to work ok. No undecodeable avps in wireshark. Where you have code 0, I have 258. It's important to know as well whether you made any custom modifications to your version of go-diameter.

muxalko commented 7 years ago

Well, not me. It is used in Cgrates project here . The dictionary I am trying to load is here.

I 've added code to log supported applications when appended, and tried to comment parser Default in ./autogen.sh:

func init() {
    Default, _ = NewParser()
//  Default.Load(bytes.NewReader([]byte(baseXML)))
    Default.Load(bytes.NewReader([]byte(creditcontrolXML)))
//  Default.Load(bytes.NewReader([]byte(networkaccessserverXML)))
//  Default.Load(bytes.NewReader([]byte(tgpprorfXML)))
}

Steps to apply the change:

~/proj/src/github.com/cgrates/cgrates/vendor/github.com/fiorix/go-diameter/diam$ ./autogen.sh 
commands.go
avp/codes.go
dict/default.go
~/proj/src/github.com/cgrates/cgrates/vendor/github.com/fiorix/go-diameter/diam$ go build
~/proj/src/github.com/cgrates/cgrates$ ./build.sh 
Building CGRateS ...

And here is the log:

2017/04/19 18:17:08 Append diameter locally supported app: &{4 auth 0}
2017/04/19 18:17:08 Append diameter locally supported app: &{4  10415}

Removed comments, changes to :

2017/04/20 00:44:57 Append diameter locally supported app: &{3 acct 0}
2017/04/20 00:44:57 Append diameter locally supported app: &{4 auth 0}
2017/04/20 00:44:57 Append diameter locally supported app: &{1 auth 0}
2017/04/20 00:44:57 Append diameter locally supported app: &{4  10415}

&{4 10415} looks wrong to me.

Looks like addApp object values are messed when loading dict. Will try to debug it.

jaroszan commented 7 years ago

I can't help you with custom version of go-diameter. I tried to reproduce the error but with no luck. Can you try with original version of go-diameter and check if the error is there?

I also can't understand why you have removed base dictionary, according to RFC 6733:

Diameter servers MUST support the base protocol

muxalko commented 7 years ago

It was my mistake. Thanks for helping. The issue was with custom dict xml file, somehow it has overridden app values. After the fix, all applications advertised correctly.

2017/04/22 03:39:40 Append diameter locally supported app: &{3 acct 0} 2017/04/22 03:39:40 Append diameter locally supported app: &{4 auth 0} 2017/04/22 03:39:40 Append diameter locally supported app: &{1 auth 0} 2017/04/22 03:39:40 Append diameter locally supported app: &{4 acct 10415} 2017/04/22 03:39:40 Append diameter locally supported app: &{4 acct 0} 2017/04/22 03:39:40 Append diameter locally supported app: &{4 acct 94} 2017/04/22 03:39:40 Append diameter locally supported app: &{4 auth 28458}

Though, I am trying to understand the diameter basics: if client CER includes

   AVP: Origin-Host(264) l=53 f=-M- val=GatewayService-7-10-1.PT
    AVP: Origin-Realm(296) l=24 f=-M- val=golantelecom.net
    AVP: Host-IP-Address(257) l=14 f=-M- val=10.224.234.230
    AVP: Vendor-Id(266) l=12 f=-M- val=28458
    AVP: Product-Name(269) l=16 f=--- val=FLEXI NG
    AVP: Supported-Vendor-Id(265) l=12 f=-M- val=10415
    AVP: Supported-Vendor-Id(265) l=12 f=-M- val=28458
    AVP: Supported-Vendor-Id(265) l=12 f=-M- val=94
    AVP: Auth-Application-Id(258) l=12 f=-M- val=Diameter Credit Control Application (4)

what should be in CEA for success ? must I have dictionary for all Supported-Vendor-Id values with application type/id?

jaroszan commented 7 years ago

If you plan to dive into diameter I suggest that you download your own version of RFC 6733 (https://tools.ietf.org/html/rfc6733). For this particular enquiry I can help (it's all in RFC ):

The receiver of the Capabilities-Exchange-Request (CER) MUST determine common applications by computing the intersection of its own set of supported Application Ids against all of the Application-Id AVPs (Auth-Application-Id, Acct-Application-Id, and Vendor-Specific-Application-Id) present in the CER. The value of the Vendor-Id AVP in the Vendor-Specific-Application-Id MUST NOT be used during computation. The sender of the Capabilities-Exchange-Answer (CEA) SHOULD include all of its supported applications as a hint to the receiver regarding all of its application capabilities.

muxalko commented 7 years ago

Thank you jarosan, The issue was related to custom dictionary file, not sure exactly why, perhaps conflicting AVP. Resolved by recopying xml files from repo. Something weird and strange.

jaroszan commented 7 years ago

It's important to verify that custom dictionaries are properly formatted and do not overwrite already existing dictionaries. You can try, for example, to use one of the last AVPs/commands in the dictionary for test and check if it's properly encoded and decoded.

It's easier to declare custom dictionary the same way as it is done here with helloDictionary. It is easier to troubleshoot than to go through the whole autogen cycle each time something goes awry.

It's still possible that there is a bug somewhere, but it must be reproducible in order to be addressed.

muxalko commented 7 years ago

Thanks a lot !

abrahamaa commented 5 years ago

Hi,

Am cannot see diameter messages of my custom diameter application (Application-ID 16777255, code: 8388620) in Wireshark after modifying the go-diameter example server and client codes. Can you explain why am not seeing my custom application AVPs in Wireshark. What I did was to write a custom dictionary for my application and load it at both the server and the client, similar to the "hellodictionary" but defining all the application as well. I also modify the client settings to advertise the VendorSpecificApplication which is a group AVP containg the VendorID and ApplicationID AVPs. I modify the hello message request (HMR) and HMA handler at both client and server to contain the AVPs of my custom application (sendPLR/handlePLA). Running the server and the client code and using Wireshark,

  1. The transport connection (TCP) was established OK
  2. CER/CEA messages were exchanges successfully (Result Code=Success) however, I could not see the "sendPLR" from the client as well as the "handlePLA" response from the server in Wireshark before the DWR/DWA messages.

Is modifying the HM the right thing to do to be able to send my custom application AVPs (example MSISDN, IMEI, Location-Type, etc) which I have defined them in my custom dictionary or I have to do something else?

abrahamafriyie commented 5 years ago

Hi All,

It it possible to modify the go-diameter package to implement Diameter EAP Application in the base protocol, instead of the current capabilities exchange request and answer, CER/CEA exchange, i would like to do a DER/DEA request and answer. If possible how would go about it, which files or part of the diameter code did i need to modify to achieve this. Is it also possible to make changes to the baseXML?

fiorix commented 5 years ago

go-diameter provides the baseXML and default dictionaries only for convenience. You can completely ignore them and develop diameter applications with your own dictionaries.

I'd suggest you make it work with your entire set of dictionaries first, and when things work, if there's a fit for the general purpose convenient dictionaries, then add it there.