search

mouse07410 / asn1c

The ASN.1 Compiler
http://lionet.info/asn1c/
BSD 2-Clause "Simplified" License
97 stars 71 forks source link

The new version per_decode NGAP protocol NGSetupResopnse error #195

Closed shuimoshusheng closed 2 months ago

shuimoshusheng commented 2 months ago

Hi，When using aper_decode to decode the following hexadecimal string, it prompts a decoding failure. It can be encoded into this hexadecimal string using XER format XML, but it cannot be decoded onto the PDU. The old version of asn1c can decode the following hexadecimal string. Has the decoding file been modified. Thanks！

This version can decode asn1c-vlm_master.zip

asn file used wireshark，

The command to generate NGAP PDU structure source code is as follows: ASN1C_PREFIX=NGAP_ asn1c -pdu=all -fcompound-names -findirect-choice -fno-include-deps ./asn_file/ngap/*.asn

hexstring：201500520000040001000a03803567732d616d663000600008000064f62902004000564001ff0050002c3064f629000010080000010064f629000010080000020064f629000010080000030064f62900001008000004

old asn1c decode xer pdu :

<NGAP-PDU>
    <successfulOutcome>
        <procedureCode>21</procedureCode>
        <criticality><reject/></criticality>
        <value>
            <NGSetupResponse>
                <protocolIEs>
                    <NGSetupResponseIEs>
                        <id>1</id>
                        <criticality><reject/></criticality>
                        <value>
                            <AMFName>5gs-amf0</AMFName>
                        </value>
                    </NGSetupResponseIEs>
                    <NGSetupResponseIEs>
                        <id>96</id>
                        <criticality><reject/></criticality>
                        <value>
                            <ServedGUAMIList>
                                <ServedGUAMIItem>
                                    <gUAMI>
                                        <pLMNIdentity>64 F6 29</pLMNIdentity>
                                        <aMFRegionID>
                                            00000010
                                        </aMFRegionID>
                                        <aMFSetID>
                                            0000000001
                                        </aMFSetID>
                                        <aMFPointer>
                                            000000
                                        </aMFPointer>
                                    </gUAMI>
                                </ServedGUAMIItem>
                            </ServedGUAMIList>
                        </value>
                    </NGSetupResponseIEs>
                    <NGSetupResponseIEs>
                        <id>86</id>
                        <criticality><ignore/></criticality>
                        <value>
                            <RelativeAMFCapacity>255</RelativeAMFCapacity>
                        </value>
                    </NGSetupResponseIEs>
                    <NGSetupResponseIEs>
                        <id>80</id>
                        <criticality><reject/></criticality>
                        <value>
                            <PLMNSupportList>
                                <PLMNSupportItem>
                                    <pLMNIdentity>64 F6 29</pLMNIdentity>
                                    <sliceSupportList>
                                        <SliceSupportItem>
                                            <s-NSSAI>
                                                <sST>01</sST>
                                                <sD>00 00 01</sD>
                                            </s-NSSAI>
                                        </SliceSupportItem>
                                    </sliceSupportList>
                                </PLMNSupportItem>
                                <PLMNSupportItem>
                                    <pLMNIdentity>64 F6 29</pLMNIdentity>
                                    <sliceSupportList>
                                        <SliceSupportItem>
                                            <s-NSSAI>
                                                <sST>01</sST>
                                                <sD>00 00 02</sD>
                                            </s-NSSAI>
                                        </SliceSupportItem>
                                    </sliceSupportList>
                                </PLMNSupportItem>
                                <PLMNSupportItem>
                                    <pLMNIdentity>64 F6 29</pLMNIdentity>
                                    <sliceSupportList>
                                        <SliceSupportItem>
                                            <s-NSSAI>
                                                <sST>01</sST>
                                                <sD>00 00 03</sD>
                                            </s-NSSAI>
                                        </SliceSupportItem>
                                    </sliceSupportList>
                                </PLMNSupportItem>
                                <PLMNSupportItem>
                                    <pLMNIdentity>64 F6 29</pLMNIdentity>
                                    <sliceSupportList>
                                        <SliceSupportItem>
                                            <s-NSSAI>
                                                <sST>01</sST>
                                                <sD>00 00 04</sD>
                                            </s-NSSAI>
                                        </SliceSupportItem>
                                    </sliceSupportList>
                                </PLMNSupportItem>
                            </PLMNSupportList>
                        </value>
                    </NGSetupResponseIEs>
                </protocolIEs>
            </NGSetupResponse>
        </value>
    </successfulOutcome>
</NGAP-PDU>

old asn1c asn_print:

SuccessfulOutcome ::= {
    procedureCode: 21
    criticality: 0 (reject)
    value: NGSetupResponse ::= {
        protocolIEs: ProtocolIE-Container ::= {
            NGSetupResponseIEs ::= {
                id: 1
                criticality: 0 (reject)
                value: 5gs-amf0
            }
            NGSetupResponseIEs ::= {
                id: 96
                criticality: 0 (reject)
                value: ServedGUAMIList ::= {
                    ServedGUAMIItem ::= {
                        gUAMI: GUAMI ::= {
                            pLMNIdentity: 64 F6 29
                            aMFRegionID: 02
                            aMFSetID: 00 40 (6 bits unused)
                            aMFPointer: 00 (2 bits unused)
                        }
                    }
                }
            }
            NGSetupResponseIEs ::= {
                id: 86
                criticality: 1 (ignore)
                value: 255
            }
            NGSetupResponseIEs ::= {
                id: 80
                criticality: 0 (reject)
                value: PLMNSupportList ::= {
                    PLMNSupportItem ::= {
                        pLMNIdentity: 64 F6 29
                        sliceSupportList: SliceSupportList ::= {
                            SliceSupportItem ::= {
                                s-NSSAI: S-NSSAI ::= {
                                    sST: 01
                                    sD: 00 00 01
                                }
                            }
                        }
                    }
                    PLMNSupportItem ::= {
                        pLMNIdentity: 64 F6 29
                        sliceSupportList: SliceSupportList ::= {
                            SliceSupportItem ::= {
                                s-NSSAI: S-NSSAI ::= {
                                    sST: 01
                                    sD: 00 00 02
                                }
                            }
                        }
                    }
                    PLMNSupportItem ::= {
                        pLMNIdentity: 64 F6 29
                        sliceSupportList: SliceSupportList ::= {
                            SliceSupportItem ::= {
                                s-NSSAI: S-NSSAI ::= {
                                    sST: 01
                                    sD: 00 00 03
                                }
                            }
                        }
                    }
                    PLMNSupportItem ::= {
                        pLMNIdentity: 64 F6 29
                        sliceSupportList: SliceSupportList ::= {
                            SliceSupportItem ::= {
                                s-NSSAI: S-NSSAI ::= {
                                    sST: 01
                                    sD: 00 00 04
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}
mouse07410 commented 2 months ago

Yes there was a change. Let me see if I can/should reverse it.

shuimoshusheng commented 2 months ago

Yes there was a change. Let me see if I can/should reverse it.

Thank you for your reply. This issue was discovered while attempting to upgrade to the latest version. Due to a lack of understanding of the internal implementation, intelligent questioning cannot solve the problem. Thank you for your efforts!😀

mouse07410 commented 2 months ago

I think I'll revert the offending change (@v0-e, please note). Would have to wait till I get to the right computer (probably next week).

In the meanwhile, would you mind explaining what produced that JER file?

And if asn1c would have to re-encode it to JER/JSON - how would it know whether to output a contiguous string, or a string broken by CR/LF into, say, 40-character chunks?

shuimoshusheng commented 2 months ago

I think I'll revert the offending change (@v0-e, please note). Would have to wait till I get to the right computer (probably next week).

In the meanwhile, would you mind explaining what produced that JER file?

And if asn1c would have to re-encode it to JER/JSON - how would it know whether to output a contiguous string, or a string broken by CR/LF into, say, 40-character chunks?

Is so sorry, I just saw the message. This xer file and jer file was printed using the following function:

xer_fprint(stdout, &asn_DEF_NGAP_NGAP_PDU, (void **)pdu);
asn_fprint(stdout, &asn_DEF_NGAP_NGAP_PDU, (void **)pdu);

That pdu uses the following function to decode the hexadecimal value of the packet response:

aper_decode(NULL, &asn_DEF_NGAP_NGAP_PDU, (void **)pdu,  buffer, buffer_size, 0, 0);

response packet hexadecimal: 201500520000040001000a03803567732d616d663000600008000064f62902004000564001ff0050002c3064f629000010080000010064f629000010080000020064f629000010080000030064f62900001008000004

the pcap (AMF N2 response): output.zip

mouse07410 commented 2 months ago

Should be fixed by https://github.com/mouse07410/asn1c/commit/2782c366dfe542e16e81aaa463691dbbb250842d

Please confirm.

shuimoshusheng commented 2 months ago

Should be fixed by 2782c36

Please confirm.

I'm very sorry, but I still have problems after trying this commit.

Use xer_decode to decode this pdu: 

<NGAP-PDU>
    <successfulOutcome>
        <procedureCode>21</procedureCode>
        <criticality><reject/></criticality>
        <value>
            <NGSetupResponse>
                <protocolIEs>
                    <NGSetupResponseIEs>
                        <id>1</id>
                        <criticality><reject/></criticality>
                        <value>
                            <AMFName>5gs-amf0</AMFName>
                        </value>
                    </NGSetupResponseIEs>
                    <NGSetupResponseIEs>
                        <id>96</id>
                        <criticality><reject/></criticality>
                        <value>
                            <ServedGUAMIList>
                                <ServedGUAMIItem>
                                    <gUAMI>
                                        <pLMNIdentity>64 F6 29</pLMNIdentity>
                                        <aMFRegionID>
                                            00000010
                                        </aMFRegionID>
                                        <aMFSetID>
                                            0000000001
                                        </aMFSetID>
                                        <aMFPointer>
                                            000000
                                        </aMFPointer>
                                    </gUAMI>
                                </ServedGUAMIItem>
                            </ServedGUAMIList>
                        </value>
                    </NGSetupResponseIEs>
                    <NGSetupResponseIEs>
                        <id>86</id>
                        <criticality><ignore/></criticality>
                        <value>
                            <RelativeAMFCapacity>255</RelativeAMFCapacity>
                        </value>
                    </NGSetupResponseIEs>
                    <NGSetupResponseIEs>
                        <id>80</id>
                        <criticality><reject/></criticality>
                        <value>
                            <PLMNSupportList>
                                <PLMNSupportItem>
                                    <pLMNIdentity>64 F6 29</pLMNIdentity>
                                    <sliceSupportList>
                                        <SliceSupportItem>
                                            <s-NSSAI>
                                                <sST>01</sST>
                                                <sD>00 00 01</sD>
                                            </s-NSSAI>
                                        </SliceSupportItem>
                                    </sliceSupportList>
                                </PLMNSupportItem>
                                <PLMNSupportItem>
                                    <pLMNIdentity>64 F6 29</pLMNIdentity>
                                    <sliceSupportList>
                                        <SliceSupportItem>
                                            <s-NSSAI>
                                                <sST>01</sST>
                                                <sD>00 00 02</sD>
                                            </s-NSSAI>
                                        </SliceSupportItem>
                                    </sliceSupportList>
                                </PLMNSupportItem>
                                <PLMNSupportItem>
                                    <pLMNIdentity>64 F6 29</pLMNIdentity>
                                    <sliceSupportList>
                                        <SliceSupportItem>
                                            <s-NSSAI>
                                                <sST>01</sST>
                                                <sD>00 00 03</sD>
                                            </s-NSSAI>
                                        </SliceSupportItem>
                                    </sliceSupportList>
                                </PLMNSupportItem>
                                <PLMNSupportItem>
                                    <pLMNIdentity>64 F6 29</pLMNIdentity>
                                    <sliceSupportList>
                                        <SliceSupportItem>
                                            <s-NSSAI>
                                                <sST>01</sST>
                                                <sD>00 00 04</sD>
                                            </s-NSSAI>
                                        </SliceSupportItem>
                                    </sliceSupportList>
                                </PLMNSupportItem>
                            </PLMNSupportList>
                        </value>
                    </NGSetupResponseIEs>
                </protocolIEs>
            </NGSetupResponse>
        </value>
    </successfulOutcome>
</NGAP-PDU>

, and then use asn_encode_to_new_buffer to perform PER encoding on the decoded PDU, generating the following hexadecimal string: 201500520000040001000a03803567732d616d663000600008000064f62902004000564001ff0050002c3064f629000010080000010064f629000010080000020064f629000010080000030064f62900001008000004

However, it is not possible to decode the hexadecimal string generated by asn_encode_to_new_buffer back using aper_decode

asn_encode_to_new_buffer(NULL, ATS_ALIGNED_CANONICAL_PER, &asn_DEF_NGAP_NGAP_PDU, pdu);
mouse07410 commented 2 months ago

Commit https://github.com/mouse07410/asn1c/commit/ab8ed188d95b1515dca8931fe3c17d1ca9e05d82 completely reverses the change that I believe is responsible.

Please confirm that now it works as expected.

Update

Oops - you're talking about APER decoding, not JER decoding! I don't recall changes to APER... Will need to look deeper. In the meanwhile, I'd still appreciate if you can check the latest commit.

Update 2

Can you revert commit 998e7ea on your machine and tell me if it helped?

v0-e commented 2 months ago

Commit 5fa129c may also be related given the comments there.

mouse07410 commented 2 months ago

@v0-e thanks! @shuimoshusheng please try reverting one at a time.

Offhand, https://github.com/mouse07410/asn1c/issues/195#issuecomment-2273377986 seems to point correctly at the offending one (again, thanks @v0-e ). The question is - what should we do about it, because that commit enables correct (AFAIK) processing of constraints.

shuimoshusheng commented 2 months ago

Commit ab8ed18 completely reverses the change that I believe is responsible.

Please confirm that now it works as expected.

Update

Oops - you're talking about APER decoding, not JER decoding! I don't recall changes to APER... Will need to look deeper. In the meanwhile, I'd still appreciate if you can check the latest commit.

Update 2

Can you revert commit 998e7ea on your machine and tell me if it helped?

This commit 998e7ea on my machine cannot decode normally either.

mouse07410 commented 2 months ago

OK, please restore 998e7ea and try with reverted 5fa129c

mouse07410 commented 2 months ago

Clarifying.

This commit 998e7ea on my machine cannot decode normally either.

Do you mean to say that with https://github.com/mouse07410/asn1c/commit/998e7ea24d6a33ca42b45b667b5e42ab4f44a4bc your decoder fails, and without it decoding succeeds?

@shuimoshusheng please clarify ASAP.

shuimoshusheng commented 2 months ago

Clarifying.

This commit 998e7ea on my machine cannot decode normally either.

Do you mean to say that with 998e7ea your decoder fails, and without it decoding succeeds?

@shuimoshusheng please clarify ASAP.

Sorry, I've been busy lately and don't have time to reply. I'll test it tomorrow.

shuimoshusheng commented 2 months ago

OK, please restore 998e7ea and try with reverted 5fa129c

Hi，I tested restore these two submissions and they can be decoded normally. Both have been restored, and decoding has become normal!

mouse07410 commented 2 months ago

Sorry, I do not understand.

Are you saying that you had to revert (remove) both of those APER-related commits to get decoding working again?

And reverting either one of these two alone did not help?

I need to know which of the two commits is failing your decoding. That means - you need to revert one, try, if decoding still doesn't work - re-apply that commit and revert the other one, then try again.

Or are you saying that the current vlm_master works for you as-is, with all the current commits applied???

shuimoshusheng commented 2 months ago

Sorry, I do not understand.

Are you saying that you had to revert (remove) both of those APER-related commits to get decoding working again?

And reverting either one of these two alone did not help?

I need to know which of the two commits is failing your decoding. That means - you need to revert one, try, if decoding still doesn't work - re-apply that commit and revert the other one, then try again.

Or are you saying that the current vlm_master works for you as-is, with all the current commits applied???

Here’s what I found after retesting the scenario:  Initially, I reverted the commit 998e7ea. When I attempted to decode, the operation failed and returned a status code of 2. Subsequently, I reapplied the 998e7ea commit and then reverted the 5fa129c commit. This time, the decoding process succeeded. This indicates that the 5fa129c commit introduced an issue specifically with decoding the OCTET STRING type. When inspecting the value of cval->range_fits, it displayed as 7.

shuimoshusheng commented 2 months ago

Commit 5fa129c may also be related given the comments there.

You are right!

mouse07410 commented 2 months ago

Thank you! Let us think how to fix this problem.

shuimoshusheng commented 2 months ago

Thank you! Let us think how to fix this problem.

Why cancel the annotation? When parsing the PrintableString type, setting OCTET STRING to 7 instead of 8 resulted in parsing errors.

AMFName ::= PrintableString (SIZE(1..150, ...))
RANNodeName ::= PrintableString (SIZE(1..150, ...))
mouse07410 commented 2 months ago

When parsing the PrintableString type, setting OCTET STRING to 7 instead of 8 resulted in parsing errors.

I'm sorry, could you please be more specific?

What (lines of) code set what value to 8? And is that operation based on some input value that is read from the encoded data?

Update

On the other hand, thinking about this more - PrintableString is ASCII, which is a 7-bit code, aka - each character takes 7 bits to encode. That is more compact than using 8 bits for every character, which is what PER/APER is about (compactness of encoding). So, perhaps, an encoder that "saves" one bit per character for PrintableString is correct? One question then is why it is not decoded correctly. Another question is why when you tell the decoder to ignore the constraints and use 8 bits for every character, it decodes the string correctly, even though it should in this case (in my opinion) mess up the decoding, taking one or more bit from follow-up characters.

mouse07410 commented 2 months ago

@shuimoshusheng would you be able to craft a simple reproducer of this problem? A simple ASN.1 PDU that could trigger this failure?

shuimoshusheng commented 2 months ago

@shuimoshusheng would you be able to craft a simple reproducer of this problem? A simple ASN.1 PDU that could trigger this failure?

If this code is not commented out and unit_bits is set to 8, it will decode normally。

 case ASN_OSUBV_ANY:
    case ASN_OSUBV_STR:
        canonical_unit_bits = unit_bits = 8;
/*
        if(cval->flags & APC_CONSTRAINED)
            unit_bits = cval->range_bits;
*/
        bpc = OS__BPC_CHAR;
        break;
    case ASN_OSUBV_U16:

After removing the comments from this code, when parsing the PrintableString, unit_bits = cval->range_bits = 7；

I guess this is 8-bit. I understand that even compact encoding should make up a byte of 8 bits instead of 7 bits. This is just my personal understanding, and the specific standard has not been found yet. I will share the code later. Thank you

mouse07410 commented 2 months ago

What code/application produces the encoding with cval->range_bits = 7?

And why do you expect characters of PrintableString to be encoded into 8 bits each, when 7 bits seems sufficient?

Also, can you test the current vlm_master?

mouse07410 commented 2 months ago

I did some experimentation, and ended up reverting the offending commit (actually, a bit more complicated than that, but...).

Please check the current vlm_master - it should now work for you.

shuimoshusheng commented 2 months ago

I did some experimentation, and ended up reverting the offending commit (actually, a bit more complicated than that, but...).

Please check the current vlm_master - it should now work for you.

I have tested the current vlm_master , but it still cannot parse the hexadecimal string above. However, it can encode the XML and decode the hexadecimal string after encoding the XML. I think updating all 5G open source programs that use this program should solve the problem. With the current version, there should be no problem.

The result of encoding XML as a hexadecimal string using vlm_master is as follows

201500510000040001000903806b9f9adc3b733000600008000064f62902004000564001ff0050002c3064f629000010080000010064f629000010080000020064f629000010080000030064f62900001008000004

Different from the previous hexadecimal string：

201500520000040001000a03803567732d616d663000600008000064f62902004000564001ff0050002c3064f629000010080000010064f629000010080000020064f629000010080000030064f62900001008000004

These two differences lie in the inconsistent length when encoding PrintableString. The new version is missing one byte, as shown below： new: 51 90 3806b9f9adc3b733
old: 52 a0 3803567732d616d663

From the actual coding effect, your method is correct and the coding PrintableString type is more compact. Thank you very much！

shuimoshusheng commented 2 months ago

ended

cval->range_bits = 7

The new aper encoding is more compact than the original, resulting in errors when using the new decoder to decode the old encoding result. cval ->range_fits is set to 7, while the old version is set to 8. Not every byte is 8 bits as I understand it. Thank you! 👍

shuimoshusheng commented 2 months ago

In this file ITU-Z.14 14.15.1chapter, Octet is set 8 Bitstring. image

So please confirm if it is always set to 8 bits in 5fa129c。

OCTET_STRING_encode_aper and OCTET_STRING_decode_aper should be consistent

mouse07410 commented 2 months ago

Both encode and decode should be consistent now, and set the value to 8 bits.

shuimoshusheng commented 2 months ago

Both encode and decode should be consistent now, and set the value to 8 bits.

Thank you, Please update and restore comments. I will close this issues.

mouse07410 commented 2 months ago

@shuimoshusheng I believe now APER encoding and decoding of OCTET STRING conforms to the ITU spec.

Please double-check the current version on GitHub.