APER problems - Githubissues

mhanna123 commented 7 years ago

Hello,

I am using asn1c to generate the code for the sbc-ap protocol based on the asn1 in 3GPP 29.168 Release 14

In my tests, I am encoding the write-replace-warning-request (aper), and checking the encoding in wireshark.

I faced 2 problems

1- In encoding in OCTET_STRING_encode_aper: an extra 1bit length with value 0 was added in cases where it should not be present (based on APER encoding, if the tag has a fixed length, there is no need to add the length)

I modified the code as follows:

1954         if(csiz->effective_bits >= 0) {
1955                 ASN_DEBUG("Encoding %d bytes (%lld), length in %d bits",
1956                                 st->size, sizeinunits - csiz->lower_bound,
1957                                 csiz->effective_bits);
                   if(csiz->effective_bits) // added line
                   { // added line
1958                 ret = aper_put_length(po, csiz->upper_bound - csiz->lower_bound + 1, sizeinunits - csiz->lower_bound);
1959                 if(ret) ASN__ENCODE_FAILED;
                   } // added line
1960                 if (st->size > 2) { /* X.691 #16 NOTE 1 */
1961                         if (aper_put_align(po) < 0)
1962                                 ASN__ENCODE_FAILED;
1963                 }

You can check test1.pcap (without the fix), test2.pcap (after the fix)

2- the decoding of the Warning-Area-List was failing the encoding was giving the following data: 00 00 01 00 43 65 87 31 32 33 30 43 65 87 41 42 43 30 the decoding was considering the PDU not complete

I am attaching the output: out_err.log

The problem seems to be in SEQUENCE_decode_aper. The last field in a sequence is a bitstring (CellIdentity ::= BIT STRING (SIZE (28))), and the next sequence should start with a bitstring (for the extension). There should not be any padding after the Cell-Id.

I did the following modifications in constr_SEQUENCE.c:

1318                 if(padding > 0)
1319                   ASN_DEBUG("For element %s,offset= %d Padding bits = %d", td->name, pd->moved, padding);
1320                 per_get_few_bits(pd, padding);

instead of line 1320, I added the following:

      if(edx != (td->elements_count-1))
         per_get_few_bits(pd, padding);
      else //
      {
         if(specs->roms_count && (padding > 0))
            ASN_DEBUG(">>>>> not skipping padding of %d bits for element:%d out of %d", padding, edx, td->elements_count);
         else
            per_get_few_bits(pd, padding);
      }

I am not an expert in APER, so I would appreciate if you can check if what I've done is correct

test_pcap.zip

Thx and best regards

mhanna123 commented 7 years ago

the missing out_err log

out_error.txt

mouse07410 commented 7 years ago

Thank you for your report and proposed fix. Let me review it, and get back to you here.

@velichkov, if you're around, would you mind taking a look at this too? Thanks!

@mhanna123 while I'm verifying this, one question: are you sure your PCAP is APER and not PER/UPER? I.e., that your PCAP shows Aligned PER encoding? What happens if you try to generate/use PER rather than APER code? Would it be possible to post here the fragment of ASN.1 that describes the binary/encoded data we're discussing?

mhanna123 commented 7 years ago

@mouse07410 I am using Aligned PER. PER encoding give a completely different encoding. For example, for the warning-area-list using APER: 00 00 01 00 43 65 87 31 32 33 30 43 65 87 41 42 43 30
using PER: 00 08 86 CB 0E 62 64 66 62 1B 2C 3A 0A 12 19 80

Based on the specs, APER should be used, so I didn't try encoding the complete PDU and passing it through wireshark.

The complete asn1 is defined in 29.168 (I am using rel 14). This is a stripped part just for the warning-area-list:

itu-t (0) identified-organization (4) etsi (0) mobileDomain (0) 
eps-Access (21) modules (3) sbc-AP (3) version1 (1) sbc-AP-IEs (2)}

DEFINITIONS AUTOMATIC TAGS ::= 

BEGIN

Warning-Area-List          ::= CHOICE {
   cell-ID-List            ECGIList,
   tracking-Area-List-for-Warning               TAI-List-for-Warning,
   emergency-Area-ID-List     Emergency-Area-ID-List,
   ...
}

ECGIList             ::= SEQUENCE (SIZE(1..1024)) OF EUTRAN-CGI

EUTRAN-CGI ::= SEQUENCE {
   pLMNidentity         PLMNidentity,
   cell-ID              CellIdentity,
   iE-Extensions        ProtocolExtensionContainer { {EUTRAN-CGI-ExtIEs} } OPTIONAL,
   ...
}

EUTRAN-CGI-ExtIEs SBC-AP-PROTOCOL-EXTENSION ::= {
   ...
}

ProtocolExtensionContainer {SBC-AP-PROTOCOL-EXTENSION : ExtensionSetParam} ::= 
   SEQUENCE (SIZE (1..1024)) OF
   ProtocolExtensionField {{ExtensionSetParam}}

ProtocolExtensionField {SBC-AP-PROTOCOL-EXTENSION : ExtensionSetParam} ::= SEQUENCE {
   id          SBC-AP-PROTOCOL-EXTENSION.&id       ({ExtensionSetParam}),
   criticality       SBC-AP-PROTOCOL-EXTENSION.&criticality    ({ExtensionSetParam}{@id}),
   extensionValue       SBC-AP-PROTOCOL-EXTENSION.&Extension      ({ExtensionSetParam}{@id})
}

PLMNidentity            ::= TBCD-STRING 

TBCD-STRING ::= OCTET STRING (SIZE (3))

CellIdentity         ::= BIT STRING (SIZE (28))

SBC-AP-PROTOCOL-EXTENSION ::= CLASS {
   &id            ProtocolExtensionID  UNIQUE,
   &criticality         Criticality    DEFAULT ignore,
   &Extension,
   &presence      Presence
}
WITH SYNTAX {
   ID          &id
   CRITICALITY       &criticality
   EXTENSION         &Extension
   PRESENCE       &presence
}

Criticality    ::= ENUMERATED { reject, ignore, notify }

Presence    ::= ENUMERATED { optional, conditional, mandatory }

ProcedureCode     ::= INTEGER (0..255)

ProtocolExtensionID  ::= INTEGER (0..65535)

ProtocolIE-ID     ::= INTEGER (0..65535)

TriggeringMessage    ::= ENUMERATED {initiating-message, successful-outcome, unsuccessful-outcome, outcome}

TAC ::= OCTET STRING (SIZE (2)) 

TAI-List-for-Warning ::= SEQUENCE (SIZE(1.. 1024)) OF TAI

TAI ::= SEQUENCE {
   pLMNidentity         PLMNidentity,
   tAC                  TAC,
   iE-Extensions        ProtocolExtensionContainer { {TAI-ExtIEs} } OPTIONAL
}

TAI-ExtIEs SBC-AP-PROTOCOL-EXTENSION ::= {
   ...
}

Emergency-Area-ID-List     ::=   SEQUENCE (SIZE(1..1024)) OF Emergency-Area-ID

Emergency-Area-ID       ::=   OCTET STRING (SIZE (3)) 

END

the values used:

rec1value Warning-Area-List ::= cell-ID-List : 
  {
    {
      pLMNidentity '436587'H,
      cell-ID '00110001 00110010 00110011 0011'B
    },
    {
      pLMNidentity '436587'H,
      cell-ID '01000001 01000010 01000011 0011'B
    }
  }

mouse07410 commented 7 years ago

@mhanna123 your proposed fix looks reasonable, so I integrated it in the code. Thank you for reporting!

I will experiment with this a bit more, but the current master should be usable for you now.

oaitamrane commented 7 years ago

@mhanna123 I am tying to generate code for the PCAP protocol based on the asn1 in 3GPP TS 25.453 V11.0.0 to check the encoding in wireshark. I am using asn1c-0.9.28 but this version does not support APER and Information Object Class. As sbc-ap uses these features where did you get the asn1c to generate your code.

mhanna123 commented 7 years ago

@oaitamrane pcap = packet capture, not Positioning Calculation Application Part I am using the asn1 definition for the SBC-AP (E-UTRAN Cell Broadcast Application Proocol) from 3GPP 29.168 rel14

@mouse07410 BTW, I had some warnings while generating the sources: WARNING: Parameterized type maxProtocolIEs expected for maxProtocolIEs at line 77 in container_def.asn1 WARNING: Parameterized type SBC-AP-PROTOCOL-IES expected for SBC-AP-PROTOCOL-IES at line 81 in container_def.asn1 WARNING: Parameterized type SBC-AP-PROTOCOL-IES expected for SBC-AP-PROTOCOL-IES at line 81 in container_def.asn1

I didn't get any declaration of the values in constant definitions asn1 section. I manually added them

oaitamrane commented 7 years ago

@mhanna123 what is the version of asn1c you are using ?

oaitamrane commented 7 years ago

I surely did not explain clearly my problem. I want to to generate code for the Positioning Calculation Application Part (PCAP). In a first step I want to use 3GPP TS 25.453 V11.0.0 to check ASN1 encoding with Wireshark. I am using asn1c-0.9.28 but this version does not support APER and Information Object Class (IoC). APER and IoC support is needed for PCAP. As sbc-ap needs APER and IoC support I want to know how to get the asn1c you are using.

Can you please help me.

mouse07410 commented 7 years ago

I surely did not explain clearly my problem.

I concur. ;-)

I am using asn1c-0.9.28 but this version does not support APER and Information Object Class (IoC).

You're posting to an issue on a fork of that asn1c, not the original project. This fork happens to support APER and IOC. Why don't you clone it, build it, and give it a try? Something like

$ git clone https://github.com/mouse07410/asn1c.git
$ cd asn1c
$ autoreconf -iv
$ ./configure
$ make clean && make -j 2 && make check && sudo make install

After the above succeeds (assuming it does :) you'll have a working asn1c installed in /usr/local/bin.

oaitamrane commented 7 years ago

@mouse07410 What is the compile option to generate APER

mouse07410 commented 7 years ago

What is the compile option to generate APER

If you'd bother to do asn1c -h, you'd get output like this:

$ asn1c -h
ASN.1 Compiler, v1.0.0
Copyright (c) 2003-2017 Lev Walkin <vlm@lionet.info>
Usage: asn1c [options] file ...
Options:
  -V                    Print version of the  ASN.1 compiler and exit
  -v                    Print version of the  ASN.1 compiler and exit
  -E                    Run only the ASN.1 parser and print out the tree
  -F                    During -E operation, also perform tree fixing
. . . . .
 -gen-PER              Generate PER support code
 -pdu={all|auto|Type}  Generate PDU table (discover PDUs automatically)
. . . . .

Needless to say that the option you're looking for is -gen-PER. It generates both UPER and APER.

mouse07410 commented 7 years ago

Assuming the problem has been resolved. Re-open with more details if it isn't.

oaitamrane commented 7 years ago

@mouse07410, I have installed the version you suggested and compiled Positioning Calculation Application Part (PCAP) 3GPP TS 25.453 V11.0.0 procedures. asn1c showed no waning or error when invoked, but it uses a structure "struct ProtocolIE_Field" in the structures defining ASN1 messages. Unfortunately I find nowhere the declaration of "struct ProtocolIE_Field" !!!

oaitamrane commented 7 years ago

@mouse07410,

Forgot to explain that in the ASN.1 definition (see below) ProtocolIE-Field is defined as:

ProtocolIE-Container {PCAP-PROTOCOL-IES : IEsSetParam} ::= SEQUENCE (SIZE (0..maxProtocolIEs)) OF ProtocolIE-Field {{IEsSetParam}}

ProtocolIE-Field {PCAP-PROTOCOL-IES : IEsSetParam} ::= SEQUENCE { id PCAP-PROTOCOL-IES.&id ({IEsSetParam}), criticality PCAP-PROTOCOL-IES.&criticality ({IEsSetParam}{@id}), value PCAP-PROTOCOL-IES.&Value ({IEsSetParam}{@id}) }

velichkov commented 7 years ago

Hi @oaitamrane,

Could you share the exact ASN files you are working with? Extracting the asn from the 3gpp docs is a time consuming task.

I'm able to compile successfully the PCAP asn files from the wireshark https://github.com/wireshark/wireshark/tree/2832f4e97d77324b4e46aac40dae0ce898ae559d/epan/dissectors/asn1/pcap

$  ../../asn1c_mouse/asn1c/asn1c  -fcompound-names -gen-PER *.asn -pdu=PCAP-PDU
$ make -f Makefile.am.sample 
$ ./progname --help

oaitamrane commented 7 years ago

@velichkov, Thank you for your help. As I don't how to attach the spec I send it to your private email address. Regatds

velichkov commented 7 years ago

Hi @oaitamrane,

As I don't how to attach the spec I send it to your private email address.c

You could attach files in the github issues - it is described how at the bottom of the form used to write comments

Attach files by dragging & dropping,, or pasting from the clipboard.

or you could upload them in google drive, dopbox or various other services and post a link.

I'm able to successfully compile the asn files you've sent me using the mouse07410/asn1c.git with the latest master (commit 165e8504bb61e229349ade814cc0a192bb86dc8d)

asn1c -pdu=auto -fcompound-names -gen-PER *.asn -pdu=PCAP-PDU

Although this is not the problem specify the -pdu= only once, e.g. only -pdu=PCAP-PDU

$grep "struct ProtocolIE_Field" *.h  | grep -v A_SEQUENCE_OF 
ProtocolIE-Container.h:struct ProtocolIE_Field;
ProtocolIE-Field.h:typedef struct ProtocolIE_Field_106P0 {
ProtocolIE-Field.h:typedef struct ProtocolIE_Field_106P1 {
ProtocolIE-Field.h:typedef struct ProtocolIE_Field_106P2 {
ProtocolIE-Field.h:typedef struct ProtocolIE_Field_106P3 {
ProtocolIE-Field.h:typedef struct ProtocolIE_Field_106P4 {
ProtocolIE-Field.h:typedef struct ProtocolIE_Field_106P5 {
ProtocolIE-Field.h:typedef struct ProtocolIE_Field_106P6 {
ProtocolIE-Field.h:typedef struct ProtocolIE_Field_106P7 {
ProtocolIE-Field.h:typedef struct ProtocolIE_Field_106P8 {
ProtocolIE-Field.h:typedef struct ProtocolIE_Field_106P9 {
ProtocolIE-Field.h:typedef struct ProtocolIE_Field_106P10 {
ProtocolIE-Field.h:typedef struct ProtocolIE_Field_106P11 {
ProtocolIE-Field.h:typedef struct ProtocolIE_Field_106P12 {
ProtocolIE-Field.h:typedef struct ProtocolIE_Field_106P13 {
ProtocolIE-Field.h:typedef struct ProtocolIE_Field_106P14 {
ProtocolIE-Field.h:typedef struct ProtocolIE_Field_106P15 {
ProtocolIE-Field.h:typedef struct ProtocolIE_Field_106P16 {
ProtocolIE-Field.h:typedef struct ProtocolIE_Field_106P17 {
ProtocolIE-Field.h:typedef struct ProtocolIE_Field_106P18 {
ProtocolIE-Field.h:typedef struct ProtocolIE_Field_106P19 {
ProtocolIE-Field.h:typedef struct ProtocolIE_Field_106P20 {
ProtocolIE-Field.h:typedef struct ProtocolIE_Field_106P21 {

It generates multiple definitions of ProtocolIE_Field with suffixes and all definitions are the same. I guess that you need to additionally decode the value depending on the id.

Could you share some binary PCAP messages or a wireshark pcap trace?

oaitamrane commented 7 years ago

Hi @velichkov, Thank you for your help.

I will probably get PCAP wireshark traces in 4 to 6 weeks and will share them with you at this time.

Regarding the generated C code I still don't find the definition of the inner structures. As an example let us focus on the procedure PositionInitiationRequest. Beginning from the PCAP_PDU structure I can go deeper until I reach the ProtocolIE_Container_100P6 member that is defined as:
A_SEQUENCE_OF(struct ProtocolIE_Field) list; And after this I don't understand how to find the Type of the members of this sequence: ProtocolIE_Field_106P0 or ProtocolIE_Field_106P1 ... or ProtocolIE_Field_106P21. Am I clear enough ?

velichkov commented 7 years ago

Hi @oaitamrane,

Currently the support for Information Object Class (IoC) is not as good as you might expect but it should be possible to decode/encode the inner structures like PositionInitiationRequest, you just need to do it in two or three steps with some additional code that you have to write.

First as usual you need to pass the whole APER encoded message to the aper_decode using the asn_DEF_PCAP_PDU and you will get something like (in XER format)

<PCAP-PDU>
    <initiatingMessage>
        <procedureCode>9</procedureCode>
        <criticality><ignore/></criticality>
        <transactionID>
            <shortTID>64</shortTID>
        </transactionID>
        <value>AA BB CC DD EE FF</value>
    </initiatingMessage>
</PCAP-PDU>

Here the procedureCode indicates that this is a PositionInitiation and the value is encoded.

You need to call again the aper_decode, this time with asn_DEF_PositionInitiationRequest and the above HEX value and you will get a list of protocolIEs and each element will contains an id and a value. You need to iterate over that list and call aper_decode for each element using the appropriate definitions depending on the id

velichkov commented 7 years ago

ProtocolIE_Field_106P0 or ProtocolIE_Field_106P1 ... or ProtocolIE_Field_106P21.

It doesn't matter whether it is ProtocolIE_Field_106P0 or ProtocolIE_Field_106P1 because all ProtocolIE_Field_106* structs are the same, all have an id, criticality and value members so when decoding you could cast to any ProtocolIE_Field of your choice, then you get the id and value and calls the aper_decode with the correct struct definition depending on the id

oaitamrane commented 7 years ago

Hi @velichkov, Thank you for your help regarding message decoding. In a first step I would like to encode a PositionInitiationRequest message for example. Is there a special requirement for this ?

velichkov commented 7 years ago

Hi @oaitamrane,

In a first step I would like to encode a PositionInitiationRequest message for example. Is there a special requirement for this ?

I don't know.

oaitamrane commented 7 years ago

Hi @velichkov, With code generated by 1.0.0 version of asn1c, GCC returns an error: ‘asn_TYPE_descriptor_t’ has no member named ‘free_struct’ While this did not cause errors with code generated by 0.9.28 version of asn1. Do you know why ?

velichkov commented 7 years ago

Hi @oaitamrane,

With code generated by 1.0.0 version of asn1c, GCC returns an error: ‘asn_TYPE_descriptor_t’ has no member named ‘free_struct’

That's because the API was changed (See vlm/asn1c#143, #7, #10 and #11) so you need to fix your code.

Every type has free, print, check_constraints, ber_decoder, der_encoder, xer_decoder, xer_encoder, uper_decoder, uper_encoder and outmost_tag operations. We move them out to a separate structure asn_TYPE_operation_t.

Instead of calling free_struct directly you could use the ASN_STRUCT_FREE macro instead.

oaitamrane commented 7 years ago

Hi @velichkov,

I was previously using "asn_DEF_PDU.free_struct(&asn_DEF_PDU, Pdu, 0)" as chown in asn1c Users Guide. I finally used the ASN_STRUCT_FREE macro and it worked. Thank you so much for your Help

oaitamrane commented 7 years ago

Hi @velichkov, When encoding a PositionInitiationRequest message, I don't know how I can fill the value part of "struct ProtocolIE_Field" that is a ANY_t. Let us focus on RequestType see "PCAP-PDU-Contents.asn".

In my test sample I write I first declare ProtocolIE_Field_106P0_t p_Field; RequestType_t requestType;

When reaching the RequestType part I write:

/ id-RequestType ProtocolIE-ID ::= 28 / p_Field = calloc(1, sizeof(ProtocolIE_Field_106P0_t)); if(NULL == p_Field) { printf("calloc(p_Field) failed\n"); goto FreeMem; } p_Field->id = 28; p_Field->criticality = Criticality_reject;

After this I don't know how to set up the value part that is a ANY_t and should be set as a RequestType_t (see RequestType.h) ??? If you did not keep the files I sent you by email please let me know so I send you back the archive.

Thank you for your Help

velichkov commented 7 years ago

After this I don't know how to set up the value part that is a ANY_t and should be set as a RequestType_t (see RequestType.h) ???

Use ANY_fromType_aper function

oaitamrane commented 7 years ago

Hi @velichkov, I have executed the test sample I have written for PositionInitiationRequest encoding and decoding. Encoding seems OK even if the first Octet of the encoded stream is 00 !!! Decoding of the Encoded stream fails !!! Can you take a look of my very basic sample.

Thank you for your Help

3GPP_PCAP.zip

oaitamrane commented 7 years ago

Hi @velichkov, After activating DEBUG "-DEMIT_ASN_DEBUG=1", I notice strange behaviour when reaching encoding/decoding of TransactionID: When encoding TransactionID as a TransactionID_PR_shortTID with a value 0x12 (18), aper_decoder discovers aTransactionID of TransactionID_PR_longTID type with a value of 0x1700 (5888). And when encoding TransactionID as a TransactionID_PR_longTID with a value 0x1234 (4660), aper_decoder discovers aTransactionID of TransactionID_PR_shortTID type with a value of 0x12 (18). Did you encounter any problem with encoding/decoping of structures such as:

typedef struct TransactionID { TransactionID_PR present; union TransactionID_u { long shortTID; long longTID; } choice; asn_struct_ctx_t _asn_ctx; } TransactionID_t;

with:

typedef enum TransactionID_PR { TransactionID_PR_NOTHING, TransactionID_PR_shortTID, TransactionID_PR_longTID } TransactionID_PR;

Why is there a TransactionID_PR_NOTHING member while TransactionID is either a short or long Mandatory parameter and not Optional ?

Thank you for you help.

velichkov commented 7 years ago

When encoding TransactionID as a TransactionID_PR_shortTID with a value 0x12 (18), aper_decoder discovers aTransactionID of TransactionID_PR_longTID type with a value of 0x1700 (5888). And when encoding TransactionID as a TransactionID_PR_longTID with a value 0x1234 (4660), aper_decoder discovers aTransactionID of TransactionID_PR_shortTID type with a value of 0x12 (18).

It seems you found a bug, now it will be good if you fix it and contribute the changes back.

Did you encounter any problem with encoding/decoping of structures such as:

No.

Why is there a TransactionID_PR_NOTHING member while TransactionID is either a short or long Mandatory parameter and not Optional ?

Probably because you want a value for the cases when the TransactionID is Optional.

P.S. Could you stop posting in this issue, it has already been closed as the initially reported problems have been solved.

mouse07410 / asn1c

APER problems #30