search

vlm / asn1c

The ASN.1 Compiler
http://lionet.info/asn1c/
BSD 2-Clause "Simplified" License
1.03k stars 553 forks source link

DSRC ETSI Regional Extenions #356

Open timstirling opened 4 years ago

timstirling commented 4 years ago

There have been a few posts related to getting DSRC regional extensions working but nothing satisfactory. I have been trying for a few days without luck.

The ASN schema can be downloaded form https://forge.etsi.org/rep/LIBS/LibIts/tree/master/asn1/IS In particular I am trying to parse SPAT messages. For the most part this works, but some messages have a regional extension in the MovementEvent:

   regional     SEQUENCE (SIZE(1..4)) OF 
                RegionalExtension {{Reg-MovementEvent}} OPTIONAL,

The Reg-MovementEvent is a stub, but in the data I receive it is just an integer Regional Extension value. I added the integer but this doesn't work. Not least because if I look at the generated code in MovementEvent.h the 'regional' data doesn't reference the specific Reg-Movement event definition, but looks like this:

struct MovementEvent__regional __regional {
                 A_SEQUENCE_OF(struct RegionalExtension) list;

                 /* Context for parsing across buffer boundaries */
                 asn_struct_ctx_t _asn_ctx;
         } *regional;

However, "RegionalExtension" is not actually defined as anything.

Even if we look at one of the regional extension that is defined in the ASN the code will look the same, e.g.

Reg-ConnectionManeuverAssist    REG-EXT-ID-AND-TYPE ::= { 
    {ConnectionManeuverAssist-addGrpC   IDENTIFIED BY addGrpC} ,
    ... 
}

ConnectionManeuverAssist-addGrpC .c and .h files are generated, but ConnectionManeuverAssist-addGrpC is not referenced from ConnectionManeuverAssist.h, and the generic RegionalExtension struct is used.

An example hex string that fails to parse is: 010479AD012C00381644598D9837BCD6809680080075F7B1A781A4058330300340583202003405836060034058350501300C05190160D01000D0160E42400D0160E02000D0160DC1C00D0160C40404C0301464058B20C01300C05190162C42C04C0301464058B00A003405BB01900540593811003405937100034059360F0034059350E0034059340D0034059B0120034059B8180034059B7170034059B6160034059B5150034059B4140034059B313003405BB11A005405BB21B005

Testing online here https://www.marben-products.com/cgi-bin/asn1tools/free-online-asn1-decoder.pl We get an output that includes the regional extension:

<MovementState>
            <movementName>05</movementName>
            <signalGroup>5</signalGroup>
            <state-time-speed>
              <MovementEvent>
                <eventState>
                  <stop-And-Remain/>
                </eventState>
                **<regional>
                  <SEQUENCE>
                    <regionId>3</regionId>
                    <regExtValue>46</regExtValue>
                  </SEQUENCE>
                </regional>**
              </MovementEvent>
            </state-time-speed>
          </MovementState>

I think the first steps to get thsi working will be to define the Reg-MovementEvent to something like this Reg-MovementEvent REG-EXT-ID-AND-TYPE ::= { {MovementEvent-addGrpC IDENTIFIED BY addGrpC} , ... } MovementEvent-addGrpC ::= INTEGER

Any advice? The asn1c tool get almost everything working so it is unfortunate I can't resolve this issue.

velichkov commented 4 years ago

Hi @timstirling,

Can you try with the vlm_master branch from mouse07410 fork or velichkov_s1ap_plus_option_group_plus_adding_trailing_ull branch from brchiu fork.

Both forks contain a lot of fixes and improved support of Information Object Class/Set.

DavidKelleySCSC commented 4 years ago

I really do not have advice regarding the ETSI Regional Extensions, but I can provide the context from which they took the DSRC regional extensions from the SAE DSRC J2735 work. And I think that will work for you. The extension process was intended to clearly separate the core ASN.1 spec from any changes that a deployment might needs to make for local needs. In other words, the base ASN.1 file should never need to be edited. Further, if there is not a regional stub present in a data frame, the deployment is not allowed to make any changes and still claim conformance to the standard. The raw truth is the a great many uses of the the standards in deployments are in fact not valid (from a stds conformance point of view) due to these local tweaks and this was developed to address this. You will note that the Regional Extensions live in their own namespace in their own file, and that is where any edits are supposed to be made.

And the SAE issued specifications work that way. Less so for the ETSI work, or for "19091" which this uses. In theory these were to 'reference' the J2725 work, but as the links above will show, large sections of the SAE J2735 (copyrighted) text has been reused without attribution to SAE or J2735 or permission. As the original author of that, that was work for hire from me for SAE; as to its reuse here, ask your lawyer. SAE Detroit seems fairly clueless on how to promote or protect their committees work.

Now the comment...["RegionalExtension" is not actually defined as anything.] indicates to me that ETSI just did not import everything that is needed. That item should have been imported. In fact it is also clear that much the "core" unique concept of MAP (that a delta offset center line based polygon points with various attributes at points and between segments is the best way to describe an ITS lane level map for different user needs) seems to not be copied but references what resides in the J2735 work. That concept is a good one and will likely outlive DSRC. In the current standard there are three defined regions (US, JPN, EU) but more were/are expected to be defined. And (and this is the key for your deployment needs) you are free to use values from 128 to 255 in that Octet to define your own region. Do so.

You can find some worked examples of this in the actual SAE J2735 document (as well as a number of normative items that do not show up in the raw ASN.1 specification). The basic design pattern is to take the 'local' file with the "Reg-xxx" entry you want to add content to, and add the desired content after the "..." (you could in fact replace the file with your own). Within that file it would be best to develop your own namespace (and file) for any unique content and re-use DSRC (or ETSI) content when suitable.

timstirling commented 4 years ago

Thank you for the detailed background David. i did have some questions in regards to copyright, although we have purchased DSRC files from SAE anyway.

velichkov, I compiled the mouse07410 fork, and with some additional CL options ( -fno-include-deps -findirect-choice ) I have code that looks much closer to working. But it still fails to compile.

I modified the ASN 19091 as follows:

MovementEvent-addGrpC ::=  INTEGER

Reg-MovementEvent               REG-EXT-ID-AND-TYPE ::= {
       {MovementEvent-addGrpC IDENTIFIED BY addGrpC},
        ...
 }

The code looks much better: MovementEvent.h

typedef struct MovementEvent {
        MovementPhaseState_t     eventState;
        struct TimeChangeDetails        *timing;        /* OPTIONAL */
        struct AdvisorySpeedList        *speeds;        /* OPTIONAL */
        struct MovementEvent__regional {
                A_SEQUENCE_OF(struct Reg_MovementEvent) list;

                /* Context for parsing across buffer boundaries */
                asn_struct_ctx_t _asn_ctx;
        } *regional;
        /*
         * This type is extensible,
         * possible extensions are below.
         */
        /* Context for parsing across buffer boundaries */
        asn_struct_ctx_t _asn_ctx;
} MovementEvent_t;

MovementEvent-addGrpC.h typedef long MovementEvent_addGrpC_t;

RegionalExtensions.h have a lot of code that looks correct, e.g.

typedef enum Reg_MovementEvent__regExtValue_PR {
        Reg_MovementEvent__regExtValue_PR_NOTHING,  /* No components present */
        Reg_MovementEvent__regExtValue_PR_MovementEvent_addGrpC
} Reg_MovementEvent__regExtValue_PR;

...
typedef struct Reg_MovementEvent {
        RegionId_t   regionId;
        struct Reg_MovementEvent__regExtValue {
                Reg_MovementEvent__regExtValue_PR present;
                union Reg_MovementEvent__regExtValue_u {
                        MovementEvent_addGrpC_t  MovementEvent_addGrpC;
                } choice;

                /* Context for parsing across buffer boundaries */
                asn_struct_ctx_t _asn_ctx;
        } regExtValue;

        /* Context for parsing across buffer boundaries */
        asn_struct_ctx_t _asn_ctx;
} Reg_MovementEvent_t;

This looks like it should all work so it is frustrating that it still doens't work. Any tips for helping to debug this? The ASN code is quite complex.

At this stage I am not too worried about the reading these regional extension values but I really want to parse the rest of the message.

velichkov commented 4 years ago

Hi @timstirling,

velichkov, I compiled the mouse07410 fork, and with some additional CL options ( -fno-include-deps -findirect-choice ) I have code that looks much closer to working. But it still fails to compile.

What error do you get? Provide all commands (with all arguments) that you are using to compile it.

This looks like it should all work so it is frustrating that it still doens't work.

Any tips for helping to debug this? The ASN code is quite complex.

Compile the C code with -DASN_EMIT_DEBUG=1 and first try without any asn.1 modifications.

timstirling commented 4 years ago

Sorry for the late reply. First of all, when using asn1c to compile the schema I get a lot of relevant warnings:

WARNING: Parameterized type REG-EXT-ID-AND-TYPE expected for REG-EXT-ID-AND-TYPE at line 272 in /home/timstirling/dsrc2/ITS/ISO_TS_19091.asn WARNING: Parameterized type REG-EXT-ID-AND-TYPE expected for REG-EXT-ID-AND-TYPE at line 273 in /home/timstirling/dsrc2/ITS/ISO_TS_19091.asn

THE ASN looks like this -- Regional extensions support

REG-EXT-ID-AND-TYPE ::= CLASS {
   &id     RegionId UNIQUE,
   &Type
   } WITH SYNTAX {&Type IDENTIFIED BY &id}

RegionalExtension {REG-EXT-ID-AND-TYPE : Set} ::= SEQUENCE {
   regionId     REG-EXT-ID-AND-TYPE.&id( {Set} ),
   regExtValue  REG-EXT-ID-AND-TYPE.&Type( {Set}{@regionId} )
   }

The debug flag helps a lot. Using the original ASN schema the relevant error is here:

Decoding member "regional" in MovementEvent (/home/timstirling/CS/CS/libraries/dsrcdecoder/src/constr_SEQUENCE.c:1170)
  [PER got  2<=1172 bits => span 390 +0[6..1176]:30 (1170) => 0x0] (/home/timstirling/CS/CS/libraries/dsrcdecoder/src/asn_bit_data.c:132)
Preparing to fetch 0+1 elements from regional (/home/timstirling/CS/CS/libraries/dsrcdecoder/src/constr_SET_OF.c:962)
SET OF Reg-MovementEvent decoding (/home/timstirling/CS/CS/libraries/dsrcdecoder/src/constr_SET_OF.c:981)
Decoding Reg-MovementEvent as SEQUENCE (UPER) (/home/timstirling/CS/CS/libraries/dsrcdecoder/src/constr_SEQUENCE.c:1098)
Decoding member "regionId" in Reg-MovementEvent (/home/timstirling/CS/CS/libraries/dsrcdecoder/src/constr_SEQUENCE.c:1170)
Decoding NativeInteger RegionId (UPER) (/home/timstirling/CS/CS/libraries/dsrcdecoder/src/NativeInteger.c:272)
Integer with range 8 bits (/home/timstirling/CS/CS/libraries/dsrcdecoder/src/INTEGER.c:637)
  [PER got  8<=1170 bits => span 398 +0[14..1176]:30 (1162) => 0x3] (/home/timstirling/CS/CS/libraries/dsrcdecoder/src/asn_bit_data.c:132)
Got value 3 + low 0 (/home/timstirling/CS/CS/libraries/dsrcdecoder/src/INTEGER.c:658)
NativeInteger RegionId got value 3 (/home/timstirling/CS/CS/libraries/dsrcdecoder/src/NativeInteger.c:288)
Freeing INTEGER as a primitive type (/home/timstirling/CS/CS/libraries/dsrcdecoder/src/asn_codecs_prim.c:125)
Decoding member "regExtValue" in Reg-MovementEvent (/home/timstirling/CS/CS/libraries/dsrcdecoder/src/constr_SEQUENCE.c:1170)
Type selector is not defined for Open Type Reg-MovementEvent->regExtValue->regExtValue (/home/timstirling/CS/CS/libraries/dsrcdecoder/src/OPEN_TYPE.c:304)
Failed to decode element Reg-MovementEvent (/home/timstirling/CS/CS/libraries/dsrcdecoder/src/OPEN_TYPE.c:306)
Failed decode regExtValue in Reg-MovementEvent (/home/timstirling/CS/CS/libraries/dsrcdecoder/src/constr_SEQUENCE.c:1179)
regional SET OF Reg-MovementEvent decoded 2, 0x25f3bf0 (/home/timstirling/CS/CS/libraries/dsrcdecoder/src/constr_SET_OF.c:984)
Failed decoding Reg-MovementEvent of regional (SET OF) (/home/timstirling/CS/CS/libraries/dsrcdecoder/src/constr_SET_OF.c:999)

This is not surprising given I haven't defined the Reg-MovementEvent . If I modify the ASN adding this:

Reg-MovementEvent REG-EXT-ID-AND-TYPE ::= {
{MovementEvent-addGrpC IDENTIFIED BY addGrpC} ,
...
}
MovementEvent-addGrpC ::= INTEGER

Then I get this error:

Member MovementEvent->regional is optional, p=1 (3->3) (/home/timstirling/CS/CS/libraries/dsrcdecoder/src/constr_SEQUENCE.c:1150)
Decoding member "regional" in MovementEvent (/home/timstirling/CS/CS/libraries/dsrcdecoder/src/constr_SEQUENCE.c:1170)
  [PER got  2<=1172 bits => span 390 +0[6..1176]:30 (1170) => 0x0] (/home/timstirling/CS/CS/libraries/dsrcdecoder/src/asn_bit_data.c:132)
Preparing to fetch 0+1 elements from regional (/home/timstirling/CS/CS/libraries/dsrcdecoder/src/constr_SET_OF.c:962)
SET OF Reg-MovementEvent decoding (/home/timstirling/CS/CS/libraries/dsrcdecoder/src/constr_SET_OF.c:981)
Decoding Reg-MovementEvent as SEQUENCE (UPER) (/home/timstirling/CS/CS/libraries/dsrcdecoder/src/constr_SEQUENCE.c:1098)
Decoding member "regionId" in Reg-MovementEvent (/home/timstirling/CS/CS/libraries/dsrcdecoder/src/constr_SEQUENCE.c:1170)
Decoding NativeInteger RegionId (UPER) (/home/timstirling/CS/CS/libraries/dsrcdecoder/src/NativeInteger.c:272)
Integer with range 8 bits (/home/timstirling/CS/CS/libraries/dsrcdecoder/src/INTEGER.c:637)
  [PER got  8<=1170 bits => span 398 +0[14..1176]:30 (1162) => 0x3] (/home/timstirling/CS/CS/libraries/dsrcdecoder/src/asn_bit_data.c:132)
Got value 3 + low 0 (/home/timstirling/CS/CS/libraries/dsrcdecoder/src/INTEGER.c:658)
NativeInteger RegionId got value 3 (/home/timstirling/CS/CS/libraries/dsrcdecoder/src/NativeInteger.c:288)
Freeing INTEGER as a primitive type (/home/timstirling/CS/CS/libraries/dsrcdecoder/src/asn_codecs_prim.c:125)
Decoding member "regExtValue" in Reg-MovementEvent (/home/timstirling/CS/CS/libraries/dsrcdecoder/src/constr_SEQUENCE.c:1170)
Getting open type MovementEvent-addGrpC... (/home/timstirling/CS/CS/libraries/dsrcdecoder/src/per_opentype.c:83)
  [PER got  8<=1162 bits => span 406 +1[14..1168]:0c (1154) => 0x1] (/home/timstirling/CS/CS/libraries/dsrcdecoder/src/asn_bit_data.c:132)
  [PER got  8<=1154 bits => span 414 +2[14..1160]:05 (1146) => 0x46] (/home/timstirling/CS/CS/libraries/dsrcdecoder/src/asn_bit_data.c:132)
Getting open type MovementEvent-addGrpC encoded in 1 bytes (/home/timstirling/CS/CS/libraries/dsrcdecoder/src/per_opentype.c:108)
    Decoding NativeInteger MovementEvent-addGrpC (UPER) (/home/timstirling/CS/CS/libraries/dsrcdecoder/src/NativeInteger.c:272)
    Decoding unconstrained integer MovementEvent-addGrpC (/home/timstirling/CS/CS/libraries/dsrcdecoder/src/INTEGER.c:669)
      [PER got  8<= 8 bits => span 8 +0[8..8]:46 (0) => 0x46] (/home/timstirling/CS/CS/libraries/dsrcdecoder/src/asn_bit_data.c:132)
    Freeing INTEGER as a primitive type (/home/timstirling/CS/CS/libraries/dsrcdecoder/src/asn_codecs_prim.c:125)
Freeing MovementEvent-addGrpC as INTEGER (2, 0x10f1c00, Native) (/home/timstirling/CS/CS/libraries/dsrcdecoder/src/NativeInteger.c:428)
Failed decode regExtValue in Reg-MovementEvent (/home/timstirling/CS/CS/libraries/dsrcdecoder/src/constr_SEQUENCE.c:1179)
regional SET OF Reg-MovementEvent decoded 2, 0x10f1bf0 (/home/timstirling/CS/CS/libraries/dsrcdecoder/src/constr_SET_OF.c:984)
Failed decoding Reg-MovementEvent of regional (SET OF) (/home/timstirling/CS/CS/libraries/dsrcdecoder/src/constr_SET_OF.c:999)
Freeing Reg-MovementEvent as SEQUENCE (/home/timstirling/CS/CS/libraries/dsrcdecoder/src/constr_SEQUENCE.c:994)
Freeing RegionId as INTEGER (1, 0x10f1bf0, Native) (/home/timstirling/CS/CS/libraries/dsrcdecoder/src/NativeInteger.c:428)
Freeing regExtValue as CHOICE (/home/timstirling/CS/CS/libraries/dsrcdecoder/src/constr_CHOICE.c:1234)
Failed decode regional in MovementEvent (/home/timstirling/CS/CS/libraries/dsrcdecoder/src/constr_SEQUENCE.c:1179)

I am wondering if the type MovementEvent-addGrpC is either a constrained integer or an emum , and with uper decoding it is parsing the wrong number of bits? Sadly it is proving hard to get hold of this regional definition.

timstirling commented 4 years ago

OK, That last minute brainstorm was correct. Just goes to show that sometimes explaining the coding problem is all you need to solve it yourself!

I set a size constrained integer

MovementEvent-addGrpC ::= INTEGER(0 .. 100)

And it worked. Probasbly this is an enum which uper encoding case store in 1 byte? In which case INTEGER(0 .. 255) will be safe.

velichkov commented 4 years ago

Hi @timstirling,

It's great that you've been able to identify the problem and found a solution. I'm not familiar with DSRC protocols and I can't tell what the definition of MovementEvent-addGrpC should be.

Looking at the asn1c's output and comparing it with the one you've provided in your first post the lower bound should be 11 and not 0

WARNING: Parameterized type REG-EXT-ID-AND-TYPE expected for REG-EXT-ID-AND-TYPE at line 272 in /home/timstirling/dsrc2/ITS/ISO_TS_19091.asn WARNING: Parameterized type REG-EXT-ID-AND-TYPE expected for REG-EXT-ID-AND-TYPE at line 273 in /home/timstirling/dsrc2/ITS/ISO_TS_19091.asn

You could ignore these warnings, most of the time they are harmless.

velichkov commented 4 years ago

Looking at the asn1c's output and comparing it with the one you've provided in your first post the lower bound should be 11 and not 0

e.g it should be 

MovementEvent-addGrpC ::= INTEGER(11 .. 255)

I don't know about the upper bound.

See ITU-T X.691

13 Encoding the integer type
13.2.2 If PER-visible constraints restrict the integer value to be a constrained whole number, then it shall be converted to a field according to the procedures of 11.5 (encoding of a constrained whole number), and the procedures of 13.2.5 to 13.2.6 shall then be applied.

11.5 Encoding of a constrained whole number
11.5.6 In the case of the UNALIGNED variant the value ("n" – "lb") shall be encoded as a non-negative-binary-integer in a bit-field as specified in 11.3 with the minimum number of bits necessary to represent the range.
NOTE – If "range" satisfies the inequality 2^m < "range" <= 2^m+1 , then the number of bits = m+1.
timstirling commented 4 years ago

Thanks so much for your help. Interesting that you state it should be >= 11. The only documentation I have lists example values form 0 to 12. which is provably incomplete in any case as the most common values appear to be 35 and 46.