P1sec / pycrate

A Python library to ease the development of encoders and decoders for various protocols and file formats; contains ASN.1 and CSN.1 compilers.
GNU Lesser General Public License v2.1
381 stars 132 forks source link

[ASN.1 ITS CAM] invalid undef count value, 63 #156

Closed coscar68 closed 2 years ago

coscar68 commented 2 years ago

I have tried to decode the string below (CAM message). It produces an 'Error: invalid undef count value, 63' message. This string is decoded without any issues by online decoders.

02020000D900B1E74059D824554CC4C2D79FFFFFFC2230D41E58622FC0000082B88A800FFD01FFF8807FE013C0400009FFFF7FFFD8CE00

p1-bmu commented 2 years ago

What version of the CAM spec are you using ? The one embedded in the library is quite old, I guess. Moreover, please provide more contextual information (how is your program made that leads to this error, what version of Python and pycrate are you using...). Otherwise, I can't help you.

coscar68 commented 2 years ago

I guess I need to use ETSI CAM EN 302 637-2 v1.4.1

p1-bmu commented 2 years ago

So, what are you expecting from this issue ?

coscar68 commented 2 years ago

I would like to know if the library is going to be updated for the latest CAM release (v1.4.1)

p1-bmu commented 2 years ago

If you need such a version, you are very welcome to help: pycrate is open-source, PRs are welcome. Moreover, it would benefit to all other people requiring more recent ITS ASN1 specs.

Now more specifically to your error that leaded to this issue, as you did not provide any additional contextual information (as requested): I still can't help you.

coscar68 commented 2 years ago

Ok, I will see if it is possible that I help, and will try to provide teh contextual information that you request

coscar68 commented 2 years ago

Python 3.6. Pycrate uploaded to current version. Basically, I copy the above string (CAM message in a string), asndata_message, and use the following

from pycrate_asn1dir import ITS from binascii import unhexlify

m = ITS.CAM_PDU_Descriptions.CAM m.from_uper(unhexlify(asndata_message))

coscar68 commented 2 years ago

I have compared the definitions of ETSI CAM standard (v1.4.1) with the ones existing in the pycrate library (file pycrate_asn1dir\ETSI_ITS_r1318\CAM.asn), and they are exactly the same (just changes in a version number: itu-t (0) identified-organization (4) etsi (0) itsDomain (5) wg1 (1) ts (102894) cdd (2) version (2))

p1-bmu commented 2 years ago

When I try to decode your message buffer with Marben, I get an error:

V2X message: ETSI pre-2018 CAM EN 302 637-2 v1.3.2
Filename: cam_msg_hex.txt

*** DECODING ***
<encoding>
02020000 D900B1E7 4059D824 554CC4C2 D79FFFFF FC2230D4 1E58622F C0000082
B88A800F FD01FFF8 
</encoding>

<CAM>
  <header>
    <protocolVersion>2</protocolVersion>
    <messageID>2</messageID>
    <stationID>55552</stationID>
  </header>
  <cam>
    <generationDeltaTime>45543</generationDeltaTime>
    <camParameters>
      <basicContainer>
        <stationType>5</stationType>
        <referencePosition>
          <latitude>421280170</latitude>
          <longitude>-86227780</longitude>
          <positionConfidenceEllipse>
            <semiMajorConfidence>4095</semiMajorConfidence>
            <semiMinorConfidence>4095</semiMinorConfidence>
            <semiMajorOrientation>3601</semiMajorOrientation>
          </positionConfidenceEllipse>
          <altitude>
            <altitudeValue>0</altitudeValue>
            <altitudeConfidence>
              <unavailable/>
            </altitudeConfidence>
          </altitude>
        </referencePosition>
      </basicContainer>
      <highFrequencyContainer>
        <basicVehicleContainerHighFrequency>
          <heading>
            <headingValue>1570</headingValue>
            <headingConfidence>127</headingConfidence>
          </heading>
          <speed>
            <speedValue>0</speedValue>
            <speedConfidence>1</speedConfidence>
          </speed>
          <driveDirection>
            <unavailable/>
          </driveDirection>
          <vehicleLength>
            <vehicleLengthValue>44</vehicleLengthValue>
            <vehicleLengthConfidenceIndication>
              <unavailable/>
            </vehicleLengthConfidenceIndication>
          </vehicleLength>
          <vehicleWidth>18</vehicleWidth>
          <longitudinalAcceleration>
            <longitudinalAccelerationValue>0</longitudinalAccelerationValue>
            <longitudinalAccelerationConfidence>1</longitudinalAccelerationConfidence>
          </longitudinalAcceleration>
          <curvature>
            <curvatureValue>35440</curvatureValue>
            <curvatureConfidence>
              <onePerMeter-0-0001/>
            </curvatureConfidence>
          </curvature>
          <curvatureCalculationMode>
            <yawRateUsed/>
          </curvatureCalculationMode>
          <yawRate>
            <!-- Mandatory Field Missing -->
          </yawRate>
        </basicVehicleContainerHighFrequency>
      </highFrequencyContainer>
    </camParameters>
  </cam>
</CAM>

<error>
  <description>the capacity of the Java long type is exceeded</description>
  <nature>fr.marben.asnsdk.japi.LimitationException</nature>
  <ErrorOffset>40</ErrorOffset>
  <ValuePath>CAM.cam.camParameters.highFrequencyContainer.basicVehicleContainerHighFrequency.curvatureCalculationMode</ValuePath>
</error>

*** DECODING FAILED ***

The error seems to lie straight after the curvatureCalculationMode object. When I decode the buffer with pycrate, I get an error while decoding this curvatureCalculationMode object, where the UPER value for an ENUMERATED content seems invalid:

ASN1PERDecodeErr: invalid undef count value, 63

In [13]: %debug                                                                                                                                           
> /home/mich/python/pycrate_asn1rt/codecs.py(133)decode_count()
    131                     return cla._CntUndef_LUT[cnt]
    132                 except KeyError:
--> 133                     raise(ASN1PERDecodeErr('invalid undef count value, {0}'.format(cnt)))
    134             else:
    135                 cnt = char.get_uint(14)

ipdb> cnt                                                                                                                                                 
63
ipdb> cla._CntUndef_LUT                                                                                                                                   
{1: 16384, 2: 32768, 3: 49152, 4: 65536, 16384: 1, 32768: 2, 49152: 3, 65536: 4}
ipdb> u                                                                                                                                                   
> /home/mich/python/pycrate_asn1rt/codecs.py(228)decode_intunconst()
    226             cla.decode_pad(char)
    227         # 1) get byte-length determinant
--> 228         ldet = cla.decode_count(char)
    229         # 2) get value, byte-aligned
    230         if ldet == 0:

ipdb> u                                                                                                                                                   
> /home/mich/python/pycrate_asn1rt/asnobj_basic.py(1342)_from_per()
   1340                 if big:
   1341                     # 2) not-small index value (>= 64)
-> 1342                     ind = ASN1CodecPER.decode_intunconst(char, 0)
   1343                 else:
   1344                     # 3) normally-small index value (< 64)

ipdb> self                                                                                                                                                
<curvatureCalculationMode ([CurvatureCalculationMode] ENUMERATED)>

Please indicate where you could decode this buffer successfully.

coscar68 commented 2 years ago

https://www.marben-products.com/decoder-asn1-automotive/

coscar68 commented 2 years ago

output.txt

p1-bmu commented 2 years ago

So I suppose we need to update pycrate's CAM (and ITS-Container) ASN.1 definition to ETSI CAM EN 302 637-2 v1.4.1, as even Marben cannot decode correctly based on ETSI pre-2018 CAM EN 302 637-2 v1.3.2, and pycrate's ASN.1 definition is more than 4 years old ?

coscar68 commented 2 years ago

As I said I have compared with 1.4.1 and the definitions seem to be exactly the same 141.txt

coscar68 commented 2 years ago

I checked that in ETSI TS 102 894-2 V1.3.1 (2018-08) (referenced in v1.4.1) CurvatureCalculationMode ::= ENUMERATED {yawRateUsed(0), yawRateNotUsed(1), unavailable(2),... }

exactly the same than in previous ETSI TS 102 894-2 V1.2.1 (2014-09) CurvatureCalculationMode ::= ENUMERATED {yawRateUsed(0), yawRateNotUsed(1), unavailable(2),... }

p1-bmu commented 2 years ago

And what about all the other objects ? And the dependencies (imported modules) ? Generally, when the version in the module OID is bumped, this means something changed and may not be backward-compatible.

coscar68 commented 2 years ago

You are right. The curvature value has changed. New value is

CurvatureValue ::= INTEGER {straight(0), unavailable(1023)} (-1023..1023)

I attach the correct ASN.1 module of the common data dictionary dictionary.txt

p1-bmu commented 2 years ago

I updated the compiler last week, for some specific cases, and introduced some of the more recent ETSI ITS ASN.1 modules, including the CAM one. It seems now to work smoothly to decode your message:

In [1]: from pycrate_asn1dir.ITS_CAM_2 import *                                                                                                           

In [2]: m = CAM_PDU_Descriptions.CAM                                                                                                                      

In [3]: m.from_uper(unhexlify('02020000D900B1E74059D824554CC4C2D79FFFFFFC2230D41E58622FC0000082B88A800FFD01FFF8807FE013C0400009FFFF7FFFD8CE00'))          

In [4]: print(m.to_asn1())                                                                                                                                
{
  header {
    protocolVersion 2,
    messageID 2 -- cam --,
    stationID 55552
  },
  cam {
    generationDeltaTime 45543,
    camParameters {
      basicContainer {
        stationType 5 -- passengerCar --,
        referencePosition {
          latitude 421280170,
          longitude -86227780,
          positionConfidenceEllipse {
            semiMajorConfidence 4095 -- unavailable --,
            semiMinorConfidence 4095 -- unavailable --,
            semiMajorOrientation 3601 -- unavailable --
          },
          altitude {
            altitudeValue 0 -- referenceEllipsoidSurface --,
            altitudeConfidence unavailable
          }
        }
      },
      highFrequencyContainer basicVehicleContainerHighFrequency : {
        heading {
          headingValue 1570,
          headingConfidence 127 -- unavailable --
        },
        speed {
          speedValue 0 -- standstill --,
          speedConfidence 1 -- equalOrWithinOneCentimeterPerSec --
        },
        driveDirection unavailable,
        vehicleLength {
          vehicleLengthValue 44,
          vehicleLengthConfidenceIndication unavailable
        },
        vehicleWidth 18,
        longitudinalAcceleration {
          longitudinalAccelerationValue 0,
          longitudinalAccelerationConfidence 1 -- pointOneMeterPerSecSquared --
        },
        curvature {
          curvatureValue 1022,
          curvatureConfidence onePerMeter-0-00002
        },
        curvatureCalculationMode yawRateUsed,
        yawRate {
          yawRateValue 0 -- straight --,
          yawRateConfidence degSec-000-10
        },
        accelerationControl '0000000'B --  --,
        steeringWheelAngle {
          steeringWheelAngleValue 512 -- unavailable --,
          steeringWheelAngleConfidence 1 -- equalOrWithinOnePointFiveDegree --
        },
        lateralAcceleration {
          lateralAccelerationValue -2,
          lateralAccelerationConfidence 1 -- pointOneMeterPerSecSquared --
        }
      },
      lowFrequencyContainer basicVehicleContainerLowFrequency : {
        vehicleRole default,
        exteriorLights '00'H --  --,
        pathHistory {
          {
            pathPosition {
              deltaLatitude 0,
              deltaLongitude 0,
              deltaAltitude 0
            }
          }
        }
      }
    }
  }
}
coscar68 commented 2 years ago

Hi, I have downloaded current version, and run it again but I get the same error. I do not know if I am missing something.

p1-bmu commented 2 years ago

You certainly have still an old install somewhere.