Input Processing Error

[ForceFaction]

Hi, i'm trying to decode a FEF file which is specified here: http://www.cs.tut.fi/~varri/tc251/pt40/4002n004.zip. I'm using asn1c from git 88ed3b5c.

There is an extracted Version of the ASN.1 DEFINITIONS here: https://github.com/donnchadh/biosig/blob/master/biosig4c%2B%2B/t241/CENTS14271-2003.asn1.

I added the following because it was missing:

BITS-64 ::= BIT STRING (SIZE (64))

Preamble ::= SEQUENCE {
    magic      BITS-64,
    version    BITS-64,
    encoding   BITS-64,
    endianness BITS-64
}

FEF ::= SEQUENCE {
    preamble Preamble,
    sas      SessionArchiveSection
}

I generated the parser with asn1c -fcompound-names -fbless-SIZE -pdu=FEF definition/fef.asn1.

I get the following Error:

$ ./converter-example -dd -iber -onull 10_b.fef
(Suppressing binary output to a terminal.)
AD: Processing 10_b.fef
AD: Decoding 8192 bytes
AD: decode(0) consumed 0+0b (8192), code 2
AD: Clean up partially decoded FEF
AD: ofp 1, no=0, oo=0, dbl=0
10_b.fef: Decode failed past byte 0: Input processing error

I'm new to asn1c and ASN.1 in general, so i'm not sure how to interpret this. Does it mean my input file does not correspond to the FEF format or did i make an error using asn1c or is it asn1c which isn't working as expected ?

What am i doing wrong ? Any help appreciated :D

Best Regards forcefaction

[ForceFaction]

I just had the time to experiment further and it turns out that my additions are worng. I just need to set the pdu to SessionArchiveSection.

Now the generated Parser manages to consume 7 bytes, but stops with the same error again.

I tested with python and asn1tools, because it gives a little better error output (it tells what exactly it expected to find and what it actually found). It seems that it wanted to see a 30 and got a 5f. But in the Definition

SessionArchiveSection ::= [APPLICATION 7001] SEQUENCE {
    handle               Handle,
    s-archive-id         [APPLICATION 2507] FEFString,
    s-archive-name       [APPLICATION 2513] FEFString,
    s-archive-comments   [APPLICATION 2530] FEFString OPTIONAL,
    starttime            [APPLICATION 2538] AbsoluteTime,
    stoptime             [APPLICATION 2539] AbsoluteTime,
    protection           [APPLICATION 2519] ArchiveProtection OPTIONAL,
    placeholder          Placeholder OPTIONAL,
    manufacturerspecific [APPLICATION 7002] ManufacturerSpecificSection OPTIONAL,
    healthcareprovider   [APPLICATION 7003] HealthCareProviderSection OPTIONAL,
    demographics         [APPLICATION 7004] PatientDemographicsSection,
    sessions             [APPLICATION 7015] SEQUENCE SIZE (1..MAX) OF SessionTestSection,
    notes                [APPLICATION 7016] SEQUENCE OF SessionNotesSection DEFAULT {}
}

The Sequence gets the Tag APPLICATION 7001 which translates to 0x76bf59, but the next identifier is 0x5f9221 and pyasn expexts 0x30 and asn1c also stops here. I don't understand.

[velichkov]

Hi @ForceFaction,

There is an extracted Version of the ASN.1 DEFINITIONS here: https://github.com/donnchadh/biosig/blob/master/biosig4c%2B%2B/t241/CENTS14271-2003.asn1.

This does not compile.

asn1c -fcompound-names -fbless-SIZE -pdu=FEF CENTS14271-2003.asn1 
ASN.1 grammar parse error near CENTS14271-2003.asn1:58 (token "SIZE"): syntax error, unexpected TOK_SIZE, expecting TOK_END
Cannot parse "CENTS14271-2003.asn1"

I generated the parser with asn1c -fcompound-names -fbless-SIZE -pdu=FEF definition/fef.asn1.

Could you provide definition/fef.asn1 and all the other asn1 files that are needed to compile it?

I get the following Error: 10_b.fef: Decode failed past byte 0: Input processing error

Also could you provide 10_b.fef? Are you sure that this file contains a SessionArchiveSection PDU? In order to get some more debug logs add -DASN_EMIT_DEBUG=1 to the CFLAGS and clean and recompile the converter-example.

[velichkov]

The Sequence gets the Tag APPLICATION 7001 which translates to 0x76bf59, but the next identifier is 0x5f9221 and pyasn expexts 0x30 and asn1c also stops here. I don't understand.

Try adding IMPLICIT TAGS between DEFINITIONS and ::=

[ForceFaction]

Here the asn file :). fef.txt

Ah i added the IMPLICIT TAGS you suggested (now i need to google what this means). It now runs through the whole file and says decoded succesfully. :).

Thanks for looking at it @velichkov

[ForceFaction]

A little file to play with. [removed]

I just need to ask you one more thing @velichkov: I took the asn file to Nokalvas Online Compiler and it gives the following Error:

line 659 (FEF-Module): A0174S: The element 'enumadditionaldata' has an ANY DEFINED BY identifier 'enumobservedvalue' which is not an INTEGER, an OBJECT IDENTIFIER, or ENUMERATED.

This seems to lead to some files not being decoded by asn1c. I set -DASN_EMIT_DEBUG=1 as you suggested and the log says

Inside SEQUENCE EnumerationMeasuredDataSection MF2 (constr_SEQUENCE.c:384)
Decoding ChoiceOfEnumTimeStamp as CHOICE (constr_CHOICE.c:127)
Structure consumes -1 bytes, buffer 32 (constr_CHOICE.c:176)

shortly after that the parser stops. So how do i fix the above error ? Do you have any hints?

Thank you in advance :).

[velichkov]

line 659 (FEF-Module): A0174S: The element 'enumadditionaldata' has an ANY DEFINED BY identifier 'enumobservedvalue' which is not an INTEGER, an OBJECT IDENTIFIER, or ENUMERATED.

That's because the enumobservedvalue is of type ChoiceOfEnumObsValue that is a CHOICE and most probably is not supported by their compiler. I'm not sure if this is valid definition and whether it is supported by asn1c or not.

shortly after that the parser stops.

Could you share the binary fef file you are testing with together with the exact command and all parameters.

So how do i fix the above error ? Do you have any hints?

If you don't need the enumadditionaldata you could rewrite the definition to

EnumerationMeasuredDataSection ::= SEQUENCE {
    metricref          [APPLICATION 6050] HandleRef,
    enumobservedvalue  ChoiceOfEnumObsValue,
    enumadditionaldata [APPLICATION 2498] ANY OPTIONAL,
    enumtimestamp      ChoiceOfEnumTimeStamp
}

[velichkov]

Could you share the binary fef file

I just saw the demo.txt file, the github does not send emails when you edit a message so such changes are easy to miss.

Are all SessionArchiveSection usually that big and how many PDUs (records) are in this file? By default ./converter-example uses a 8192 bytes buffer so when the PDU is bigger than that it's better to specify a larger buffer - ./converter-example -b 4639581 -iber demo.txt

[velichkov]

--- definition/fef.asn1.orig    2019-02-26 01:53:50.839557329 +0200
+++ definition/fef.asn1 2019-02-26 02:02:10.037219569 +0200
@@ -657,7 +657,8 @@
     metricref          [APPLICATION 6050] HandleRef,
     enumobservedvalue  ChoiceOfEnumObsValue,
     enumadditionaldata [APPLICATION 2498] ANY DEFINED BY enumobservedvalue OPTIONAL,
-    enumtimestamp      ChoiceOfEnumTimeStamp
+    -- enumtimestamp      ChoiceOfEnumTimeStamp
+    ...
 }

After applying this it gets decoded, I suspect that some of the enumtimestamp (or its sub-parameters) are not encoded correctly.

[ForceFaction]

The ASN.1 Specification itself is extremely old and the format didn't get as much adopted as the authors wanted, so it is possible that it contains errors.

The Software which generates the files is also a bit old, so it is possible that it incorrectly encodes the timestamp. :(

Regarding the PDU size, the example file i gave you is a small example. The file and therefore the SessionArchiveSection is normally ~100 - ~200 MiB in size. asn1c does not use a buffer greater than 16M ... I noticed that the converter-example manages to consume more bytes if i choose a smaller buffer and less if it gets larger. That's normal behaviour ?

At least the generated parser now parses every file i'm interested in :D. Thank you.

[velichkov]

The ASN.1 Specification itself is extremely old and the format didn't get as much adopted as the authors wanted, so it is possible that it contains errors.

It turns out the the definition of ObservationTime in fef.asn1 and CENTS14271-2003.asn1 was modified in an incompatible way - the OPTIONAL flag was removed and commented.

--- definition/fef.asn1.orig    2019-02-26 01:53:50.839557329 +0200
+++ definition/fef.asn1 2019-02-26 12:55:51.607896463 +0200
@@ -57,9 +57,9 @@
 HighResRelativeTime ::= INT-U64

 ObservationTime ::= SEQUENCE {
-    absolutetimestamp      [APPLICATION 2448] AbsoluteTime,
-    relativetimestamp      [APPLICATION 2449] RelativeTime,
-    hirestimerelativestamp [APPLICATION 2537] HighResRelativeTime
+    absolutetimestamp      [APPLICATION 2448] AbsoluteTime OPTIONAL,
+    relativetimestamp      [APPLICATION 2449] RelativeTime OPTIONAL,
+    hirestimerelativestamp [APPLICATION 2537] HighResRelativeTime OPTIONAL
 }

 SimpleColour ::= INTEGER {

Without OPTIONAL flag the BER decoder expects to find all three parameters and when it find only the first one it fails.

asn1c does not use a buffer greater than 16M

The converter-example has a limit of 16M but it is just an example program and you could modify it to suite your needs.

I noticed that the converter-example manages to consume more bytes if i choose a smaller buffer and less if it gets larger. That's normal behaviour ?

$ ./converter-example -iber -oxer demo.txt 
demo.txt: Decode failed past byte 4636672: Input processing error
$ ./converter-example -b 4639581 -iber -oxer demo.txt 
demo.txt: Decode failed past byte 312: Input processing error

It seems to consume the same amount of bytes but in the error message reports a different offset for some reason. It's probably a bug but right now I can't spend time debugging this.

Please close this issue if you have no more questions.

[velichkov]

With the previous modifications reverted and the patch adding the OPTIONAL flag is applied it decodes demo.txt successfully.

[ForceFaction]

A big Thank You for investing your time :-).