vlm / asn1c

The ASN.1 Compiler
http://lionet.info/asn1c/
BSD 2-Clause "Simplified" License
1.05k stars 559 forks source link

PER encoder does not return buffer size when called with null write_out() #137

Closed mouse07410 closed 7 years ago

mouse07410 commented 7 years ago

Expected behavior

Similar to DER encoder, I'd expect the call to

asn_enc_rval_t ret = uper_encode(&asn_DEF_Test, test_pdu, 0, 0);

provide me with the exact size of the buffer required to UPER-encode the given PDU, which I would later convert to bytes like size_t bytes_needed = (ret.encoded + 7)/8;

Actually happens

asn_enc_rval_t ret = uper_encode(&asn_DEF_Test, test_pdu, 0, 0);

causes SEGV (application crashes with segmentation fault, trying to encode octet string).

If PER encoder is not supposed to behave like DER encoder in this aspect - what's the correct way of determining the size of the buffer to hold the UPER-encoded PDU?

I'm aware of uper_encode_to_new_buffer(), and it works fine. But it is documented that this function uses "complete" encoding, and therefore may take more space than necessary.

What's the correct way of using uper_encode_to_buffer()? How do I know how big a buffer to pre-allocate?

And as we're at it, asn1c-usage.pdf mentions _writestream() callback, but never actually shows how to use/write it. What's the example? What are the parameters (const void *buffer, size_t size, app_key)? Just the number of bytes in the given buffer that represent the encoded value?

xeirwn commented 7 years ago

What write_stream is can be found in the manual under the name write_out. This callback receives as input the pointer to an element (const void *buffer), the size of that element (size_t size) and some context (void *app_key).

/* Dump the buffer out to the specified FILE */
static int write_out(const void *buffer, size_t size, void *key) {
  FILE *fp = (FILE *)key;
  return (fwrite(buffer, 1, size, fp) == size) ? 0 : -1;
}

In this example, we can see that the user is using der_encode() which accepts a FILE * as the last parameter, which later is passed to write_out() as the context.

mouse07410 commented 7 years ago

What write_stream is can be found in the manual under the name write_out.

Thank you! After you showed the example (and the correct name), I was able to locate it under "A 'Rectangle' Encoder" in "Step-by-step examples" section.

This callback receives as input the pointer to an element

A pointer to the encoded element (seems obvious, but I want to be "explicitly" sure :). Correct?

Also, do you know if there's a way to determine the required size of the buffer to allocate for the PER-encoded PDU? (APER and/or UPER?)

mouse07410 commented 7 years ago

@vlm as you appear to be back, could you tell if you're planning to add a function/capability to determine the required size of the buffer to allocate for a PER-encoded PDU?

An encoder would really love to know how big a buffer to allocate for a given PDU, and you already provide that capability at least for DER.

@velichkov might you know the answer here?

vlm commented 7 years ago

The buffer size can be estimated by invoking per_encode without a buffer. It'll return size.

Same as with DER.

The reason is that in the dynamic world you pretty much have to serialize the data (even into /dev/null) to figure out the resulting size.

mouse07410 commented 7 years ago

The buffer size can be estimated by invoking per_encode without a buffer. It'll return size.

Could you please be a bit more specific? Should I be able to call

  asn_enc_rval_t er9 = uper_encode(&asn_DEF_Test2b, der_dec, NULL, NULL);
  if (er9.encoded < 1) {
     fprintf(stderr,
        "Failed to determine the buffer size for UPER-encoded Test2b_t PDU... (rc=%ld)\n",
        er9.encoded);
   } else {
       printf("UPER-encoded Test2b_t PDU requires buffer of %ld bits\n", er9.encoded);
   }

Update. It crashes with SEGV:

. . . . .
Process 8909 stopped
* thread #1: tid = 0x328ffa, 0x0000000000000000, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
    frame #0: 0x0000000000000000
error: memory read failed for 0x0
(lldb) bt
warning: could not load any Objective-C class information. This will significantly reduce the quality of type information available.
* thread #1: tid = 0x328ffa, 0x0000000000000000, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
  * frame #0: 0x0000000000000000
    frame #1: 0x0000000100010f91 m1ba`per_put_few_bits(po=0x00007fff5fbfeaa0, bits=<unavailable>, obits=24) + 145 at per_support.c:444 [opt]
    frame #2: 0x000000010001138f m1ba`per_put_many_bits(po=<unavailable>, src="??\x9b??Q,ao?, nbits=80) + 79 at per_support.c:572 [opt]
    frame #3: 0x000000010000898f m1ba`OCTET_STRING_encode_uper(td=<unavailable>, constraints=<unavailable>, sptr=<unavailable>, po=<unavailable>) + 575 at OCTET_STRING.c:1789 [opt]
    frame #4: 0x000000010000e3fa m1ba`SEQUENCE_encode_uper(td=<unavailable>, constraints=<unavailable>, sptr=<unavailable>, po=<unavailable>) + 682 at constr_SEQUENCE.c:1663 [opt]
    frame #5: 0x000000010000fba1 m1ba`uper_encode_internal(td=0x0000000100014af0, constraints=<unavailable>, sptr=0x0000000100200320, cb=<unavailable>, app_key=<unavailable>) + 129 at per_encoder.c:201 [opt]
    frame #6: 0x000000010000fb0c m1ba`uper_encode(td=<unavailable>, sptr=<unavailable>, cb=<unavailable>, app_key=<unavailable>) + 28 at per_encoder.c:11 [opt]
    frame #7: 0x000000010000179f m1ba`try_string(input_str="?o5??7?_??,aj?{\x8b\"?-;??\x02s}\x80??\x9b??Q,ao??8'\x8e?, len=40) + 1663 at m1b.c:154
    frame #8: 0x0000000100001fec m1ba`main(argc=2, argv=0x00007fff5fbfef28) + 236 at m1b.c:306
    frame #9: 0x00007fff8ac275ad libdyld.dylib`start + 1
    frame #10: 0x00007fff8ac275ad libdyld.dylib`start + 1
(lldb) 

And the line in question (m1ba.c line 154) is

asn_enc_rval_t er9 = uper_encode(&asn_DEF_Test2b, der_dec, NULL, NULL);

@vlm Could you please show a complete function call? What function exactly should I call, and what parameters exactly (besides the type descriptor pointer and the structure to-be-encoded pointer)?

P.S. Attempt to call

  er9 = uper_encode_to_buffer(&asn_DEF_Test2b, der_dec, NULL, 0);

results in -1 returned in er9.encoded (but at least it does not crash).

P.P.S. Needless to say, this

    er = der_encode(&asn_DEF_Test2b, t_p, 0, 0);

works perfectly. So the problem seems to be in PER encoder(s) that (unlike DER and XER encoders) cannot handle empty buffer.

mouse07410 commented 7 years ago

Maybe this output would be more informative:

. . . . .
Successfully BER-decoded Test2b_t PDU (47 bytes)
Validated constraints on BER-decoded Test2b_t PDU
Visual check:
<Test2b>
    <count>40</count>
    <data>
        C3 40 CB 01 95 F1 CE B7 75 6B 43 59 BC FE 51 3F 
        98 B1 E3 05 BC B2 22 FE 9D 5A A7 AD 64 1B C2 08 
        3B E9 ED 03 06 F8 0E B7
    </data>
</Test2b>

Encoding Test2b as SEQUENCE (UPER) (constr_SEQUENCE.c:1579)
ext_after = -1, ec = 2, eb = -1 (constr_SEQUENCE.c:1629)
About to encode INTEGER (constr_SEQUENCE.c:1640)
Encoding Test2b->count (constr_SEQUENCE.c:1662)
Encoding NativeInteger INTEGER 40 (UPER) (NativeInteger.c:334)
Value 40 (28/1) lb 40 ub 20000 fix (INTEGER.c:876)
Encoding integer 40 (0) with range 15 bits (INTEGER.c:892)
[PER put 15 bits 0 to 0x7fff5fbfec98+0 bits] (per_support.c:426)
[PER out 15 0/0 (t=0,o=15) 27&0=0] (per_support.c:467)
[PER out 0/0 => 00 buf+0] (per_support.c:498)
Freeing INTEGER as a primitive type (asn_codecs_prim.c:125)
About to encode OCTET STRING (constr_SEQUENCE.c:1640)
Encoding Test2b->data (constr_SEQUENCE.c:1662)
Encoding OCTET STRING into 40 units of 8 bits (40..20000, effective 15) (OCTET_STRING.c:1753)
Encoding 40 bytes (0), length in 15 bits (OCTET_STRING.c:1784)
[PER put 15 bits 0 to 0x7fff5fbfec98+15 bits] (per_support.c:426)
[PER out 15 0/0 (t=7,o=22) 0&fe=0] (per_support.c:467)
[PER out 0/0 => 00 buf+1] (per_support.c:498)
Squeezing 40 characters into (0..0):8 (1 bpc) (OCTET_STRING.c:1258)
[PER put 24 bits c340cb to 0x7fff5fbfec99+22 bits] (per_support.c:426)
[PER out 24 12796107/c340cb (t=6,o=30) 0&fc=0] (per_support.c:467)
[PER out 51184428/30d032c => 03 buf+3] (per_support.c:498)
[PER put 24 bits 195f1 to 0x7fff5fbfec9b+30 bits] (per_support.c:426)
[PER out 24 103921/195f1 (t=6,o=30) 2c&fc=2c] (per_support.c:467)
[PER out 415684/657c4 => 2c buf+6] (per_support.c:498)
[PER put 24 bits ceb775 to 0x7fff5fbfec9e+30 bits] (per_support.c:426)
[PER out 24 13547381/ceb775 (t=6,o=30) c4&fc=c4] (per_support.c:467)
[PER out 54189524/33addd4 => c7 buf+9] (per_support.c:498)
[PER put 24 bits 6b4359 to 0x7fff5fbfeca1+30 bits] (per_support.c:426)
[PER out 24 7029593/6b4359 (t=6,o=30) d4&fc=d4] (per_support.c:467)
[PER out 28118372/1ad0d64 => d5 buf+12] (per_support.c:498)
[PER put 24 bits bcfe51 to 0x7fff5fbfeca4+30 bits] (per_support.c:426)
[PER out 24 12385873/bcfe51 (t=6,o=30) 64&fc=64] (per_support.c:467)
[PER out 49543492/2f3f944 => 66 buf+15] (per_support.c:498)
[PER put 24 bits 3f98b1 to 0x7fff5fbfeca7+30 bits] (per_support.c:426)
[PER out 24 4167857/3f98b1 (t=6,o=30) 44&fc=44] (per_support.c:467)
[PER out 16671428/fe62c4 => 44 buf+18] (per_support.c:498)
[PER put 24 bits e305bc to 0x7fff5fbfecaa+30 bits] (per_support.c:426)
[PER out 24 14878140/e305bc (t=6,o=30) c4&fc=c4] (per_support.c:467)
[PER out 59512560/38c16f0 => c7 buf+21] (per_support.c:498)
[PER put 24 bits b222fe to 0x7fff5fbfecad+30 bits] (per_support.c:426)
[PER out 24 11674366/b222fe (t=6,o=30) f0&fc=f0] (per_support.c:467)
[PER out 46697464/2c88bf8 => f2 buf+24] (per_support.c:498)
[PER put 24 bits 9d5aa7 to 0x7fff5fbfecb0+30 bits] (per_support.c:426)
[PER out 24 10312359/9d5aa7 (t=6,o=30) f8&fc=f8] (per_support.c:467)
[PER out 41249436/2756a9c => fa buf+27] (per_support.c:498)
[PER put 24 bits ad641b to 0x7fff5fbfecb3+30 bits] (per_support.c:426)
[PER output 30 complete + 0] (per_support.c:443)
Process 15034 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
    frame #0: 0x0000000000000000
error: memory read failed for 0x0
(lldb) list
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
  * frame #0: 0x0000000000000000
    frame #1: 0x0000000100019a61 m1b`per_put_few_bits(po=0x00007fff5fbfec80, bits=<unavailable>, obits=<unavailable>) at per_support.c:444 [opt]
    frame #2: 0x000000010001a08f m1b`per_put_many_bits(po=0x00007fff5fbfec80, src="\b;??\x06??, nbits=80) at per_support.c:572 [opt]
    frame #3: 0x00000001000115e4 m1b`OCTET_STRING_encode_uper(td=<unavailable>, constraints=<unavailable>, sptr=<unavailable>, po=0x00007fff5fbfec80) at OCTET_STRING.c:1789 [opt]
    frame #4: 0x000000010000d25f m1b`SEQUENCE_encode_uper(td=<unavailable>, constraints=<unavailable>, sptr=0x0000000100102cd0, po=<unavailable>) at constr_SEQUENCE.c:1663 [opt]
    frame #5: 0x000000010001ac21 m1b`uper_encode_internal(td=0x0000000100020630, constraints=<unavailable>, sptr=0x0000000100102cd0, cb=<unavailable>, app_key=<unavailable>) at per_encoder.c:201 [opt]
    frame #6: 0x000000010001ab8c m1b`uper_encode(td=<unavailable>, sptr=<unavailable>, cb=<unavailable>, app_key=<unavailable>) at per_encoder.c:11 [opt]
    frame #7: 0x000000010001c54b m1b`try_string + 1435
    frame #8: 0x000000010001cd20 m1b`main + 240
    frame #9: 0x00007fff93c90235 libdyld.dylib`start + 1
    frame #10: 0x00007fff93c90235 libdyld.dylib`start + 1
(lldb) 
mouse07410 commented 7 years ago

@vlm...?

mouse07410 commented 7 years ago

Solved by @velichkov in my fork