shamblett / coap

A Coap package for dart
Other
16 stars 13 forks source link

Example tdls #145

Closed vincent-iQontrol closed 1 year ago

vincent-iQontrol commented 1 year ago

I need to connect to a coap server with a certificate X509 (secp256r1) There are 2 options: tinydtls and openssl. The readme is not sufficient enough to make it work.

JKRhb commented 1 year ago

Hi @vincent-iQontrol, you are right, there is definitely room for improvement. What have you tried so far?

For connecting to a CoAP server via DTLS, you need to use a config with OpenSSL defined as the DTLS backend. A minimal example could look like the listing below. For this to work, you need to have libssl.so available somewhere (on Linux, it should work by default). On Android, you probably need to include a plugin that ships the OpenSSL binary and makes it available to the Flutter application.

Unfortunately, the dtls library is not maintained anymore and also has some issues at times. Therefore, this part of the CoAP library must be considered highly experimental (while the PSK/RPK features provided by dart_tinydtls are a lot more stable already).

I hope I was able to help – otherwise, let me know if you have any more questions/encounter any problems!

The promised example:

class DtlsConfig extends DefaultCoapConfig {
  @override
  final dtlsBackend = DtlsBackend.OpenSsl;
}

FutureOr<void> main() async {
  final conf = DtlsConfig();
  final uri = Uri(
    scheme: 'coaps',
    host: 'californium.eclipseprojects.io',
    port: conf.defaultSecurePort,
  );
  final client = CoapClient(
    uri,
    config: conf,
  );

  try {
    print('Sending get /test to ${uri.host}');
    final response = await client.get('test');
    print('/test response: ${response.payloadString}');
  } on Exception catch (e) {
    print('CoAP encountered an exception: $e');
  }

  client.close();
}
vincent-iQontrol commented 1 year ago

Thank you @JKRhb for the explanation. I have a custom root ca certificate. In your example I cannot add the certificate.

JKRhb commented 1 year ago

Thank you @JKRhb for the explanation. I have a custom root ca certificate. In your example I cannot add the certificate.

Thank you for your feedback! The DTLS library does provide the feature of using a custom root certificate, however, this is currently not exposed in this library yet. I will open a PR for this feature later today or, at the latest, tomorrow – sorry for the inconvenience!

vincent-iQontrol commented 1 year ago

That would be nice @JKRhb. I have a .pem root certificate that is used.

ECDSA cipher secp256r1 curve

JKRhb commented 1 year ago

Can you maybe try out #146? :) Decoding of the .pem certificate should be possible via this package: https://pub.dev/packages/pem

vincent-iQontrol commented 1 year ago

Can you maybe try out #146? :) Decoding of the .pem certificate should be possible via this package: https://pub.dev/packages/pem

Thank you Jan

In the comments of the dtls package I see:

/// [verify] enables certificate verification (recommended). /// /// To allow the verification to succeed, system certificates have to be /// imported using [withTrustedRoots], or custom root certificates /// in DER format need to be imported with [rootCertificates]. /// System certificates are available only when OpenSSL is installed by /// the system. /// /// [ciphers] controls the cipher suites offered to the server.

So it has to be in DER format.

JKRhb commented 1 year ago

Oh, I see! If I see it correctly, you should be able to convert it from PEM to DER using OpenSSL (see, e.g., here).

vincent-iQontrol commented 1 year ago

I did a test with reading the certificates, it makes no difference between pem via https://pub.dev/packages/pem or der via rootBundle.load

The thing that could be a problem is that SSL_CTX_set1_curves_list is not implementd in the dtls package. Only SSL_CTX_set_cipher_list is implemented. Maybe I can fork and add the function.

P.s. I also use https://cocoapods.org/pods/OpenSSL-Universal for ios

vincent-iQontrol commented 1 year ago

@JosefWN did you get this example running?

FutureOr main() async { final conf = DtlsConfig(); final uri = Uri( scheme: 'coaps', host: 'californium.eclipseprojects.io', port: conf.defaultSecurePort, ); final client = CoapClient( uri, config: conf, );

try { print('Sending get /test to ${uri.host}'); final response = await client.get('test'); print('/test response: ${response.payloadString}'); } on Exception catch (e) { print('CoAP encountered an exception: $e'); }

client.close(); } I get only timeouts

JosefWN commented 1 year ago

I haven't been that involved in DTLS support I'm afraid, maybe you meant @JKRhb? He's the one who wrote the code.

vincent-iQontrol commented 1 year ago

@JosefWN excuse me wrong tag :) @JKRhb

JKRhb commented 1 year ago

@vincent-iQontrol I've got to admit that I haven't tried it out myself, and you are right, currently there is only a timeout error being thrown. Taking a closer look at what is actually being exchanged with the Californium test server using Wireshark, I noticed that there seems to be a bug in the dtls library at the moment that causes the send buffer to not being cleaned properly. I will try to come up with a fix for that :)

JKRhb commented 1 year ago

Hmm, okay, I did some investigation and came up with the following results:

So I think if you adjust the config like the following, you should be able to experiment a bit more. With prime256v1 it seems as if you can specify that you want to use secp256r1:

class DtlsConfig extends DefaultCoapConfig {
  @override
  final dtlsBackend = DtlsBackend.OpenSsl;

  @override
  final dtlsCiphers = "prime256v1";
}
vincent-iQontrol commented 1 year ago

Hi @JKRhb thank you. So i don't have to wait for the fix? I will try this tomorrow and let you know!

I did try yesterday with other ciphers like String? get dtlsCiphers => 'ECDSA_SECP256R1'; with no luck.

JKRhb commented 1 year ago

Hi @JKRhb thank you. So i don't have to wait for the fix? I will try this tomorrow and let you know!

Oh, I think you still need to use a forked version in order to be able to set the custom certificates, but other than that it could work I guess :)

I did try yesterday with other ciphers like String? get dtlsCiphers => 'ECDSA_SECP256R1'; with no luck.

If I see it correctly, OpenSSL is a bit strange when it comes to the naming of ciphers, at least in this regard. So in order to get secp256r1, apparently you need to use prime256v1.

vincent-iQontrol commented 1 year ago

@JKRhb unfortunately with String? get dtlsCiphers => 'prime256v1'; i get the following error: [log] error TlsException: error:141A90B5:SSL routines:ssl_cipher_list_to_bytes:no ciphers available

with String? get dtlsCiphers => 'ECDSA'; I get no SSL errors, but it is timing out also. Maybe the curve is not set correct automaticly? Or maybe also problem with the messages/buffers?

JKRhb commented 1 year ago

with String? get dtlsCiphers => 'prime256v1'; i get the following error: [log] error TlsException: error:141A90B5:SSL routines:ssl_cipher_list_to_bytes:no ciphers available

Hmm, that's unfortunate :/ Is this a client or a server issue, though? I.e., is it thrown after the first round of the handshake?

with String? get dtlsCiphers => 'ECDSA'; I get no SSL errors, but it is timing out also. Maybe the curve is not set correct automaticly? Or maybe also problem with the messages/buffers?

Hmm, yeah, it could be that with this cipher string too many values are included in the handshake. You could try having a look at the individual packets using Wireshark, I always find that quite handy for debugging.

vincent-iQontrol commented 1 year ago

@JKRhb
from wireshirk i get bad certificate after 3 Client hello

This is maybe strange "Version: DTLS 1.0 (0xfeff)" : Datagram Transport Layer Security DTLSv1.2 Record Layer: Handshake Protocol: Client Hello Content Type: Handshake (22) Version: DTLS 1.0 (0xfeff) Epoch: 0 Sequence Number: 3 Length: 225 Handshake Protocol: Client Hello Handshake Type: Client Hello (1) Length: 213 Message Sequence: 1 Fragment Offset: 0 Fragment Length: 213 Version: DTLS 1.2 (0xfefd) Random: 5a05d3d9d3cab8e07fa22d2518f26c3a8ac33869d4d7fd7f16dcd71679d5536f GMT Unix Time: Nov 10, 2017 17:29:13.000000000 CET Random Bytes: d3cab8e07fa22d2518f26c3a8ac33869d4d7fd7f16dcd71679d5536f Session ID Length: 0 Cookie Length: 32

I expect Version: DTLS 1.2 (0xfefd) Also the GMT Unix Time: Nov 10, 2017 is strange < 2022 but its random anyway

vincent-iQontrol commented 1 year ago

@JKRhb for testing purposes, is it possible to make ffi openssl bindings with:

I tried to generate these on my Mac without succes

vincent-iQontrol commented 1 year ago

@JKRhb I am little bit further, no certificate errors anymore, but still no data in coming in. Only error CoapRequestTimeoutException: Request timed out after 4 retransmits.

No. Time Source Destination Protocol Length Info 20 5.081209 10.0.10.24 10.0.10.56 DTLSv1.2 234 Client Hello 21 5.081263 10.0.10.56 10.0.10.24 DTLSv1.2 88 Hello Verify Request 22 5.081339 10.0.10.24 10.0.10.56 DTLSv1.2 266 Client Hello 25 6.084343 10.0.10.24 10.0.10.56 DTLSv1.2 266 Client Hello 30 8.084244 10.0.10.24 10.0.10.56 DTLSv1.2 266 Client Hello 31 8.084329 10.0.10.24 10.0.10.56 DTLSv1.2 148 Application Data 34 9.086234 10.0.10.56 10.0.10.24 DTLSv1.2 953 Server Hello, Certificate, Server Key Exchange, Server Hello Done 35 9.086342 10.0.10.56 10.0.10.24 DTLSv1.2 953 Server Hello, Certificate, Server Key Exchange, Server Hello Done 36 9.086410 10.0.10.56 10.0.10.24 DTLSv1.2 953 Server Hello, Certificate, Server Key Exchange, Server Hello Done 37 9.086512 10.0.10.56 10.0.10.24 DTLSv1.2 163 Application Data 38 9.086561 10.0.10.24 10.0.10.56 DTLSv1.2 262 Client Key Exchange, Change Cipher Spec, Encrypted Handshake Message 44 10.091463 10.0.10.24 10.0.10.56 DTLSv1.2 262 Client Key Exchange, Change Cipher Spec, Encrypted Handshake Message 49 11.090570 10.0.10.56 10.0.10.24 DTLSv1.2 103 Change Cipher Spec, Encrypted Handshake Message 50 11.090641 10.0.10.56 10.0.10.24 DTLSv1.2 103 Change Cipher Spec, Encrypted Handshake Message 51 11.090979 10.0.10.24 10.0.10.56 DTLSv1.2 148 Application Data 52 11.091789 10.0.10.56 10.0.10.24 DTLSv1.2 163 Application Data 73 13.100341 10.0.10.24 10.0.10.56 DTLSv1.2 148 Application Data 74 13.100410 10.0.10.56 10.0.10.24 DTLSv1.2 163 Application Data 100 17.110203 10.0.10.24 10.0.10.56 DTLSv1.2 148 Application Data 101 17.110500 10.0.10.56 10.0.10.24 DTLSv1.2 163 Application Data 128 25.127007 10.0.10.24 10.0.10.56 DTLSv1.2 148 Application Data 129 25.127566 10.0.10.56 10.0.10.24 DTLSv1.2 163 Application Data 209 41.166755 10.0.10.24 10.0.10.56 DTLSv1.2 148 Application Data 210 41.166797 10.0.10.56 10.0.10.24 DTLSv1.2 163 Application Data

JKRhb commented 1 year ago

@vincent-iQontrol That's awesome! Hmm, but it seems as if there is a CoAP message exchange, right? Maybe the incoming responses cannot be processed/matched for some reason?

vincent-iQontrol commented 1 year ago

@JKRhb ok how can we debug this?

vincent-iQontrol commented 1 year ago

@JKRhb I'm debugging the dtls_client.dart from the dtls package. It seems that the data is indeed comming in, that is great.

When I do:
print(utf8.decode(_buffer.asTypedList(ret), allowMalformed: true)); In the _maintainState function I see the right data (it is json) printed in the console for example: hEP��5�HiD��}�2 �{"id":1,"result":{"name":"nameValue","value":"valueValue"}}

The gibberish hEP��5�HiD��}�2 �, i dont know what that is. So it looks like the data is not passed to the coap package??

JKRhb commented 1 year ago

@JKRhb I'm debugging the dtls_client.dart from the dtls package. It seems that the data is indeed comming in, that is great.

:tada:

When I do: print(utf8.decode(_buffer.asTypedList(ret), allowMalformed: true)); In the _maintainState function I see the right data (it is json) printed in the console for example: hEP��5�HiD��}�2 �{"id":1,"result":{"name":"nameValue","value":"valueValue"}}

The gibberish hEP��5�HiD��}�2 �, i dont know what that is. So it looks like the data is not passed to the coap package??

I think the gibberish should be the actual CoAP packet data (i.e., the header and options). Could you check if the packet is passed to the eventBus in the CoapNetworkUDPOpenSSL class?

https://github.com/shamblett/coap/blob/82fd9b145a930b7b972d06aad0f6b71f427f42b8/lib/src/network/coap_network_openssl.dart#L117-L122

vincent-iQontrol commented 1 year ago

@JKRhb no frames are coming in..

vincent-iQontrol commented 1 year ago

@JKRhb if i set a delay before received, I can print coap messages (frame). But they are still not coming through. Something wrong with the eventBus?

` await Future.delayed(Duration(seconds: 2));

_dtlsConnection?.received.listen(`
JKRhb commented 1 year ago

Hmm, could it be that CoapMessage.fromUdpPayload returns null due to a parsing error?

vincent-iQontrol commented 1 year ago

Hmm, could it be that CoapMessage.fromUdpPayload returns null due to a parsing error?

No there are bytes, the message var contains the right coap message like this:

<<< Response Message >>> Type: Message type 2: Acknowledgement, Code: 2.05 Content, Id: 38714, Token: 'cb4e5203c2855a1b', Options: [ E-tags: ETag: fd9c027d, Content-Type: application/json, Max Age: 0, Size 1: 0, Size 2: 0, ], Payload: {"id":1,"result":{"name":"nameValue","value":"valueValue"}}

vincent-iQontrol commented 1 year ago

is the size: 0 a problem?

JKRhb commented 1 year ago

is the size: 0 a problem?

Hmm, I think that Size2 options are actually not being handled by the client at the moment. My current theory would be that the response cannot be matched to the request for some reason – can you maybe check if the token and the message ID of both match each other?

vincent-iQontrol commented 1 year ago

Looks the same to me.

<<< Request Message >>> Type: Message type 0: Confirmable, Code: 0.03 PUT, Id: 2450, Token: '498ee136440e65c3',

<<< Response Message >>> Type: Message type 2: Acknowledgement, Code: 2.05 Content, Id: 2450, Token: '498ee136440e65c3',

JKRhb commented 1 year ago

Hmm, yeah, that does not seem to be cause. Could you maybe have a look at the events that are being logged, as in this example? https://github.com/shamblett/coap/blob/master/example/log_events.dart

vincent-iQontrol commented 1 year ago

@JosefWN I already use the log feature like this: client.events.on().listen(print);

Listening to the internal request/response event stream CoapSendingRequestEvent: <<< Request Message >>> Type: Message type 0: Confirmable, Code: 0.03 PUT, Id: 676, Token: '3d4b7c4f3a45a0d1', Options: [ Uri Port: 5684, Uri Paths: somePath, Content-Type: null, Max Age: 60, Size 1: 0, Size 2: 0, ], Payload: someJsonPayload CoapRetransmitEvent: <<< Request Message >>> Type: Message type 0: Confirmable, Code: 0.03 PUT, Id: 676, Token: '3d4b7c4f3a45a0d1', Options: [ Uri Port: 5684, Uri Paths: somePath, Content-Type: null, Max Age: 60, Size 1: 0, Size 2: 0, ], Payload: someJsonPayload CoapSendingRequestEvent: <<< Request Message >>> Type: Message type 0: Confirmable, Code: 0.03 PUT, Id: 676, Token: '3d4b7c4f3a45a0d1', Options: [ Uri Port: 5684, Uri Paths: somePath, Content-Type: null, Max Age: 60, Size 1: 0, Size 2: 0, ], Payload: someJsonPayload CoapRetransmitEvent: <<< Request Message >>> Type: Message type 0: Confirmable, Code: 0.03 PUT, Id: 676, Token: '3d4b7c4f3a45a0d1', Options: [ Uri Port: 5684, Uri Paths: somePath, Content-Type: null, Max Age: 60, Size 1: 0, Size 2: 0, ], Payload: someJsonPayload CoapSendingRequestEvent: <<< Request Message >>> Type: Message type 0: Confirmable, Code: 0.03 PUT, Id: 676, Token: '3d4b7c4f3a45a0d1', Options: [ Uri Port: 5684, Uri Paths: somePath, Content-Type: null, Max Age: 60, Size 1: 0, Size 2: 0, ], Payload: someJsonPayload CoapCompletedEvent: Exchange for request 676 (token '3d4b7c4f3a45a0d1') CoapTimedOutEvent: <<< Request Message >>> Type: Message type 0: Confirmable, Code: 0.03 PUT, Id: 676, Token: '3d4b7c4f3a45a0d1', Options: [ Uri Port: 5684, Uri Paths: somePath, Content-Type: null, Max Age: 60, Size 1: 0, Size 2: 0, ], Payload: someJsonPayload CoapCancelledEvent: Type: Message type 3: Reset, Code: 0.00 Empty, Id: 676, Token: '3d4b7c4f3a45a0d1', Options: [ Content-Type: null, Max Age: 60, Size 1: 0, Size 2: 0, ], Payload: CoAP encountered an exception: CoapRequestTimeoutException: Request timed out after 2 retransmits.

JKRhb commented 1 year ago
vincent-iQontrol commented 1 year ago

@JKRhb There are not coming events in the eventbus listener in the Endpoint class is what I see:

eventBus.on<CoapMessageReceivedEvent>().listen

vincent-iQontrol commented 1 year ago

@JKRhb Do I have to create a new issue for the eventbus issue, because it looks like something is wrong with it.