aws-greengrass / aws-greengrass-client-device-auth

Apache License 2.0
2 stars 3 forks source link

feat(uat): implement mqtt5 publish properties for python paho client #364

Closed eschastlivtsev closed 1 year ago

eschastlivtsev commented 1 year ago

Description of changes:

Why is this change necessary:

How was this change tested: Manual run of mqtt-client-control and python client. MQTT connect, subscribe, unsubscribe, publish, delivery of received MQTT message and disconnect are successful. Linking and shutdown steps are successful. T102 scenario run.

Test results: Python Paho client output

client-python-paho.exe python-paho-agent
[DEBUG] 2023-07-07 18:51:01.377 asyncio - Using selector: SelectSelector
[INFO ] 2023-07-07 18:51:01.380 GRPCLib - Initialize gRPC library
[INFO ] 2023-07-07 18:51:01.380 GRPCLink - Making gPRC client connection with 127.0.0.1:47619 as python-paho-agent...
[INFO ] 2023-07-07 18:51:01.547 GRPCLink - Client connection with Control is established, local address is 127.0.0.1
[DEBUG] 2023-07-07 18:51:01.548 grpc._cython.cygrpc - Using AsyncIOEngine.POLLER as I/O engine
[INFO ] 2023-07-07 18:51:01.623 MQTTLib - Initialize Paho MQTT library
[INFO ] 2023-07-07 18:51:01.623 GRPCLink - Handle gRPC requests
[INFO ] 2023-07-07 18:51:01.625 GRPCControlServer - Server awaiting termination
[INFO ] 2023-07-07 18:51:04.709 GRPCControlServer - createMqttConnection: clientId Python_Paho_Client broker a3t8vwpkw3sltg-ats.iot.eu-west-1.amazonaws.com:8883
[INFO ] 2023-07-07 18:51:04.716 MqttConnection - Creating MQTT 5 client with TLS
[INFO ] 2023-07-07 18:51:04.716 MqttConnection - CONNECT TX user property 'region', 'US'
[INFO ] 2023-07-07 18:51:04.717 MqttConnection - CONNECT TX user property 'type', 'JSON'
[INFO ] 2023-07-07 18:51:04.717 MqttConnection - MQTT connection ID 0 connecting...
[DEBUG] 2023-07-07 18:51:05.022 AsyncioHelper - On socket open
[DEBUG] 2023-07-07 18:51:05.023 AsyncioHelper - On socket register write
[DEBUG] 2023-07-07 18:51:05.024 AsyncioHelper - On socket unregister write
[INFO ] 2023-07-07 18:51:05.140 MqttConnection - MQTT connection ID 0 connected, client id Python_Paho_Client
[DEBUG] 2023-07-07 18:51:10.258 GRPCControlServer - Subscription: filter test/topic QoS 0 noLocal 0 retainAsPublished 0 retainHandling 2
[INFO ] 2023-07-07 18:51:10.258 GRPCControlServer - SubscribeMqtt connection_id 0
[INFO ] 2023-07-07 18:51:10.259 MqttConnection - SUBSCRIBE TX user property 'region', 'US'
[INFO ] 2023-07-07 18:51:10.263 MqttConnection - SUBSCRIBE TX user property 'type', 'JSON'
[DEBUG] 2023-07-07 18:51:10.263 AsyncioHelper - On socket register write
[DEBUG] 2023-07-07 18:51:10.264 AsyncioHelper - On socket unregister write
[DEBUG] 2023-07-07 18:51:10.371 MqttConnection - Subscribed on filters 'test/topic' with QoS 0 no local False retain as published False retain handing 2 with message id 1
[INFO ] 2023-07-07 18:51:20.403 GRPCControlServer - PublishMqtt connection_id 0 topic test/topic retain 0
[INFO ] 2023-07-07 18:51:20.404 MqttConnection - PUBLISH TX user property 'region', 'US'
[INFO ] 2023-07-07 18:51:20.405 MqttConnection - PUBLISH TX user property 'type', 'JSON'
[INFO ] 2023-07-07 18:51:20.408 MqttConnection - Copied TX payload format indicator 'True'
[INFO ] 2023-07-07 18:51:20.408 MqttConnection - Copied TX message expiry interval 3600
[INFO ] 2023-07-07 18:51:20.411 MqttConnection - Copied TX content type 'text/plain; charset=utf-8'
[DEBUG] 2023-07-07 18:51:20.411 AsyncioHelper - On socket register write
[DEBUG] 2023-07-07 18:51:20.412 AsyncioHelper - On socket unregister write
[DEBUG] 2023-07-07 18:51:20.492 MqttConnection - Published on topic 'test/topic' QoS 1 retain 0 with rc 0 message id 2
[DEBUG] 2023-07-07 18:51:20.494 MqttConnection - onMessage message b'Hello World!' from topic 'test/topic'
[INFO ] 2023-07-07 18:51:20.495 MqttConnection - PUBLISH RX user property 'region', 'US'
[INFO ] 2023-07-07 18:51:20.497 MqttConnection - PUBLISH RX user property 'type', 'JSON'
[INFO ] 2023-07-07 18:51:20.498 MqttConnection - Copied RX payload format indicator '1'
[INFO ] 2023-07-07 18:51:20.499 MqttConnection - Copied RX message expiry interval 3599
[INFO ] 2023-07-07 18:51:20.501 MqttConnection - Copied RX content type 'text/plain; charset=utf-8'
[INFO ] 2023-07-07 18:51:20.504 GRPCDiscoveryClient - Sending OnReceiveMessage request agent_id 'python-paho-agent' connection_id 0 message b'Hello World!' on topic 'test/topic'
[DEBUG] 2023-07-07 18:51:25.512 GRPCControlServer - Unsubscription: filter test/topic
[INFO ] 2023-07-07 18:51:25.512 GRPCControlServer - UnsubscribeMqtt connection_id 0
[INFO ] 2023-07-07 18:51:25.515 MqttConnection - UNSUBSCRIBE TX user property 'region', 'US'
[INFO ] 2023-07-07 18:51:25.518 MqttConnection - UNSUBSCRIBE TX user property 'type', 'JSON'
[DEBUG] 2023-07-07 18:51:25.519 AsyncioHelper - On socket register write
[DEBUG] 2023-07-07 18:51:25.520 AsyncioHelper - On socket unregister write
[DEBUG] 2023-07-07 18:51:25.640 MqttConnection - Unsubscribed from filters 'test/topic' with message id 3
[INFO ] 2023-07-07 18:51:35.665 GRPCControlServer - CloseMqttConnection connection_id 0 reason 4
[INFO ] 2023-07-07 18:51:35.666 MqttConnection - Disconnect MQTT connection with reason code 4
[INFO ] 2023-07-07 18:51:35.668 MqttConnection - DISCONNECT TX user property 'region', 'US'
[INFO ] 2023-07-07 18:51:35.670 MqttConnection - DISCONNECT TX user property 'type', 'JSON'
[DEBUG] 2023-07-07 18:51:35.674 AsyncioHelper - On socket register write
[INFO ] 2023-07-07 18:51:35.675 GRPCDiscoveryClient - Sending OnMqttDisconnect request agent_id 'python-paho-agent' connection_id 0 error 'None'
[DEBUG] 2023-07-07 18:51:35.683 AsyncioHelper - On socket unregister write
[DEBUG] 2023-07-07 18:51:35.683 AsyncioHelper - On socket close
[INFO ] 2023-07-07 18:51:35.696 GRPCControlServer - shutdownAgent: reason That's it.
[INFO ] 2023-07-07 18:51:35.698 GRPCControlServer - Server termination done
[INFO ] 2023-07-07 18:51:35.699 GRPCLink - Shutdown gPRC link
[INFO ] 2023-07-07 18:51:35.710 Main - Execution done successfully

Mqtt Client Control output

java -Dmqtt_client_id=Python_Paho_Client -Dmqtt_broker_addr=a3t8vwpkw3sltg-ats.iot.eu-west-1.amazonaws.com -Dmqtt_client_ca_file=creds_aws/rootCA.pem -Dmqtt_client_cert_file=creds_aws/thingCert.crt -Dmqtt_client_key_file=creds_aws/privKey.key -jar target/aws-greengrass-testing-mqtt-client-control.jar 47619 true true
[INFO ] 2023-07-07 18:50:53.663 [main] ExampleControl - Control: port 47619, with TLS, MQTT v5
WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
[INFO ] 2023-07-07 18:50:55.159 [main] EngineControlImpl - MQTT client control gRPC server started, listening on 47619
[INFO ] 2023-07-07 18:51:01.524 [grpc-default-executor-0] GRPCDiscoveryServer - RegisterAgent: agentId python-paho-agent
[INFO ] 2023-07-07 18:51:01.557 [grpc-default-executor-0] GRPCDiscoveryServer - DiscoveryClient: agentId python-paho-agent address 127.0.0.1 port 64863
[INFO ] 2023-07-07 18:51:01.561 [grpc-default-executor-0] EngineControlImpl - Created new agent control for python-paho-agent on 127.0.0.1:64863
[INFO ] 2023-07-07 18:51:01.611 [grpc-default-executor-0] ExampleControl - Agent python-paho-agent is connected
[INFO ] 2023-07-07 18:51:01.620 [pool-2-thread-1] AgentTestScenario - Playing test scenario for agent id python-paho-agent
[INFO ] 2023-07-07 18:51:04.633 [pool-2-thread-1] AgentTestScenario - Connect Set user property: region, US
[INFO ] 2023-07-07 18:51:04.633 [pool-2-thread-1] AgentTestScenario - Connect Set user property: type, JSON
[INFO ] 2023-07-07 18:51:05.223 [pool-2-thread-1] AgentControlImpl - Created connection with id 0 CONNACK 'sessionPresent: false
reasonCode: 0
receiveMaximum: 100
maximumQoS: 1
retainAvailable: true
maximumPacketSize: 149504
wildcardSubscriptionsAvailable: true
subscriptionIdentifiersAvailable: false
sharedSubscriptionsAvailable: true
serverKeepAlive: 60
topicAliasMaximum: 8
'
[INFO ] 2023-07-07 18:51:05.230 [pool-2-thread-1] AgentControlImpl - createMqttConnection: MQTT connectionId 0 created
[INFO ] 2023-07-07 18:51:05.234 [pool-2-thread-1] AgentTestScenario - MQTT connection with id 0 is established
[INFO ] 2023-07-07 18:51:10.240 [pool-2-thread-1] AgentTestScenario - Subscribe Set user property: region, US
[INFO ] 2023-07-07 18:51:10.241 [pool-2-thread-1] AgentTestScenario - Subscribe Set user property: type, JSON
[INFO ] 2023-07-07 18:51:10.248 [pool-2-thread-1] AgentControlImpl - SubscribeMqtt: subscribe on connection 0
[INFO ] 2023-07-07 18:51:10.380 [pool-2-thread-1] AgentTestScenario - Subscribe response: connectionId 0 reason codes [0] reason string ''
[INFO ] 2023-07-07 18:51:20.385 [pool-2-thread-1] AgentTestScenario - Set property payload format indicator true
[INFO ] 2023-07-07 18:51:20.386 [pool-2-thread-1] AgentTestScenario - Set property message expiry interval 3600
[INFO ] 2023-07-07 18:51:20.389 [pool-2-thread-1] AgentTestScenario - Publish Set user property: region, US
[INFO ] 2023-07-07 18:51:20.391 [pool-2-thread-1] AgentTestScenario - Publish Set user property: type, JSON
[INFO ] 2023-07-07 18:51:20.392 [pool-2-thread-1] AgentTestScenario - Set property content type text/plain; charset=utf-8
[INFO ] 2023-07-07 18:51:20.397 [pool-2-thread-1] AgentControlImpl - PublishMqtt: publishing on connectionId 0 topic test/topic
[INFO ] 2023-07-07 18:51:20.497 [pool-2-thread-1] AgentTestScenario - Published connectionId 0 reason code 0 reason string ''
[INFO ] 2023-07-07 18:51:20.510 [grpc-default-executor-0] GRPCDiscoveryServer - OnReceiveMessage: agentId python-paho-agent connectionId 0 topic test/topic QoS 0
[INFO ] 2023-07-07 18:51:20.511 [grpc-default-executor-0] AgentTestScenario - Message received on agentId python-paho-agent connectionId 0 topic test/topic QoS 0 content <ByteString@3bffc032 size=12 contents="Hello World!">
[INFO ] 2023-07-07 18:51:20.512 [grpc-default-executor-0] AgentTestScenario - Message has payload format indicator true
[INFO ] 2023-07-07 18:51:20.512 [grpc-default-executor-0] AgentTestScenario - Message has message expiry interval 3599
[INFO ] 2023-07-07 18:51:20.513 [grpc-default-executor-0] AgentTestScenario - Message has user property key region value US
[INFO ] 2023-07-07 18:51:20.513 [grpc-default-executor-0] AgentTestScenario - Message has user property key type value JSON
[INFO ] 2023-07-07 18:51:20.513 [grpc-default-executor-0] AgentTestScenario - Message has content type 'text/plain; charset=utf-8'
[INFO ] 2023-07-07 18:51:25.498 [pool-2-thread-1] AgentTestScenario - Unsubscribe Set user property: region, US
[INFO ] 2023-07-07 18:51:25.498 [pool-2-thread-1] AgentTestScenario - Unsubscribe Set user property: type, JSON
[INFO ] 2023-07-07 18:51:25.507 [pool-2-thread-1] AgentControlImpl - UnsubscribeMqtt: unsubscribe on connectionId 0
[INFO ] 2023-07-07 18:51:25.642 [pool-2-thread-1] AgentTestScenario - Unsubscribe response: connectionId 0 reason codes [0] reason string ''
[INFO ] 2023-07-07 18:51:35.653 [pool-2-thread-1] AgentTestScenario - Disconnect Set user property: region, US
[INFO ] 2023-07-07 18:51:35.654 [pool-2-thread-1] AgentTestScenario - Disconnect Set user property: type, JSON
[INFO ] 2023-07-07 18:51:35.681 [grpc-default-executor-0] GRPCDiscoveryServer - OnMqttDisconnect: agentId python-paho-agent connectionId 0 disconnect 'reasonCode: 0
' error ''
[INFO ] 2023-07-07 18:51:35.681 [grpc-default-executor-0] AgentTestScenario - MQTT disconnected on agentId python-paho-agent connectionId 0 disconnect 'reasonCode: 0
' error ''
[INFO ] 2023-07-07 18:51:35.691 [pool-2-thread-1] AgentControlImpl - closeMqttConnection: MQTT connectionId 0 closed
[INFO ] 2023-07-07 18:51:35.698 [pool-2-thread-1] AgentControlImpl - shutdown request sent successfully
[INFO ] 2023-07-07 18:51:35.705 [grpc-default-executor-1] GRPCDiscoveryServer - UnregisterAgent: agentId python-paho-agent reason Agent shutdown by OTF request 'That's it.'
[INFO ] 2023-07-07 18:51:35.707 [grpc-default-executor-1] ExampleControl - Agent python-paho-agent is disconnected

T102 scenario run results

[INFO ] 2023-07-10 18:05:37.400 [main] StepTrackingReporting - Passed: 'GGMQ-1-T102-v5-sdk-java: As a customer, I can use publish retain flag,subscribe retain handling, content type using MQTT V5.0'
[INFO ] 2023-07-10 18:05:37.400 [main] StepTrackingReporting - Passed: 'GGMQ-1-T102-v5-mosquitto-c: As a customer, I can use publish retain flag,subscribe retain handling, content type using MQTT V5.0'
[INFO ] 2023-07-10 18:05:37.400 [main] StepTrackingReporting - Passed: 'GGMQ-1-T102-v5-paho-java: As a customer, I can use publish retain flag,subscribe retain handling, content type using MQTT V5.0'
[INFO ] 2023-07-10 18:05:37.400 [main] StepTrackingReporting - Passed: 'GGMQ-1-T102-v5-paho-python: As a customer, I can use publish retain flag,subscribe retain handling, content type using MQTT V5.0'

Any additional information or context required to review the change:

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.