Azure / azure-uamqp-c

AMQP library for C
Other
60 stars 63 forks source link

SEGFAULT on link_transfer_async when session_send_transfer fails with SESSION_SEND_TRANSFER_ERROR #197

Closed ewertons closed 6 years ago

ewertons commented 6 years ago

This issue was experienced using azure-iot-sdk-c (1.1.28).

Environment: Raspbian on Pi3 (Linux raspberrypi 4.9.35-v7+ #1014 SMP Fri Jun 30 14:47:43 BST 2017 armv7l GNU/Linux)

Here are the repro steps:

  1. Run sample https://github.com/Azure/azure-iot-sdk-c/tree/master/iothub_client/samples/iothub_client_sample_x509
    • Sample was modified to send 5000+ messages, one per second (attached).
  2. Let the first messages succeed to be sent;
  3. Unplug network cable
  4. Wait for 4 minutes
  5. Reconnection is triggered, and crash occurs.

Callstack:

#0  __GI___libc_free (mem=0x16dd7c) at malloc.c:2945
#1  0x000bba8c in  (link=0x132150, message_format=2147563264, payloads=0x7efeb524, payload_count=1, on_delivery_settled=0xc0e44 <on_delivery_settled>, callback_context=0x191d20, link_transfer_error=0x7efeb510, timeout=0)
    at /home/ewertons/azure-iot-sdk-c/uamqp/src/link.c:1252
#2  0x000c2250 in send_one_message (message_sender=0x12cd78, pending_send=0x191d20, message=0x191c38) at /home/ewertons/azure-iot-sdk-c/uamqp/src/message_sender.c:521
#3  0x000c31e4 in messagesender_send_async (message_sender=0x12cd78, message=0x191c38, on_message_send_complete=0x70628 <internal_on_event_send_complete_callback>, callback_context=0x191a28, timeout=0)
    at /home/ewertons/azure-iot-sdk-c/uamqp/src/message_sender.c:898
#4  0x00070aac in send_batched_message_and_reset_state (instance=0x12d2b8, send_pending_events_state=0x7efeb6a0) at /home/ewertons/azure-iot-sdk-c/iothub_client/src/iothubtransport_amqp_telemetry_messenger.c:1133
#5  0x00071228 in send_pending_events (instance=0x12d2b8) at /home/ewertons/azure-iot-sdk-c/iothub_client/src/iothubtransport_amqp_telemetry_messenger.c:1279
#6  0x00072dac in telemetry_messenger_do_work (messenger_handle=0x12d2b8) at /home/ewertons/azure-iot-sdk-c/iothub_client/src/iothubtransport_amqp_telemetry_messenger.c:1880
#7  0x0006812c in device_do_work (handle=0x12d208) at /home/ewertons/azure-iot-sdk-c/iothub_client/src/iothubtransport_amqp_device.c:1098
#8  0x00060bdc in IoTHubTransport_AMQP_Common_Device_DoWork (registered_device=0x12d178) at /home/ewertons/azure-iot-sdk-c/iothub_client/src/iothubtransport_amqp_common.c:1130
#9  0x00061fdc in IoTHubTransport_AMQP_Common_DoWork (handle=0x12d068, iotHubClientHandle=0x12cf58) at /home/ewertons/azure-iot-sdk-c/iothub_client/src/iothubtransport_amqp_common.c:1544
#10 0x0005e1b0 in IoTHubTransportAMQP_DoWork (handle=0x12d068, iotHubClientHandle=0x12cf58) at /home/ewertons/azure-iot-sdk-c/iothub_client/src/iothubtransportamqp.c:67
#11 0x00027d74 in IoTHubClient_LL_DoWork (iotHubClientHandle=0x12cf58) at /home/ewertons/azure-iot-sdk-c/iothub_client/src/iothub_client_ll.c:1313
#12 0x0002215c in main () at /home/ewertons/azure-iot-sdk-c/iothub_client/samples/iothub_client_sample_x509/iothub_client_sample_x509.c:346
ewertons commented 6 years ago

Issue was root-caused to be in this code change: 34efcbb18d986ab48581499d0c87c82064830b36

+                                        case SESSION_SEND_TRANSFER_ERROR: 
+                                            singlylinkedlist_remove(link->pending_deliveries, delivery_instance_list_item); 
+                                            free(pending_delivery); 
+                                            *link_transfer_error = LINK_TRANSFER_ERROR; 
+                                            async_operation_destroy(result); 
+                                            result = NULL; 
+                                            break; 

free(pending_delivery) should not be invoked (here and everywhere) since the deallocation is done through async_operation_destroy.

ewertons commented 6 years ago

Issue fixed.