Azure / azure-event-hubs-c

C client library for Azure Event Hubs https://azure.microsoft.com/services/event-hubs
Other
7 stars 18 forks source link

Azure uamqp library: Eventhub crashes on sending data over 1MB #60

Closed ssdbest closed 4 years ago

ssdbest commented 4 years ago

I am using one of the samples from uAMQP library, eh_sender_with_sas_token_sample(https://github.com/Azure/azure-uamqp-c). I modified to send 1MB data as our event hub supports 1MB bandwidth. Attached is the modified code which takes file(1MB) as an command line argument and try to send twice in a loop. Please have a look at it. But what I noticed is I can send up to ~65KB, beyond that the library is crashing in IO processing event loop. Most likely crash is due to processing removed io event from the list again.

Trace: Info: Removed:(List=00000295FE388E10, Item=00000295FE379330) Connection_do_work:-------------------2 Info: Removed:(List=00000295FE379510, Item=00000295FE379330) --> Accessed and removed twice same list item from two different list.

And also I remember that we were successful in sending 1MB data year back or so. Don’t remember which version of library we used during our initial stage of development. I tried with latest version of library(master) which breaks CBS authentication. Also explored degrading to 1.2.9 and 1.2.10 version. Currently we are using 1.2.11 version. Crash is observed on both Windows and Linux system.

ssdbest commented 4 years ago

main.txt

uploaded sample code for reference.

dcristoloveanu commented 4 years ago

Hi @ssdbest,

I'm having a look at this. This seems to be related to uamqp, rather than the event-hubs-c SDK, but regardless I'm having a look at it as it seems to be a regression.

Dan

ssdbest commented 4 years ago

Thanks Dan. Microsoft support team has asked me to raise ticket here. Call stack for your reference: eh_sender_with_sas_token_sample.exe!on_send_complete(void context, IO_SEND_RESULT_TAG send_result) Line 692 C eh_sender_with_sas_token_sample.exe!socketio_dowork(void socket_io) Line 498 C eh_sender_with_sas_token_sample.exe!xio_dowork(XIO_INSTANCE_TAG xio) Line 167 C eh_sender_with_sas_token_sample.exe!tlsio_schannel_dowork(void tls_io) Line 1282 C eh_sender_with_sas_token_sample.exe!xio_dowork(XIO_INSTANCE_TAG xio) Line 167 C eh_sender_with_sas_token_sample.exe!saslclientio_dowork(void sasl_client_io) Line 1222 C eh_sender_with_sas_token_sample.exe!xio_dowork(XIO_INSTANCE_TAG xio) Line 167 C eh_sender_with_sas_token_sample.exe!connection_dowork(CONNECTION_INSTANCE_TAG connection) Line 1848 C eh_sender_with_sas_token_sample.exe!main(int argc, char argv) Line 248 C eh_sender_with_sas_token_sample.exe!invoke_main() Line 79 C++ eh_sender_with_sas_token_sample.exe!scrt_common_main_seh() Line 288 C++ eh_sender_with_sas_token_sample.exe!scrt_common_main() Line 331 C++ eh_sender_with_sas_token_sample.exe!mainCRTStartup() Line 17 C++ kernel32.dll!00007fffcc327bd4() Unknown ntdll.dll!00007fffce26ce51() Unknown

dcristoloveanu commented 4 years ago

Hi @ssdbest

No worries. I could reproduce your issue and merged a fix in the master branch of the uamqp repo (https://github.com/Azure/azure-uamqp-c). If you could snap to the latest master there and check it, that would be great. The releases are handled by the IoT team since they are the main consumer of that library, so this will be picked up next time they release uamqp.

I will close this issue here, if there is another problem with uamqp, just open a new issue in that repo.

Thanks! Dan

ssdbest commented 4 years ago

Thanks Dan for the quick fix :)