search

Azure-Samples / cognitive-services-speech-sdk

Sample code for the Microsoft Cognitive Services Speech SDK
MIT License
2.67k stars 1.78k forks source link

Unable to run 1.38 under python3.11-slim-bookworm #2420

Open bobby-beckmann opened 2 weeks ago

bobby-beckmann commented 2 weeks ago

Issue:

Can't run in bookworm. Fails to load libssl.so.3.

Build:

Dockerfile:

FROM python:3.11-slim-bookworm

RUN apt-get -y update
RUN apt-get install -y ffmpeg supervisor build-essential ca-certificates libssl-dev libasound2 wget

OpenSSL Version Check:

I ran a small binary to dynamically load OpenSSL and display the version. It works fine:

OpenSSL version: OpenSSL 3.0.11 19 Sep 2023

Logs from SDK:

[323349]: 24113ms SPX_TRACE_VERBOSE:  pal_create_module_object.cpp:98 Trying to load libssl.so.3
[323349]: 24113ms SPX_TRACE_VERBOSE:  pal_create_module_object.cpp:112 Creating CSpxDyanmic for libpal_azure_c_shared_openssl3.so.
[323349]: 24113ms SPX_TRACE_VERBOSE:  pal_create_module_object.cpp:121 Loaded libpal_azure_c_shared_openssl3.so, getting PAL method.
[323349]: 24113ms SPX_TRACE_VERBOSE:  dynamic_module.cpp:53 Loading 'libpal_azure_c_shared_openssl3.so'
[323349]: 24113ms SPX_TRACE_VERBOSE:  dynamic_module.cpp:55 dlopen('libpal_azure_c_shared_openssl3.so') returned non-NULL
[323349]: 24113ms SPX_TRACE_VERBOSE:  dynamic_module.cpp:61 dlsym('pal_get_value') returned non-NULL
[323349]: 24113ms SPX_TRACE_VERBOSE:  pal_create_module_object.cpp:67 Loaded PAL method, calling.
[323349]: 24113ms SPX_TRACE_VERBOSE:  pal_create_module_object.cpp:132 Initializing libpal_azure_c_shared_openssl3.so.
[323349]: 24113ms SPX_TRACE_ERROR: AZ_LOG_ERROR:  shim_openssl.c:55 libssl could not be loaded
[323349]: 24113ms SPX_TRACE_ERROR: AZ_LOG_ERROR:  tlsio_openssl.c:2175 Could not load libssl
[323349]: 24114ms SPX_TRACE_ERROR:  exception.cpp:130 About to throw Runtime error: Failed to initialize platform (azure-c-shared). Error: 2176 
[CALL STACK BEGIN]

/usr/local/lib/python3.11/site-packages/azure/cognitiveservices/speech/libpal_azure_c_shared_openssl3.so(+0x1ccd0) [0xffff1003ccd0]
/lib/aarch64-linux-gnu/libc.so.6(+0x83e08) [0xffffaddb3e08]
/usr/local/lib/python3.11/site-packages/azure/cognitiveservices/speech/libpal_azure_c_shared_openssl3.so(+0x1bb80) [0xffff1003bb80]
/usr/local/lib/python3.11/site-packages/azure/cognitiveservices/speech/libMicrosoft.CognitiveServices.Speech.core.so(+0xa52c8) [0xffff4c1452c8]
/lib/aarch64-linux-gnu/libc.so.6(+0x83e08) [0xffffaddb3e08]
/usr/local/lib/python3.11/site-packages/azure/cognitiveservices/speech/libMicrosoft.CognitiveServices.Speech.core.so(pal_get_value+0xb8) [0xffff4c13f394]
/usr/local/lib/python3.11/site-packages/azure/cognitiveservices/speech/libMicrosoft.CognitiveServices.Speech.core.so(+0x190aa8) [0xffff4c230aa8]
/lib/aarch64-linux-gnu/libc.so.6(+0x83e08) [0xffffaddb3e08]
/usr/local/lib/python3.11/site-packages/azure/cognitiveservices/speech/libMicrosoft.CognitiveServices.Speech.core.so(+0x866f8) [0xffff4c1266f8]
/usr/local/lib/python3.11/site-packages/azure/cognitiveservices/speech/libMicrosoft.CognitiveServices.Speech.core.so(+0xa0fc8) [0xffff4c140fc8]
/lib/aarch64-linux-gnu/libc.so.6(+0x83e08) [0xffffaddb3e08]
/usr/local/lib/python3.11/site-packages/azure/cognitiveservices/speech/libMicrosoft.CognitiveServices.Speech.core.so(+0x9ee08) [0xffff4c13ee08]
/usr/local/lib/python3.11/site-packages/azure/cognitiveservices/speech/libMicrosoft.CognitiveServices.Speech.core.so(+0x15e6e8) [0xffff4c1fe6e8]
/usr/local/lib/python3.11/site-packages/azure/cognitiveservices/speech/libMicrosoft.CognitiveServices.Speech.core.so(+0x12e8dc) [0xffff4c1ce8dc]
/usr/local/lib/python3.11/site-packages/azure/cognitiveservices/speech/libMicrosoft.CognitiveServices.Speech.core.so(+0x12f0dc) [0xffff4c1cf0dc]
/usr/local/lib/python3.11/site-packages/azure/cognitiveservices/speech/libMicrosoft.CognitiveServices.Speech.core.so(+0x12ccd8) [0xffff4c1cccd8]
/usr/local/lib/python3.11/site-packages/azure/cognitiveservices/speech/libMicrosoft.CognitiveServices.Speech.core.so(+0x13b46c) [0xffff4c1db46c]
[CALL STACK END]
bobby-beckmann commented 2 weeks ago

Upon further investigation, it seems that this is due to the ARM based build, not x86. libpal_azure_c_shared_openssl3.so references libssl.so.1.1 in ARM, but libssl.so.3 in x86.

:/

yulin-li commented 2 weeks ago

Thanks for reporting and investigating. @rhurey, could you help to check?

rhurey commented 2 weeks ago

@bobby-beckmann Thanks. I think I found the source of this mistake, and how it got missed in testing.

Once I confirm it, I'll update w/ the story.

rhurey commented 6 days ago

Ok, there was some unexpected (and unrelated challenges) verifing the fix.

What had happened is while building the arm64 (and only the arm64) version of the pal library there was a typo in the hint path for what OpenSSL version to target. As the name implies, the path was a hint, and the build fell back to the system version, which was 1.1.1 and not the 3.0+ that had been intended.

And then both of the arm64 machines we tested against happened to (for different reasons) had both 1.1.1 and 3.0+ installed. So while we'd validated the correct PAL module was loaded, it was linked against the wrong lib.

The fix will be in our next release.