Azure-Samples / cognitive-services-speech-sdk

Sample code for the Microsoft Cognitive Services Speech SDK
MIT License
2.8k stars 1.83k forks source link

OpenSSL 1.x reaches EOL on September 11, 2023 #2048

Closed niklaskorz closed 3 months ago

niklaskorz commented 1 year ago

https://www.openssl.org/blog/blog/2023/03/28/1.1.1-EOL/

Suggesting to install OpenSSL 1.x from source because it has been removed from all Linux distributions is bad advice and a potential security risk.

chschrae commented 1 year ago

Thank you for letting us know. We are working on an update for this, but do not have an ETA.

I will close this issue to keep the list of open issues fresh. Feel free to reach out again if you feel it is necessary.

jhakulin commented 1 year ago

I want to reopen and we shall provide ETA before closing this issue.

chschrae commented 1 year ago

Internal work item: 4254856

We will track it there.

niklaskorz commented 1 year ago

OpenSSL 1.x reached EOL yesterday. When can we expect this crucial security update from you? Or is Microsoft Azure going to sponsor the OpenSSL premium support contract to all customers that now have to rely on an otherwise insecure dependency?

ckpearson commented 10 months ago

Just been bitten by this after an upgrade to .NET 8 and the new base images using version 3. We're developing an app that has to meet compliance requirements and the official line from Microsoft being "install a vulnerable dependency from source" is pretty poor.

Is there any sort of ETA on this, especially given .NET 8 is now an official release?

github-actions[bot] commented 9 months ago

This item has been open without activity for 19 days. Provide a comment on status and remove "update needed" label.

jbennink commented 8 months ago

Just got bitten by the same issue. Is there any ETA, I'm not really a Linux person and don't want to build my own container. For now I could simple stick with our old container, this was just a framework update, no new features but this is very annoying.

juise commented 8 months ago

This functionality become very crucial nowadays, do you have ant ETA?

github-actions[bot] commented 7 months ago

This item has been open without activity for 19 days. Provide a comment on status and remove "update needed" label.

CodeLogist commented 7 months ago

Hey Guys, any update on this?

chschrae commented 7 months ago

We are actively working on it and hope to have a meaningful update soon.

juise commented 7 months ago

Maybe you could share some ETA with us?

Canonip commented 7 months ago

I still can't comprehend how the official microsoft docs contain a guide on how to compile an outdated OpenSSL version from scratch, just so we can use an Azure SDK.

Is there at least a prebuilt .NET 8 image that we can use on Azure Web Apps?

MrKevHunter commented 6 months ago

Can we get an update on this, we're having to delay a rollout because of this?

Dustinhoefer commented 6 months ago

Also waiting for this. Default AWS comes with openssl 3.x and most people should probably not tamper with that. Is there an ETA? 1 Week? 1 Month? 6 Month?

niklaskorz commented 6 months ago

I think at this point your best bet is trying to migrate to the Azure Cognitive Services REST API, as Microsoft clearly has no interest or resources in maintaining the native SDK anymore.

roklenardic commented 6 months ago

I think at this point your best bet is trying to migrate to the Azure Cognitive Services REST API, as Microsoft clearly has no interest or resources in maintaining the native SDK anymore.

Not sure that support speech-to-text and text-to-speech streaming, can it?

niklaskorz commented 6 months ago

I think at this point your best bet is trying to migrate to the Azure Cognitive Services REST API, as Microsoft clearly has no interest or resources in maintaining the native SDK anymore.

Not sure that support speech-to-text and text-to-speech streaming, can it?

Yup that's unfortunately not the case :(

lumpidu commented 6 months ago

And also no word-timestamps for TTS, etc., etc. ..... why don't they just make a gRPC interface then the whole SDK would be obsolete .... ?

ckpearson commented 6 months ago

And also no word-timestamps for TTS, etc., etc. ..... why don't they just make a gRPC interface then the whole SDK would be obsolete .... ?

I didn't even realise you could get timestamps for TTS? That would be so useful. Unfortunately, we've had so many issues with the SDK and now this, that we're having to use the http API directly.

We have PCI compliance to think about, and all our assurances from Microsoft about the compliance of the service itself go out the window if we tell the auditors that we're using a known EOL version of OpenSSL to talk to the damned thing.

roklenardic commented 6 months ago

Anyways, this is a major setback for us as because of this we cannot deploy to azure container apps (app boot fails on linux because of this). We tried windows app services and those performed soooo poorly doing speech to text it rendered them useless.

In a nutshell, due to this issue we'll have to find a different SpeechToText and TextToSpeech provider instead of using cognitive services we were very happy with in the lab environment.

Please fix this MS!

Dustinhoefer commented 6 months ago

If anyone is interested in working on a repo to implement the http api in java, hit me up :)

niklaskorz commented 6 months ago

If anyone is interested in working on a repo to implement the http api in java, hit me up :)

To get the full feature set, you can also try implementing the WebSocket API. There is a Rust SDK that did this, and the official JavaScript SDK also does this and is fully open source. To see what requests the JS SDK makes, you can either read the source code (no clean room reverse engineering needed, it's MIT licensed, so go ahead and copy & translate whatever you need) or load it in the browser and have a look at the network tab. It's no rocket science, so while the WebSocket API is not officially documented it's still really easy to use.

This way, you still get access to features like streaming responses or word boundary timestamps.

Tenzer commented 6 months ago

This isn't too promising: https://github.com/MicrosoftDocs/azure-docs/commit/06c39f87be22d392f0049c9e0d14e8af7403cd77

jhakulin commented 6 months ago

We are still working on supporting the OpenSSL3.x for Linux, we will let you know the exact release date once that is known, probable ETA Q2 2024.

dmi-harlow commented 5 months ago

For any .NET developers just wanting to get things working while awaiting OpenSSL3 support, adding this to my dockerfile got me back in business (adds legacy OpenSSL support to ASP.NET 8.0 container base image):

FROM mcr.microsoft.com/dotnet/aspnet:8.0 AS base USER root RUN apt-get update RUN apt-get -y install wget RUN wget http://archive.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.0g-2ubuntu4_amd64.deb RUN dpkg -i libssl1.1_1.1.0g-2ubuntu4_amd64.deb USER app...

Credit to @joulgs https://gist.github.com/joulgs/c8a85bb462f48ffc2044dd878ecaa786

roklenardic commented 5 months ago

This worked for us deploying as docker:

ROM mcr.microsoft.com/dotnet/aspnet:8.0-jammy AS base RUN apt update && apt install wget -y RUN wget http://security.ubuntu.com/ubuntu/pool/main/o/openssl/libssl-dev_1.1.1f-1ubuntu2_amd64.deb RUN wget http://security.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.1f-1ubuntu2_amd64.deb RUN wget http://security.ubuntu.com/ubuntu/pool/main/o/openssl/openssl_1.1.1f-1ubuntu2_amd64.deb RUN dpkg -i libssl1.1_1.1.1f-1ubuntu2_amd64.deb RUN dpkg -i libssl-dev_1.1.1f-1ubuntu2_amd64.deb RUN dpkg -i openssl_1.1.1f-1ubuntu2_amd64.deb

Baklap4 commented 4 months ago

We're currently trying to update to net8 since net 7.0 runs out of support on 14th of may. As Debian 12 ships openssl 3 i',m just wondering if this will be fixed beforehand or if we need to consolidate this part of our api into a new one, running a lower version of openssl which is already EOL since september the 11th 2023 so we don't expose ourselves with vulnerabilities

There are currently 6 high vulnerabilities as seen on: https://www.openssl.org/news/vulnerabilities-3.0.html

jjanczur commented 4 months ago

Is there any update? Our Snyk is complaining about it a lot. We can't upgrade the Java version to one higher than 18, because it is enforcing OpenSSL 3.X and Azure cognitive services are still using OpenSSL 1.1.1 which is already past its EOL and without support. Microsoft come on

waveBoom commented 4 months ago

python temporary solution : https://github.com/Azure-Samples/cognitive-services-speech-sdk/issues/2204#issuecomment-2075395036

msehnout commented 4 months ago

python temporary solution : #2204 (comment)

Or just use python:bullseye images:

$ docker run -it python:bullseye /bin/bash
root@6d28f69a4006:/# apt show openssl
Package: openssl
Version: 1.1.1w-0+deb11u1

Debian 11 is still supported and no need do a complicated ubuntu package installations on debian-based python images.

hmthang96 commented 4 months ago

Is there any update? Because the Amazon linux 2023 does not support openssl 1.1.1 anymore, I can't use the SDK for python anymore :(

pankopon commented 4 months ago

OpenSSL 3.x support has been implemented and will be included in the Speech SDK 1.38.0 release coming this summer.

xmkevin commented 3 months ago

OpenSSL 3.x support has been implemented and will be included in the Speech SDK 1.38.0 release coming this summer.

Can we use snapshot version ?

SparksFlyx3 commented 3 months ago

Is there a pre-release or beta version we can use now @pankopon to bridge the gap until the official summer release?

Tenzer commented 3 months ago

I can see azure-cognitiveservices-speech v1.38.0 is out now. There's however no release notes available for it yet.

I've tried to compare the ldd output against the binary files in the new version with the previous version, and I don't see any difference. Could we get confirmation this new release should work with OpenSSL 3.* please?

valentino-sm commented 3 months ago

Doesn't work for me yet

P.S. OpenSSL 3.3.1

[77079]: 24370ms SPX_TRACE_VERBOSE:  pal_create_module_object.cpp:98 Trying to load libssl.so.3
[77079]: 24370ms SPX_TRACE_VERBOSE:  pal_create_module_object.cpp:112 Creating CSpxDyanmic for libpal_azure_c_shared_openssl3.so.
[77079]: 24370ms SPX_TRACE_VERBOSE:  pal_create_module_object.cpp:121 Loaded libpal_azure_c_shared_openssl3.so, getting PAL method.
[77079]: 24370ms SPX_TRACE_VERBOSE:  dynamic_module.cpp:53 Loading 'libpal_azure_c_shared_openssl3.so'
[77079]: 24370ms SPX_TRACE_VERBOSE:  dynamic_module.cpp:55 dlopen('libpal_azure_c_shared_openssl3.so') returned non-NULL
[77079]: 24370ms SPX_TRACE_VERBOSE:  dynamic_module.cpp:61 dlsym('pal_get_value') returned non-NULL
[77079]: 24370ms SPX_TRACE_VERBOSE:  pal_create_module_object.cpp:67 Loaded PAL method, calling.
[77079]: 24370ms SPX_TRACE_VERBOSE:  pal_create_module_object.cpp:132 Initializing libpal_azure_c_shared_openssl3.so.
[77079]: 24370ms SPX_TRACE_ERROR: AZ_LOG_ERROR:  shim_openssl.c:99 Unsupported libssl version 30300010
[77079]: 24370ms SPX_TRACE_ERROR: AZ_LOG_ERROR:  tlsio_openssl.c:2175 Could not load libssl
[77079]: 24371ms SPX_TRACE_ERROR:  exception.cpp:130 About to throw Runtime error: Failed to initialize platform (azure-c-shared). Error: 2176 
pankopon commented 3 months ago

Release notes should be available soon.

Unfortunately OpenSSL 3.x support is initially limited to 3.0.x which is used in Debian 12 and Ubuntu 22.04 LTS (Ubuntu 24.04 LTS also has 3.0.x). This limitation will probably go away in the next update.

In the Speech SDK 1.38.0 release Linux platform requirements have been updated so that Ubuntu 20.04 LTS (with glibc 2.31) is the new minimum baseline. Officially supported distributions include Debian 11/12 and Ubuntu 20.04/22.04 LTS. (Ubuntu 24.04 LTS should work as well though not tested.) RHEL/CentOS 7 specific support is still included but will be dropped in the next major release after June 30.

Tenzer commented 3 months ago

I have upgraded our Docker base image from using Debian 11 (bullseye) to Debian 12 (bookworm) and can confirm that the library still works after the upgrade.

Thank you for fixing this!

pankopon commented 3 months ago

Closing the issue as the release is now out. Be sure to check the release notes and updated platform requirements.