Azure / azure-powershell

Microsoft Azure PowerShell
Other
4.19k stars 3.8k forks source link

Az.Accounts seems to bundle old version of System.Text.Encodings.Web dll (with severe CVE) #22895

Closed jorisscheppers closed 7 months ago

jorisscheppers commented 11 months ago

Description

For some time now, our container registry vulnerability scanning tool has been flagging one of our pipeline images for a severe CVE (: CVE-2021-26701). The team has flagged this as a false-positive (mainly because of https://github.com/Azure/azure-powershell/issues/21911), but since I have been digging a little deeper I have found that an affected version of the System.Text.Encodings.Web dll is actually being added to the container image, even though we are building a new image with the latest versions of both Powershell and the Az module every week.

Dockerfile for the pipeline-base image: (the CVE is not in this image, but I added it for reference)

# https://hub.docker.com/_/microsoft-dotnet-aspnet/
FROM mcr.microsoft.com/dotnet/aspnet:7.0

RUN DEBIAN_FRONTEND=noninteractive apt-get update \
  && DEBIAN_FRONTEND=noninteractive apt-get upgrade -y

RUN DEBIAN_FRONTEND=noninteractive apt-get install -y -qq --no-install-recommends \
  apt-transport-https \
  apt-utils \
  bash \
  ca-certificates \
  curl \
  git \
  jq \
  python3-pip \
  ssh \
  unzip

# SSH key for GIT SSH access to repos
COPY ***redacted*** /root/.ssh/id_rsa

# # Configure SSH key
RUN chmod 600 /root/.ssh/id_rsa \
  && echo "Host ssh.dev.azure.com\n  IdentityFile /root/.ssh/id_rsa\n  IdentitiesOnly yes\n  PubkeyAcceptedKeyTypes=ssh-rsa\n  HostkeyAlgorithms=ssh-rsa\n" >>/root/.ssh/config \
  && ssh-keyscan -H ssh.dev.azure.com >>/root/.ssh/known_hosts

# # Azure CLI with devops extension
RUN curl -sL https://aka.ms/InstallAzureCLIDeb | bash \
  && az extension add -n azure-devops

# Can be 'linux-x64', 'linux-arm64', 'linux-arm', 'rhel.6-x64'.
ENV TARGETARCH=linux-x64

WORKDIR /azp

COPY ./azuredevops-agent/agents/base/start.sh start.sh
RUN chmod +x start.sh

Dockerfile for the powershell image (the resulting image which contains the actual CVE):

FROM ***redacted***/pipeline/azdevops-base:latest

# # Add Microsoft signing key
# https://learn.microsoft.com/en-us/linux/packages#how-to-install-microsoft-software-packages-using-the-linux-repository
RUN curl -sL https://packages.microsoft.com/config/debian/11/packages-microsoft-prod.deb -O packages-microsoft-prod.deb | bash \
  && dpkg -i packages-microsoft-prod.deb \
  && rm packages-microsoft-prod.deb \
  && apt update

# # Install powershell
RUN DEBIAN_FRONTEND=noninteractive apt-get install -y -qq --no-install-recommends powershell=7.* \
  # add Powershell Azure (Az) module for pwsh Azure scripts (initiac.ps1)
  && pwsh -Command Install-Module -Name Az -Scope CurrentUser -Force

ENV AgentCustomCapability_Powershell=true

ENTRYPOINT ["./start.sh"]

Searching for the reported dll on the container filesystem gives:

# find | grep "System.Text.Encodings.Web"
./root/.local/share/powershell/Modules/Az.Accounts/2.13.1/lib/netfx/System.Text.Encodings.Web.dll
./usr/share/dotnet/shared/Microsoft.NETCore.App/7.0.11/System.Text.Encodings.Web.dll
./opt/microsoft/powershell/7/ref/System.Text.Encodings.Web.dll
./opt/microsoft/powershell/7/System.Text.Encodings.Web.dll

Exiftool output (affected dll):

ExifTool Version Number         : 12.16
File Name                       : System.Text.Encodings.Web.dll
Directory                       : ./root/.local/share/powershell/Modules/Az.Accounts/2.13.1/lib/netfx
File Size                       : 58 KiB
File Modification Date/Time     : 2023:09:20 05:23:00+00:00
File Access Date/Time           : 2023:10:03 12:21:48+00:00
File Inode Change Date/Time     : 2023:10:03 12:20:50+00:00
File Permissions                : rw-r--r--
File Type                       : Win32 DLL
File Type Extension             : dll
MIME Type                       : application/octet-stream
Machine Type                    : Intel 386 or later, and compatibles
Time Stamp                      : 2064:10:25 21:58:50+00:00
Image File Characteristics      : Executable, Large address aware, DLL
PE Type                         : PE32
Linker Version                  : 48.0
Code Size                       : 47616
Initialized Data Size           : 2560
Uninitialized Data Size         : 0
Entry Point                     : 0xb862
OS Version                      : 4.0
Image Version                   : 0.0
Subsystem Version               : 6.0
Subsystem                       : Windows command line
File Version Number             : 4.700.21.11602
Product Version Number          : 3.1.13.0
File Flags Mask                 : 0x003f
File Flags                      : (none)
File OS                         : Win32
Object File Type                : Dynamic link library
File Subtype                    : 0
Language Code                   : Neutral
Character Set                   : Unicode
Comments                        : System.Text.Encodings.Web
Company Name                    : Microsoft Corporation
File Description                : System.Text.Encodings.Web
File Version                    : 4.700.21.11602
Internal Name                   : System.Text.Encodings.Web.dll
Legal Copyright                 : © Microsoft Corporation. All rights reserved.
Original File Name              : System.Text.Encodings.Web.dll
Product Name                    : Microsoft® .NET Core
Product Version                 : 3.1.13+9299d909ff6c03c664a3f9a3ef995a9fa2eeec3e
Assembly Version                : 4.0.5.1

And for reference, Exiftool output (non-affected dll)

ExifTool Version Number         : 12.16
File Name                       : System.Text.Encodings.Web.dll
Directory                       : ./usr/share/dotnet/shared/Microsoft.NETCore.App/7.0.11
File Size                       : 116 KiB
File Modification Date/Time     : 2023:08:25 05:37:53+00:00
File Access Date/Time           : 2023:08:25 05:37:53+00:00
File Inode Change Date/Time     : 2023:09:24 16:15:25+00:00
File Permissions                : rw-r--r--
File Type                       : Win32 DLL
File Type Extension             : dll
MIME Type                       : application/octet-stream
Machine Type                    : Unknown (0xfd1d)
Time Stamp                      : 2079:01:30 13:16:28+00:00
Image File Characteristics      : Executable, Large address aware, DLL
PE Type                         : PE32+
Linker Version                  : 11.0
Code Size                       : 97792
Initialized Data Size           : 19968
Uninitialized Data Size         : 0
Entry Point                     : 0x0000
OS Version                      : 4.0
Image Version                   : 0.0
Subsystem Version               : 4.0
Subsystem                       : Windows command line
File Version Number             : 7.0.1123.42427
Product Version Number          : 7.0.11.0
File Flags Mask                 : 0x003f
File Flags                      : (none)
File OS                         : Win32
Object File Type                : Dynamic link library
File Subtype                    : 0
Language Code                   : Neutral
Character Set                   : Unicode
Comments                        : Provides types for encoding and escaping strings for use in JavaScript, HyperText Markup Language (HTML), and uniform resource locators (URL)...Commonly Used Types:.System.Text.Encodings.Web.HtmlEncoder.System.Text.Encodings.Web.UrlEncoder.System.Text.Encodings.Web.JavaScriptEncoder
Company Name                    : Microsoft Corporation
File Description                : System.Text.Encodings.Web
File Version                    : 7.0.1123.42427
Internal Name                   : System.Text.Encodings.Web.dll
Legal Copyright                 : © Microsoft Corporation. All rights reserved.
Original File Name              : System.Text.Encodings.Web.dll
Product Name                    : Microsoft® .NET
Product Version                 : 7.0.11+ecb34f85ec92e1b3c814edf7da83337e199e7f66
Assembly Version                : 7.0.0.0

Issue script & Debug output

not applicable

Environment data

PS /> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      7.3.7
PSEdition                      Core
GitCommitId                    7.3.7
OS                             Linux 5.15.0-1047-azure #54~20.04.1-Ubuntu SMP Wed Sep 6 17:49:31 UTC 2023
Platform                       Unix
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

Module versions

PS /> get-installedmodule Az*

Version              Name                                Repository           Description
-------              ----                                ----------           -----------
3.1.1                Az.Websites                         PSGallery            Microsoft Azure PowerShell - App Service (Web Apps) service cmdlets for…
4.0.0                Az.DesktopVirtualization            PSGallery            Microsoft Azure PowerShell: DesktopVirtualization cmdlets
1.10.0               Az.FrontDoor                        PSGallery            Microsoft Azure PowerShell - Front Door service cmdlets for Azure Resou…
2.0.0                Az.HealthcareApis                   PSGallery            Microsoft Azure PowerShell: HealthcareApis cmdlets
1.1.1                Az.MySql                            PSGallery            Microsoft Azure PowerShell: MySql cmdlets
1.0.0                Az.MachineLearningServices          PSGallery            Microsoft Azure PowerShell: MachineLearningServices cmdlets
1.2.0                Az.RedisEnterpriseCache             PSGallery            Microsoft Azure PowerShell: RedisEnterpriseCache cmdlets
3.1.1                Az.Cdn                              PSGallery            Microsoft Azure PowerShell: Cdn cmdlets
2.2.0                Az.Migrate                          PSGallery            Microsoft Azure PowerShell: Migrate cmdlets
1.1.2                Az.NotificationHubs                 PSGallery            Microsoft Azure PowerShell - Notification Hubs cmdlets for Azure Resour…
2.2.2                Az.ApplicationInsights              PSGallery            Microsoft Azure PowerShell: ApplicationInsights cmdlets
1.1.3                Az.MachineLearning                  PSGallery            Microsoft Azure PowerShell - Machine Learning Web Services cmdlets for …
3.2.0                Az.ServiceFabric                    PSGallery            Microsoft Azure PowerShell - Service Fabric cmdlets for Azure Resource …
6.3.0                Az.Compute                          PSGallery            Microsoft Azure PowerShell - Compute service cmdlets for Azure Resource…
10.4.1               Az                                  PSGallery            Microsoft Azure PowerShell - Cmdlets to manage resources in Azure. This…
4.1.1                Az.ContainerRegistry                PSGallery            Microsoft Azure PowerShell - Container Registry service cmdlets for Azu…
6.11.1               Az.Resources                        PSGallery            Microsoft Azure PowerShell - Azure Resource Manager and Active Director…
5.10.1               Az.Storage                          PSGallery            Microsoft Azure PowerShell - Storage service data plane and management …
2.0.0                Az.Advisor                          PSGallery            Microsoft Azure PowerShell: Advisor cmdlets
3.0.0                Az.ManagedServices                  PSGallery            Microsoft Azure PowerShell: ManagedServices cmdlets
4.6.0                Az.Monitor                          PSGallery            Microsoft Azure PowerShell - Monitor service cmdlets for Azure Resource…
2.0.0                Az.MarketplaceOrdering              PSGallery            Microsoft Azure PowerShell: MarketplaceOrdering cmdlets
1.2.1                Az.PowerBIEmbedded                  PSGallery            Microsoft Azure PowerShell - Power BI Embedded service management cmdle…
2.7.5                Az.IotHub                           PSGallery            Microsoft Azure PowerShell - IoT Hub service cmdlets for Azure Resource…
1.0.2                Az.DevTestLabs                      PSGallery            Microsoft Azure PowerShell - DevTest Labs service cmdlets for Azure Res…
1.2.0                Az.CloudService                     PSGallery            Microsoft Azure PowerShell: CloudService cmdlets
4.10.0               Az.Sql                              PSGallery            Microsoft Azure PowerShell - SQL service cmdlets for Azure Resource Man…
3.1.0                Az.SecurityInsights                 PSGallery            Microsoft Azure PowerShell: SecurityInsights cmdlets
1.1.4                Az.AnalysisServices                 PSGallery            Microsoft Azure PowerShell - Analysis Services cmdlets for Windows Powe…
1.0.0                Az.Support                          PSGallery            Microsoft Azure PowerShell - Azure Support cmdlets for Azure Resource M…
1.6.0                Az.EventGrid                        PSGallery            Microsoft Azure PowerShell - Event Grid service cmdlets for Azure Resou…
4.12.0               Az.KeyVault                         PSGallery            Microsoft Azure PowerShell - Key Vault service cmdlets for Azure Resour…
1.0.0                Az.Automanage                       PSGallery            Microsoft Azure PowerShell: Automanage cmdlets
2.0.0                Az.Relay                            PSGallery            Microsoft Azure PowerShell: Relay cmdlets
1.9.1                Az.Automation                       PSGallery            Microsoft Azure PowerShell - Automation service cmdlets for Azure Resou…
1.0.0                Az.ArcResourceBridge                PSGallery            Microsoft Azure PowerShell: ArcResourceBridge cmdlets
2.1.0                Az.DataProtection                   PSGallery            Microsoft Azure PowerShell: DataProtection cmdlets
1.3.1                Az.Maintenance                      PSGallery            Microsoft Azure PowerShell - Maintenance cmdlets for Azure Resource Man…
2.0.0                Az.Attestation                      PSGallery            Microsoft Azure PowerShell - Attestation service cmdlets for Azure Reso…
4.1.0                Az.EventHub                         PSGallery            Microsoft Azure PowerShell - Event Hubs service cmdlets for Azure Resou…
1.1.2                Az.Media                            PSGallery            Microsoft Azure PowerShell - Media service cmdlets for Azure Resource M…
1.0.1                Az.StorageMover                     PSGallery            Microsoft Azure PowerShell: StorageMover cmdlets
3.2.2                Az.ContainerInstance                PSGallery            Microsoft Azure PowerShell: ContainerInstance cmdlets
1.6.3                Az.PolicyInsights                   PSGallery            Microsoft Azure PowerShell - Azure Policy Insights cmdlets for Windows …
1.1.0                Az.DataBoxEdge                      PSGallery            Microsoft Azure PowerShell - DataBoxEdge service cmdlets for Azure Reso…
1.0.4                Az.PrivateDns                       PSGallery            Microsoft Azure PowerShell - Private DNS service cmdlets for Azure Reso…
3.0.0                Az.ServiceBus                       PSGallery            Microsoft Azure PowerShell - Service Bus service cmdlets for Azure Reso…
3.0.3                Az.Synapse                          PSGallery            Microsoft Azure PowerShell - Azure Synapse Analytics in Windows PowerSh…
1.8.0                Az.RedisCache                       PSGallery            Microsoft Azure PowerShell - Redis Cache service cmdlets for Azure Reso…
1.0.3                Az.DataLakeAnalytics                PSGallery            Microsoft Azure PowerShell - Data Lake Analytics in Windows PowerShell …
2.0.0                Az.StorageSync                      PSGallery            Microsoft Azure PowerShell - Storage Sync cmdlets in Windows PowerShell…
1.4.0                Az.Security                         PSGallery            Microsoft Azure PowerShell - Azure Security Center cmdlets in Windows P…
3.2.0                Az.OperationalInsights              PSGallery            Microsoft Azure PowerShell - Operational Insights service cmdlets for A…
1.14.0               Az.CognitiveServices                PSGallery            Microsoft Azure PowerShell - Cognitive Services management cmdlets for …
1.0.0                Az.LoadTesting                      PSGallery            Microsoft Azure PowerShell: LoadTesting cmdlets
2.2.0                Az.Kusto                            PSGallery            Microsoft Azure PowerShell: Kusto cmdlets
2.0.0                Az.SignalR                          PSGallery            Microsoft Azure PowerShell - Azure SignalR service commands for Windows…
1.0.0                Az.ConfidentialLedger               PSGallery            Microsoft Azure PowerShell: ConfidentialLedger cmdlets
6.2.0                Az.Network                          PSGallery            Microsoft Azure PowerShell - Networking service cmdlets for Azure Resou…
6.6.0                Az.RecoveryServices                 PSGallery            Microsoft Azure PowerShell - Recovery Services cmdlets for Azure Resour…
4.0.6                Az.Functions                        PSGallery            Microsoft Azure PowerShell - Azure Functions service cmdlets for Azure …
1.2.0                Az.ResourceMover                    PSGallery            Microsoft Azure PowerShell: ResourceMover cmdlets
2.1.0                Az.SqlVirtualMachine                PSGallery            Microsoft Azure PowerShell: SqlVirtualMachine cmdlets
2.0.0                Az.StreamAnalytics                  PSGallery            Microsoft Azure PowerShell: StreamAnalytics cmdlets
1.1.3                Az.Dns                              PSGallery            Microsoft Azure PowerShell - DNS service cmdlets for Azure Resource Man…
5.5.1                Az.Aks                              PSGallery            Microsoft Azure PowerShell - Azure managed Kubernetes cmdlets for Windo…
1.5.0                Az.LogicApp                         PSGallery            Microsoft Azure PowerShell - Logic Apps cmdlets for Azure Resource Mana…
1.1.0                Az.DeploymentManager                PSGallery            PowerShell .Net Core Microsoft Azure PowerShell - Deployment Manager cm…
1.2.1                Az.TrafficManager                   PSGallery            Microsoft Azure PowerShell - Traffic Manager service cmdlets for Azure …
4.0.2                Az.ApiManagement                    PSGallery            Microsoft Azure PowerShell - Api Management service cmdlets for Azure R…
1.1.1                Az.ManagedServiceIdentity           PSGallery            Microsoft Azure PowerShell: ManagedServiceIdentity cmdlets
1.12.0               Az.CosmosDB                         PSGallery            Microsoft Azure PowerShell - CosmosDB service cmdlets for Azure Resourc…
1.0.1                Az.DataShare                        PSGallery            Microsoft Azure PowerShell - DataShare service cmdlets for Azure Resour…
2.0.3                Az.Billing                          PSGallery            Microsoft Azure PowerShell - Billing service cmdlets for Azure Resource…
1.3.0                Az.DataLakeStore                    PSGallery            Microsoft Azure PowerShell - Azure Data Lake Store cmdlets in Windows P…
2.2.0                Az.StackHCI                         PSGallery            Microsoft Azure PowerShell: StackHci cmdlets
1.7.0                Az.Databricks                       PSGallery            Microsoft Azure PowerShell: Databricks cmdlets
3.5.0                Az.Batch                            PSGallery            Microsoft Azure PowerShell - Batch service cmdlets for Azure Resource M…
1.1.0                Az.PostgreSql                       PSGallery            Microsoft Azure PowerShell: PostgreSql cmdlets
2.13.1               Az.Accounts                         PSGallery            Microsoft Azure PowerShell - Accounts credential management cmdlets for…
1.3.0                Az.AppConfiguration                 PSGallery            Microsoft Azure PowerShell: AppConfiguration cmdlets
6.0.1                Az.HDInsight                        PSGallery            Microsoft Azure PowerShell - HDInsight service cmdlets for Azure Resour…
1.17.0               Az.DataFactory                      PSGallery            Microsoft Azure PowerShell - Data Factory service cmdlets for Azure Res…

Error output

No response

isra-fel commented 10 months ago

Thanks for reporting! cc @msJinLei @vidai-msft

msJinLei commented 10 months ago

Az.Accounts actually depends on System.Text.Encodings.Web 4.7.2. But we incorrectly add old version https://github.com/Azure/azure-powershell/blob/main/src/lib/netfx/System.Text.Encodings.Web.dll to our preload dependency list . Will update it to 4.7.2 in the next release.

jorisscheppers commented 10 months ago

@msJinLei would you be able to take a look at the dependency on System.Net.Http as well? Our security scanner indicates that there is another vulnerability in this image but I'm not able to perform the same analysis as with System.Text.Encodings.Web.

This is the CVE that we are warned about: https://github.com/advisories/GHSA-7jgj-8wvc-jh57.

isra-fel commented 9 months ago

@jorisscheppers we don't rely on nor ship System.Net.Http. Where did your security scanner find that? BTW we double checked that the version of System.Text.Encodings.Web we reference is 4.7.2. It is secure.

jorisscheppers commented 9 months ago

@jorisscheppers we don't rely on nor ship System.Net.Http. Where did your security scanner find that? BTW we double checked that the version of System.Text.Encodings.Web we reference is 4.7.2. It is secure.

And yet our security scanner (which is included in Microsoft Defender for Azure and uses the Qualys container image scanner tech) keeps indicating that image contains the System.Text.Encodings.Web version that has this vulnerability. I'm not sure how the scanner concludes that a vulnerable version of that dll is present on the image. I imagine that it either checks a manifest file or it uses exif information to determine version numbers. Would it help if I extracted the actual dll from the image for reference?

Regarding System.Net.Http: it might be a dependency created by another step in the image build process. I will check this out.

isra-fel commented 9 months ago

Microsoft Defender for Azure and uses the Qualys container image scanner tech

@msJinLei could you try that out?

Would it help if I extracted the actual dll from the image for reference?

@jorisscheppers thanks but that won't be necessary. I ran ExifTool and got the same results as yours. It is indeed 4.7.2. And we have been using 4.7.2 since at least last December, see https://github.com/Azure/azure-powershell/blob/main/src/lib/netfx/System.Text.Encodings.Web.dll

isra-fel commented 7 months ago

In summary, the version of System.Text.Encodings.Web we ship is absolutely secure. We'll track the issue that Defender detected the CVE internally. Thanks.