search

Azure / azure-powershell

Microsoft Azure PowerShell
Other
4.26k stars 3.86k forks source link

Set-AzStorageBlobContent : Failed to open file C:\Something.log: Illegal characters in path. #12220

Closed MarkKharitonov closed 4 years ago

MarkKharitonov commented 4 years ago

Description

Related to #8531, only happens locally. Not running any Azure Runbook.

Steps to reproduce

$k = Get-AzStorageAccountKey -Name $StorageAccountName -ResourceGroupName $ResourceGroupName | Select-Object -First 1 -ExpandProperty Value
$ctx = New-AzStorageContext -StorageAccountName $StorageAccountName -StorageAccountKey $k
Set-AzStorageBlobContent -File $SourceFile -Container $ContainerName -Context $ctx.Context -Blob $BlobName -Force
Failed to open file C:\Users\mkharitonov\AppData\Local\Temp\a10392_20200619135813\plan-data-resources.tfplan: Illegal characters in path..
At C:\Users\mkharitonov\.LocalTestPSModules\dayforce.ps.hosting\0.0.20171.32320\Dayforce.PS.Hosting.psm1:485 char:5
+     Set-AzStorageBlobContent -File $SourceFile -Container $ContainerN ...
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : CloseError: (:) [Set-AzStorageBlobContent], TransferException
    + FullyQualifiedErrorId : TransferException,Microsoft.WindowsAzure.Commands.Storage.Blob.SetAzureBlobContentCommand

Environment data

Name                           Value
----                           -----
PSVersion                      5.1.17763.1007
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.17763.1007
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1

Module versions

    Directory: C:\Users\mkharitonov\.LocalTestPSModules

ModuleType Version    Name                                ExportedCommands                                                                                                                      
---------- -------    ----                                ----------------                                                                                                                      
Script     0.0.201... Dayforce.PS.Dev                     {Add-DBUpgradeToPath, Add-DFVersionToPath, Add-SecretsFromBuildDefinitionToAzKeyVault, Add-TFSToolToPath...}                          
Script     0.0.201... Dayforce.PS.Hosting                 {Convert-GuidToSid, Get-CloudProductContext, Get-CloudRootContext, Get-ImpervaIPRanges...}                                            

    Directory: C:\Users\mkharitonov\Documents\WindowsPowerShell\Modules

ModuleType Version    Name                                ExportedCommands                                                                                                                      
---------- -------    ----                                ----------------                                                                                                                      
Script     1.8.1      Az.Accounts                         {Disable-AzDataCollection, Disable-AzContextAutosave, Enable-AzDataCollection, Enable-AzContextAutosave...}                           
Script     1.8.0      Az.Accounts                         {Disable-AzDataCollection, Disable-AzContextAutosave, Enable-AzDataCollection, Enable-AzContextAutosave...}                           
Script     1.7.5      Az.Accounts                         {Disable-AzDataCollection, Disable-AzContextAutosave, Enable-AzDataCollection, Enable-AzContextAutosave...}                           
Script     1.6.3      Az.Accounts                         {Disable-AzDataCollection, Disable-AzContextAutosave, Enable-AzDataCollection, Enable-AzContextAutosave...}                           
Script     1.1.1      Az.Advisor                          {Get-AzAdvisorRecommendation, Enable-AzAdvisorRecommendation, Disable-AzAdvisorRecommendation, Get-AzAdvisorConfiguration...}         
Script     1.0.3      Az.Aks                              {Get-AzAks, New-AzAks, Remove-AzAks, Import-AzAksCredential...}                                                                       
Script     1.1.2      Az.AnalysisServices                 {Resume-AzAnalysisServicesServer, Suspend-AzAnalysisServicesServer, Get-AzAnalysisServicesServer, Remove-AzAnalysisServicesServer...} 
Script     1.4.1      Az.ApiManagement                    {Add-AzApiManagementApiToProduct, Add-AzApiManagementProductToGroup, Add-AzApiManagementRegion, Add-AzApiManagementUserToGroup...}    
Script     1.0.3      Az.ApplicationInsights              {Get-AzApplicationInsights, New-AzApplicationInsights, Remove-AzApplicationInsights, Set-AzApplicationInsightsPricingPlan...}         
Script     1.3.6      Az.Automation                       {Get-AzAutomationHybridWorkerGroup, Remove-AzAutomationHybridWorkerGroup, Get-AzAutomationJobOutputRecord, Import-AzAutomationDscNo...
Script     2.0.2      Az.Batch                            {Remove-AzBatchAccount, Get-AzBatchAccount, Get-AzBatchAccountKey, New-AzBatchAccount...}                                             
Script     1.0.2      Az.Billing                          {Get-AzBillingInvoice, Get-AzBillingPeriod, Get-AzEnrollmentAccount, Get-AzConsumptionBudget...}                                      
Script     1.4.3      Az.Cdn                              {Get-AzCdnProfile, Get-AzCdnProfileSsoUrl, New-AzCdnProfile, Remove-AzCdnProfile...}                                                  
Script     1.3.0      Az.CognitiveServices                {Get-AzCognitiveServicesAccount, Get-AzCognitiveServicesAccountKey, Get-AzCognitiveServicesAccountSku, Get-AzCognitiveServicesAccou...
Script     3.7.0      Az.Compute                          {Remove-AzAvailabilitySet, Get-AzAvailabilitySet, New-AzAvailabilitySet, Update-AzAvailabilitySet...}                                 
Script     1.0.3      Az.ContainerInstance                {New-AzContainerGroup, Get-AzContainerGroup, Remove-AzContainerGroup, Get-AzContainerInstanceLog}                                     
Script     1.1.1      Az.ContainerRegistry                {New-AzContainerRegistry, Get-AzContainerRegistry, Update-AzContainerRegistry, Remove-AzContainerRegistry...}                         
Script     1.1.0      Az.DataBoxEdge                      {Get-AzDataBoxEdgeJob, Get-AzDataBoxEdgeDevice, Invoke-AzDataBoxEdgeDevice, New-AzDataBoxEdgeDevice...}                               
Script     1.7.0      Az.DataFactory                      {Set-AzDataFactoryV2, Update-AzDataFactoryV2, Get-AzDataFactoryV2, Remove-AzDataFactoryV2...}                                         
Script     1.0.2      Az.DataLakeAnalytics                {Get-AzDataLakeAnalyticsDataSource, New-AzDataLakeAnalyticsCatalogCredential, Remove-AzDataLakeAnalyticsCatalogCredential, Set-AzDa...
Script     1.2.8      Az.DataLakeStore                    {Get-AzDataLakeStoreTrustedIdProvider, Remove-AzDataLakeStoreTrustedIdProvider, Remove-AzDataLakeStoreFirewallRule, Set-AzDataLakeS...
Script     1.1.0      Az.DeploymentManager                {Get-AzDeploymentManagerArtifactSource, New-AzDeploymentManagerArtifactSource, Set-AzDeploymentManagerArtifactSource, Remove-AzDepl...
Script     1.0.2      Az.DevTestLabs                      {Get-AzDtlAllowedVMSizesPolicy, Get-AzDtlAutoShutdownPolicy, Get-AzDtlAutoStartPolicy, Get-AzDtlVMsPerLabPolicy...}                   
Script     1.1.2      Az.Dns                              {Get-AzDnsRecordSet, New-AzDnsRecordConfig, Remove-AzDnsRecordSet, Set-AzDnsRecordSet...}                                             
Script     1.2.3      Az.EventGrid                        {New-AzEventGridTopic, Get-AzEventGridTopic, Set-AzEventGridTopic, New-AzEventGridTopicKey...}                                        
Script     1.4.3      Az.EventHub                         {New-AzEventHubNamespace, Get-AzEventHubNamespace, Set-AzEventHubNamespace, Remove-AzEventHubNamespace...}                            
Script     1.4.0      Az.FrontDoor                        {New-AzFrontDoor, Get-AzFrontDoor, Set-AzFrontDoor, Remove-AzFrontDoor...}                                                            
Script     3.1.0      Az.HDInsight                        {Get-AzHDInsightJob, New-AzHDInsightSqoopJobDefinition, Wait-AzHDInsightJob, New-AzHDInsightStreamingMapReduceJobDefinition...}       
Script     1.0.1      Az.HealthcareApis                   {New-AzHealthcareApisService, Remove-AzHealthcareApisService, Set-AzHealthcareApisService, Get-AzHealthcareApisService}               
Script     2.4.0      Az.IotHub                           {Add-AzIotHubKey, Get-AzIotHubEventHubConsumerGroup, Get-AzIotHubConnectionString, Get-AzIotHubJob...}                                
Script     1.6.0      Az.KeyVault                         {Add-AzKeyVaultCertificate, Update-AzKeyVaultCertificate, Stop-AzKeyVaultCertificateOperation, Get-AzKeyVaultCertificateOperation...} 
Script     1.3.1      Az.KeyVault                         {Add-AzKeyVaultCertificate, Update-AzKeyVaultCertificate, Stop-AzKeyVaultCertificateOperation, Get-AzKeyVaultCertificateOperation...} 
Script     1.3.2      Az.LogicApp                         {Get-AzIntegrationAccountAgreement, Get-AzIntegrationAccountAssembly, Get-AzIntegrationAccountBatchConfiguration, Get-AzIntegration...
Script     1.1.3      Az.MachineLearning                  {Move-AzMlCommitmentAssociation, Get-AzMlCommitmentAssociation, Get-AzMlCommitmentPlanUsageHistory, Remove-AzMlCommitmentPlan...}     
Script     1.0.0      Az.Maintenance                      {Get-AzApplyUpdate, Get-AzConfigurationAssignment, Get-AzMaintenanceConfiguration, Get-AzMaintenanceUpdate...}                        
Script     1.0.2      Az.ManagedServices                  {Get-AzManagedServicesAssignment, New-AzManagedServicesAssignment, Remove-AzManagedServicesAssignment, Get-AzManagedServicesDefinit...
Script     1.0.2      Az.MarketplaceOrdering              {Get-AzMarketplaceTerms, Set-AzMarketplaceTerms}                                                                                      
Script     1.1.1      Az.Media                            {Sync-AzMediaServiceStorageKey, Set-AzMediaServiceKey, Get-AzMediaServiceKey, Get-AzMediaServiceNameAvailability...}                  
Script     1.7.0      Az.Monitor                          {Get-AzMetricDefinition, Get-AzMetric, Remove-AzLogProfile, Get-AzLogProfile...}                                                      
Script     2.5.0      Az.Network                          {Add-AzApplicationGatewayAuthenticationCertificate, Get-AzApplicationGatewayAuthenticationCertificate, New-AzApplicationGatewayAuth...
Script     1.1.1      Az.NotificationHubs                 {Get-AzNotificationHub, Get-AzNotificationHubAuthorizationRule, Get-AzNotificationHubListKey, Get-AzNotificationHubPNSCredential...}  
Script     1.3.4      Az.OperationalInsights              {New-AzOperationalInsightsAzureActivityLogDataSource, New-AzOperationalInsightsCustomLogDataSource, Disable-AzOperationalInsightsLi...
Script     1.3.0      Az.PolicyInsights                   {Get-AzPolicyEvent, Get-AzPolicyState, Get-AzPolicyStateSummary, Get-AzPolicyRemediation...}                                          
Script     1.1.1      Az.PowerBIEmbedded                  {Remove-AzPowerBIWorkspaceCollection, Get-AzPowerBIWorkspaceCollection, Get-AzPowerBIWorkspaceCollectionAccessKey, Get-AzPowerBIWor...
Script     1.0.2      Az.PrivateDns                       {Get-AzPrivateDnsZone, Remove-AzPrivateDnsZone, Set-AzPrivateDnsZone, New-AzPrivateDnsZone...}                                        
Script     2.8.0      Az.RecoveryServices                 {Get-AzRecoveryServicesBackupProperty, Get-AzRecoveryServicesVault, Get-AzRecoveryServicesVaultSettingsFile, New-AzRecoveryServices...
Script     1.2.1      Az.RedisCache                       {Remove-AzRedisCachePatchSchedule, New-AzRedisCacheScheduleEntry, Get-AzRedisCachePatchSchedule, New-AzRedisCachePatchSchedule...}    
Script     1.0.3      Az.Relay                            {New-AzRelayNamespace, Get-AzRelayNamespace, Set-AzRelayNamespace, Remove-AzRelayNamespace...}                                        
Script     1.13.0     Az.Resources                        {Get-AzProviderOperation, Remove-AzRoleAssignment, Get-AzRoleAssignment, New-AzRoleAssignment...}                                     
Script     1.4.1      Az.ServiceBus                       {New-AzServiceBusNamespace, Get-AzServiceBusNamespace, Set-AzServiceBusNamespace, Remove-AzServiceBusNamespace...}                    
Script     2.0.2      Az.ServiceFabric                    {Add-AzServiceFabricClientCertificate, Add-AzServiceFabricClusterCertificate, Add-AzServiceFabricNode, Add-AzServiceFabricNodeType...}
Script     1.1.1      Az.SignalR                          {New-AzSignalR, Get-AzSignalR, Get-AzSignalRKey, New-AzSignalRKey...}                                                                 
Script     2.7.0      Az.Sql                              {Get-AzSqlDatabaseTransparentDataEncryption, Get-AzSqlDatabaseTransparentDataEncryptionActivity, Set-AzSqlDatabaseTransparentDataEn...
Script     2.6.1      Az.Sql                              {Get-AzSqlDatabaseTransparentDataEncryption, Get-AzSqlDatabaseTransparentDataEncryptionActivity, Set-AzSqlDatabaseTransparentDataEn...
Script     2.6.0      Az.Sql                              {Get-AzSqlDatabaseTransparentDataEncryption, Get-AzSqlDatabaseTransparentDataEncryptionActivity, Set-AzSqlDatabaseTransparentDataEn...
Script     1.1.0      Az.SqlVirtualMachine                {New-AzSqlVM, Get-AzSqlVM, Update-AzSqlVM, Remove-AzSqlVM...}                                                                         
Script     2.1.0      Az.Storage                          {Get-AzStorageAccount, Get-AzStorageAccountKey, New-AzStorageAccount, New-AzStorageAccountKey...}                                     
Script     2.0.0      Az.Storage                          {Get-AzStorageAccount, Get-AzStorageAccountKey, New-AzStorageAccount, New-AzStorageAccountKey...}                                     
Script     1.14.0     Az.Storage                          {Get-AzStorageAccount, Get-AzStorageAccountKey, New-AzStorageAccount, New-AzStorageAccountKey...}                                     
Script     1.2.3      Az.StorageSync                      {Invoke-AzStorageSyncCompatibilityCheck, New-AzStorageSyncService, Get-AzStorageSyncService, Remove-AzStorageSyncService...}          
Script     1.0.1      Az.StreamAnalytics                  {Get-AzStreamAnalyticsFunction, Get-AzStreamAnalyticsDefaultFunctionDefinition, New-AzStreamAnalyticsFunction, Remove-AzStreamAnaly...
Script     1.0.0      Az.Support                          {Get-AzSupportService, Get-AzSupportProblemClassification, Get-AzSupportTicket, Get-AzSupportTicketCommunication...}                  
Script     1.0.3      Az.TrafficManager                   {Add-AzTrafficManagerCustomHeaderToEndpoint, Remove-AzTrafficManagerCustomHeaderFromEndpoint, Add-AzTrafficManagerCustomHeaderToPro...
Script     1.8.0      Az.Websites                         {Get-AzAppServicePlan, Set-AzAppServicePlan, New-AzAppServicePlan, Remove-AzAppServicePlan...}                                        
Script     1.0.201... Dayforce.PS.Core                    {Write-HostIfNotVerbose, Get-ElapsedSecondsString, Register-PSRepositoryEx, Use-Module...}                                            
Script     1.0.201... Dayforce.PS.Core                    {Write-HostIfNotVerbose, Get-ElapsedSecondsString, Register-PSRepositoryEx, Use-Module...}                                            
Script     1.0.201... Dayforce.PS.Dev                     {Add-DBUpgradeToPath, Add-DFVersionToPath, Add-SecretsFromBuildDefinitionToAzKeyVault, Add-TFSToolToPath...}                          
Script     1.0.201... Dayforce.PS.Dev                     {Add-DBUpgradeToPath, Add-DFVersionToPath, Add-SecretsFromBuildDefinitionToAzKeyVault, Add-TFSToolToPath...}                          
Script     1.0.201... Dayforce.PS.Dev                     {Add-DBUpgradeToPath, Add-DFVersionToPath, Add-SecretsFromBuildDefinitionToAzKeyVault, Add-TFSToolToPath...}                          
Script     1.0.201... Dayforce.PS.Dev                     {Add-DBUpgradeToPath, Add-DFVersionToPath, Add-SecretsFromBuildDefinitionToAzKeyVault, Add-TFSToolToPath...}                          
Script     1.0.201... Dayforce.PS.Dev                     {Add-DBUpgradeToPath, Add-DFVersionToPath, Add-SecretsFromBuildDefinitionToAzKeyVault, Add-TFSToolToPath...}                          
Script     1.0.201... Dayforce.PS.Hosting                 {Convert-GuidToSid, Get-CloudProductContext, Get-CloudRootContext, Get-ImpervaIPRanges...}                                            
Script     1.0.201... Dayforce.PS.Hosting                 {Convert-GuidToSid, Get-CloudProductContext, Get-CloudRootContext, Get-ImpervaIPRanges...}                                            
Script     1.0.201... Dayforce.PS.Hosting                 {Convert-GuidToSid, Get-CloudProductContext, Get-CloudRootContext, Get-ImpervaIPRanges...}                                            
Script     1.0.201... Dayforce.PS.Hosting                 {Convert-GuidToSid, Get-AzureResourcesPlannedForDeletion, Get-CloudProductContext, Get-CloudRootContext...}                           
Script     1.0.201... Dayforce.PS.Hosting                 {Convert-GuidToSid, Get-AzureResourcesPlannedForDeletion, Get-CloudProductContext, Get-CloudRootContext...}                           
Script     1.0.201... Dayforce.PS.Hosting                 {Convert-GuidToSid, Get-AzureResourcesPlannedForDeletion, Get-CloudProductContext, Get-CloudRootContext...}                           
Script     1.0.201... Dayforce.PS.Hosting                 {Convert-GuidToSid, Get-AzureResourcesPlannedForDeletion, Get-CloudProductContext, Get-CloudRootContext...}                           
Script     1.0.201... Dayforce.PS.Hosting                 {Convert-GuidToSid, Get-AzureResourcesPlannedForDeletion, Get-CloudProductContext, Get-CloudRootContext...}                           
Script     1.0.201... Dayforce.PS.Hosting                 {Convert-GuidToSid, Get-AzureResourcesPlannedForDeletion, Get-CloudProductContext, Get-CloudRootContext...}                           
Script     1.0.201... Dayforce.PS.Hosting                 {Convert-GuidToSid, Get-AzureResourcesPlannedForDeletion, Get-CloudProductContext, Get-CloudRootContext...}                           
Script     1.0.201... Dayforce.PS.Hosting                 {Convert-GuidToSid, Get-AzureResourcesPlannedForDeletion, Get-CloudProductContext, Get-CloudRootContext...}                           
Script     1.0.201... Dayforce.PS.Hosting                 {Convert-GuidToSid, Get-AzureResourcesPlannedForDeletion, Get-CloudProductContext, Get-CloudRootContext...}                           
Script     1.0.201... Dayforce.PS.Hosting                 {Convert-GuidToSid, Get-AzureResourcesPlannedForDeletion, Get-CloudProductContext, Get-CloudRootContext...}                           
Script     1.0.201... Dayforce.PS.Hosting                 {Convert-GuidToSid, Get-AzureResourcesPlannedForDeletion, Get-CloudProductContext, Get-CloudRootContext...}                           
Script     1.0.201... Dayforce.PS.Hosting                 {Convert-GuidToSid, Get-AzureResourcesPlannedForDeletion, Get-CloudProductContext, Get-CloudRootContext...}                           
Script     1.0.201... Dayforce.PS.Hosting                 {Convert-GuidToSid, Get-AzureResourcesPlannedForDeletion, Get-CloudProductContext, Get-CloudRootContext...}                           
Script     1.0.201... Dayforce.PS.Hosting                 {Convert-GuidToSid, Get-AzureResourcesPlannedForDeletion, Get-CloudProductContext, Get-CloudRootContext...}                           
Script     1.0.201... Dayforce.PS.Hosting                 {Convert-GuidToSid, Copy-FileFromAzureBlob, Copy-FileToAzureBlob, Get-AzureAuthenticationToken...}                                    
Script     1.0.201... Dayforce.PS.Hosting                 {Convert-GuidToSid, Get-AzureResourcesPlannedForDeletion, Get-CloudProductContext, Get-CloudRootContext...}                           
Script     1.0.201... Dayforce.PS.Hosting                 {Convert-GuidToSid, Get-AzureResourcesPlannedForDeletion, Get-CloudProductContext, Get-CloudRootContext...}                           
Script     1.0.201... Dayforce.PS.Hosting                 {Convert-GuidToSid, Copy-FileFromAzureBlob, Copy-FileToAzureBlob, Get-AzureAuthenticationToken...}                                    
Script     1.0.201... Dayforce.PS.Hosting                 {Connect-MyAzAccount, Convert-GuidToSid, Copy-FileFromAzureBlob, Copy-FileToAzureBlob...}                                             
Script     1.0.201... Dayforce.PS.Hosting                 {Convert-GuidToSid, Copy-FileFromAzureBlob, Copy-FileToAzureBlob, Get-AzureAuthenticationToken...}                                    
Script     1.0.1.141  newtonsoft.json                     {ConvertFrom-JsonNewtonsoft, ConvertTo-JsonNewtonsoft}                                                                                
Script     1.4.7      PackageManagement                   {Find-Package, Get-Package, Get-PackageProvider, Get-PackageSource...}                                                                
Script     0.7.3      posh-git                            {Invoke-NullCoalescing, Add-PoshGitToProfile, Get-PromptPath, Write-GitStatus...}                                                     
Script     0.1.7      PowerHTML                           ConvertFrom-Html                                                                                                                      
Script     2.2.4.1    PowerShellGet                       {Find-Command, Find-DSCResource, Find-Module, Find-RoleCapability...}                                                                 
Binary     0.8.1      SHiPS                                                                                                                                                                     
Binary     1.0.4.2    StrongNaming                        {Get-AssemblyName, Import-StrongNameKeyPair, Set-StrongName, Test-StrongName}                                                         
Script     0.0.2      Trackyon.Utils                      {Backup-Location, Get-OperatingSystem, New-Directory, Set-As...}                                                                      
Script     6.4.3      VSTeam                              {Add-VSTeam, Add-VSTeamAccessControlEntry, Add-VSTeamAzureRMServiceEndpoint, Add-VSTeamBuild...}                                      
Script     6.3.4      VSTeam                              {Add-VSTeam, Add-VSTeamAccessControlEntry, Add-VSTeamAzureRMServiceEndpoint, Add-VSTeamBuild...}                                      

    Directory: C:\Program Files\WindowsPowerShell\Modules

ModuleType Version    Name                                ExportedCommands                                                                                                                      
---------- -------    ----                                ----------------                                                                                                                      
Script     4.2.1      Azure.Storage                       {Get-AzureStorageTable, New-AzureStorageTableSASToken, New-AzureStorageTableStoredAccessPolicy, New-AzureStorageTable...}             
Manifest   4.0.0.0    CertificateDsc                                                                                                                                                            
Manifest   1.3.1      DSCR_Shortcut                                                                                                                                                             
Script     1.0.1      Microsoft.PowerShell.Operation.V... {Get-OperationValidation, Invoke-OperationValidation}                                                                                 
Binary     1.0.0.1    PackageManagement                   {Find-Package, Get-Package, Get-PackageProvider, Get-PackageSource...}                                                                
Script     3.4.0      Pester                              {Describe, Context, It, Should...}                                                                                                    
Script     0.7.3      posh-git                            {Invoke-NullCoalescing, Add-PoshGitToProfile, Get-PromptPath, Write-GitStatus...}                                                     
Script     1.0.0.1    PowerShellGet                       {Install-Module, Find-Module, Save-Module, Update-Module...}                                                                          
Script     2.0.0      PSReadline                          {Get-PSReadLineKeyHandler, Set-PSReadLineKeyHandler, Remove-PSReadLineKeyHandler, Get-PSReadLineOption...}                            
Manifest   2.2.0.0    SecurityPolicyDsc                                                                                                                                                         
Manifest   11.1.0.0   SqlServerDsc                                                                                                                                                              
Script     8.2.0.0    xPSDesiredStateConfiguration        {Publish-MOFToPullServer, Publish-DSCModuleAndMof, Publish-ModuleToPullServer}                                                        
Manifest   2.0.0.0    xSmbShare                                                                                                                                                                 
Manifest   1.7.0.0    xTimeZone                                                                                                                                                                 
Script     0.1        xVisualStudio2015                                                                                                                                                         

    Directory: C:\WINDOWS\system32\WindowsPowerShell\v1.0\Modules

ModuleType Version    Name                                ExportedCommands                                                                                                                      
---------- -------    ----                                ----------------                                                                                                                      
Manifest   1.0.0.0    AppBackgroundTask                   {Disable-AppBackgroundTaskDiagnosticLog, Enable-AppBackgroundTaskDiagnosticLog, Set-AppBackgroundTaskResourcePolicy, Unregister-App...
Manifest   2.0.0.0    AppLocker                           {Get-AppLockerFileInformation, Get-AppLockerPolicy, New-AppLockerPolicy, Set-AppLockerPolicy...}                                      
Manifest   1.0.0.0    AppvClient                          {Add-AppvClientConnectionGroup, Add-AppvClientPackage, Add-AppvPublishingServer, Disable-Appv...}                                     
Manifest   2.0.1.0    Appx                                {Add-AppxPackage, Get-AppxPackage, Get-AppxPackageManifest, Remove-AppxPackage...}                                                    
Script     1.0.0.0    AssignedAccess                      {Clear-AssignedAccess, Get-AssignedAccess, Set-AssignedAccess}                                                                        
Manifest   1.0.0.0    BitLocker                           {Unlock-BitLocker, Suspend-BitLocker, Resume-BitLocker, Remove-BitLockerKeyProtector...}                                              
Manifest   2.0.0.0    BitsTransfer                        {Add-BitsFile, Complete-BitsTransfer, Get-BitsTransfer, Remove-BitsTransfer...}                                                       
Manifest   1.0.0.0    BranchCache                         {Add-BCDataCacheExtension, Clear-BCCache, Disable-BC, Disable-BCDowngrading...}                                                       
Manifest   1.0.0.0    CimCmdlets                          {Get-CimAssociatedInstance, Get-CimClass, Get-CimInstance, Get-CimSession...}                                                         
Manifest   1.0        ConfigCI                            {Get-SystemDriver, New-CIPolicyRule, New-CIPolicy, Get-CIPolicy...}                                                                   
Manifest   1.0        Defender                            {Get-MpPreference, Set-MpPreference, Add-MpPreference, Remove-MpPreference...}                                                        
Manifest   1.0.1.0    DeliveryOptimization                {Get-DeliveryOptimizationStatus, Get-DeliveryOptimizationPerfSnap, Get-DeliveryOptimizationLog, Get-DOConfig...}                      
Manifest   1.0.0.0    DirectAccessClientComponents        {Disable-DAManualEntryPointSelection, Enable-DAManualEntryPointSelection, Get-DAClientExperienceConfiguration, Get-DAEntryPointTabl...
Script     3.0        Dism                                {Add-AppxProvisionedPackage, Add-WindowsDriver, Add-WindowsCapability, Add-WindowsImage...}                                           
Manifest   1.0.0.0    DnsClient                           {Resolve-DnsName, Clear-DnsClientCache, Get-DnsClient, Get-DnsClientCache...}                                                         
Manifest   1.0.0.0    EventTracingManagement              {Start-EtwTraceSession, New-EtwTraceSession, Get-EtwTraceSession, Update-EtwTraceSession...}                                          
Manifest   1.0.0.0    HgsClient                           {Get-HgsAttestationBaselinePolicy, Get-HgsClientConfiguration, Test-HgsClientConfiguration, Set-HgsClientConfiguration...}            
Manifest   1.0.0.0    HgsDiagnostics                      {New-HgsTraceTarget, Get-HgsTrace, Get-HgsTraceFileData, Test-HgsTraceTarget}                                                         
Binary     1.0.0.0    HostComputeService                  {Get-ComputeProcess, Stop-ComputeProcess}                                                                                             
Manifest   1.0.0.1    HostNetworkingService               {Remove-HnsNamespace, Remove-HnsEndpoint, Get-HnsEndpoint, Remove-HnsPolicyList...}                                                   
Binary     2.0.0.0    Hyper-V                             {Add-VMAssignableDevice, Add-VMDvdDrive, Add-VMFibreChannelHba, Add-VMGpuPartitionAdapter...}                                         
Binary     1.1        Hyper-V                             {Add-VMDvdDrive, Add-VMFibreChannelHba, Add-VMHardDiskDrive, Add-VMMigrationNetwork...}                                               
Script     1.1.0.0    IISAdministration                   {Get-IISAppPool, Start-IISCommitDelay, Stop-IISCommitDelay, Get-IISSite...}                                                           
Manifest   2.0.0.0    International                       {Get-WinDefaultInputMethodOverride, Set-WinDefaultInputMethodOverride, Get-WinHomeLocation, Set-WinHomeLocation...}                   
Manifest   1.0.0.0    iSCSI                               {Get-IscsiTargetPortal, New-IscsiTargetPortal, Remove-IscsiTargetPortal, Update-IscsiTargetPortal...}                                 
Script     1.0.0.0    ISE                                 {New-IseSnippet, Import-IseSnippet, Get-IseSnippet}                                                                                   
Manifest   1.0.0.0    Kds                                 {Add-KdsRootKey, Get-KdsRootKey, Test-KdsRootKey, Set-KdsConfiguration...}                                                            
Manifest   1.0.1.0    Microsoft.PowerShell.Archive        {Compress-Archive, Expand-Archive}                                                                                                    
Manifest   3.0.0.0    Microsoft.PowerShell.Diagnostics    {Get-WinEvent, Get-Counter, Import-Counter, Export-Counter...}                                                                        
Manifest   3.0.0.0    Microsoft.PowerShell.Host           {Start-Transcript, Stop-Transcript}                                                                                                   
Manifest   1.0.0.0    Microsoft.PowerShell.LocalAccounts  {Add-LocalGroupMember, Disable-LocalUser, Enable-LocalUser, Get-LocalGroup...}                                                        
Manifest   3.1.0.0    Microsoft.PowerShell.Management     {Add-Content, Clear-Content, Clear-ItemProperty, Join-Path...}                                                                        
Script     1.0        Microsoft.PowerShell.ODataUtils     Export-ODataEndpointProxy                                                                                                             
Manifest   3.0.0.0    Microsoft.PowerShell.Security       {Get-Acl, Set-Acl, Get-PfxCertificate, Get-Credential...}                                                                             
Manifest   3.1.0.0    Microsoft.PowerShell.Utility        {Format-List, Format-Custom, Format-Table, Format-Wide...}                                                                            
Manifest   3.0.0.0    Microsoft.WSMan.Management          {Disable-WSManCredSSP, Enable-WSManCredSSP, Get-WSManCredSSP, Set-WSManQuickConfig...}                                                
Manifest   1.0        MMAgent                             {Disable-MMAgent, Enable-MMAgent, Set-MMAgent, Get-MMAgent...}                                                                        
Manifest   1.0.0.0    MsDtc                               {New-DtcDiagnosticTransaction, Complete-DtcDiagnosticTransaction, Join-DtcDiagnosticResourceManager, Receive-DtcDiagnosticTransacti...
Manifest   2.0.0.0    NetAdapter                          {Disable-NetAdapter, Disable-NetAdapterBinding, Disable-NetAdapterChecksumOffload, Disable-NetAdapterEncapsulatedPacketTaskOffload...}
Manifest   1.0.0.0    NetConnection                       {Get-NetConnectionProfile, Set-NetConnectionProfile}                                                                                  
Manifest   1.0.0.0    NetDiagnostics                      Get-NetView                                                                                                                           
Manifest   1.0.0.0    NetEventPacketCapture               {New-NetEventSession, Remove-NetEventSession, Get-NetEventSession, Set-NetEventSession...}                                            
Manifest   2.0.0.0    NetLbfo                             {Add-NetLbfoTeamMember, Add-NetLbfoTeamNic, Get-NetLbfoTeam, Get-NetLbfoTeamMember...}                                                
Manifest   1.0.0.0    NetNat                              {Get-NetNat, Get-NetNatExternalAddress, Get-NetNatStaticMapping, Get-NetNatSession...}                                                
Manifest   2.0.0.0    NetQos                              {Get-NetQosPolicy, Set-NetQosPolicy, Remove-NetQosPolicy, New-NetQosPolicy}                                                           
Manifest   2.0.0.0    NetSecurity                         {Get-DAPolicyChange, New-NetIPsecAuthProposal, New-NetIPsecMainModeCryptoProposal, New-NetIPsecQuickModeCryptoProposal...}            
Manifest   1.0.0.0    NetSwitchTeam                       {New-NetSwitchTeam, Remove-NetSwitchTeam, Get-NetSwitchTeam, Rename-NetSwitchTeam...}                                                 
Manifest   1.0.0.0    NetTCPIP                            {Get-NetIPAddress, Get-NetIPInterface, Get-NetIPv4Protocol, Get-NetIPv6Protocol...}                                                   
Manifest   1.0.0.0    NetWNV                              {Get-NetVirtualizationProviderAddress, Get-NetVirtualizationGlobal, Get-NetVirtualizationLookupRecord, Get-NetVirtualizationCustome...
Manifest   1.0.0.0    NetworkConnectivityStatus           {Get-DAConnectionStatus, Get-NCSIPolicyConfiguration, Reset-NCSIPolicyConfiguration, Set-NCSIPolicyConfiguration}                     
Manifest   1.0.0.0    NetworkSwitchManager                {Disable-NetworkSwitchEthernetPort, Enable-NetworkSwitchEthernetPort, Get-NetworkSwitchEthernetPort, Remove-NetworkSwitchEthernetPo...
Manifest   1.0.0.0    NetworkTransition                   {Add-NetIPHttpsCertBinding, Disable-NetDnsTransitionConfiguration, Disable-NetIPHttpsProfile, Disable-NetNatTransitionConfiguration...
Manifest   1.0.0.0    PcsvDevice                          {Get-PcsvDevice, Start-PcsvDevice, Stop-PcsvDevice, Restart-PcsvDevice...}                                                            
Binary     1.0.0.0    PersistentMemory                    {Get-PmemDisk, Get-PmemPhysicalDevice, Get-PmemUnusedRegion, New-PmemDisk...}                                                         
Manifest   1.0.0.0    PKI                                 {Add-CertificateEnrollmentPolicyServer, Export-Certificate, Export-PfxCertificate, Get-CertificateAutoEnrollmentPolicy...}            
Manifest   1.0.0.0    PnpDevice                           {Get-PnpDevice, Get-PnpDeviceProperty, Enable-PnpDevice, Disable-PnpDevice}                                                           
Manifest   1.1        PrintManagement                     {Add-Printer, Add-PrinterDriver, Add-PrinterPort, Get-PrintConfiguration...}                                                          
Binary     1.0.11     ProcessMitigations                  {Get-ProcessMitigation, Set-ProcessMitigation, ConvertTo-ProcessMitigationPolicy}                                                     
Script     3.0        Provisioning                        {Install-ProvisioningPackage, Export-ProvisioningPackage, Install-TrustedProvisioningCertificate, Export-Trace...}                    
Manifest   1.1        PSDesiredStateConfiguration         {Set-DscLocalConfigurationManager, Start-DscConfiguration, Test-DscConfiguration, Publish-DscConfiguration...}                        
Script     1.0.0.0    PSDiagnostics                       {Disable-PSTrace, Disable-PSWSManCombinedTrace, Disable-WSManTrace, Enable-PSTrace...}                                                
Binary     1.1.0.0    PSScheduledJob                      {New-JobTrigger, Add-JobTrigger, Remove-JobTrigger, Get-JobTrigger...}                                                                
Manifest   2.0.0.0    PSWorkflow                          {New-PSWorkflowExecutionOption, New-PSWorkflowSession, nwsn}                                                                          
Manifest   1.0.0.0    PSWorkflowUtility                   Invoke-AsWorkflow                                                                                                                     
Manifest   1.0.0.0    ScheduledTasks                      {Get-ScheduledTask, Set-ScheduledTask, Register-ScheduledTask, Unregister-ScheduledTask...}                                           
Manifest   2.0.0.0    SecureBoot                          {Confirm-SecureBootUEFI, Set-SecureBootUEFI, Get-SecureBootUEFI, Format-SecureBootUEFI...}                                            
Manifest   2.0.0.0    SmbShare                            {Get-SmbShare, Remove-SmbShare, Set-SmbShare, Block-SmbShareAccess...}                                                                
Manifest   2.0.0.0    SmbWitness                          {Get-SmbWitnessClient, Move-SmbWitnessClient, gsmbw, msmbw...}                                                                        
Manifest   1.0.0.0    StartLayout                         {Export-StartLayout, Import-StartLayout, Export-StartLayoutEdgeAssets, Get-StartApps}                                                 
Manifest   2.0.0.0    Storage                             {Add-InitiatorIdToMaskingSet, Add-PartitionAccessPath, Add-PhysicalDisk, Add-StorageFaultDomain...}                                   
Manifest   1.0.0.0    StorageBusCache                     {Clear-StorageBusDisk, Disable-StorageBusCache, Disable-StorageBusDisk, Enable-StorageBusCache...}                                    
Manifest   2.0.0.0    TLS                                 {New-TlsSessionTicketKey, Enable-TlsSessionTicketKey, Disable-TlsSessionTicketKey, Export-TlsSessionTicketKey...}                     
Manifest   1.0.0.0    TroubleshootingPack                 {Get-TroubleshootingPack, Invoke-TroubleshootingPack}                                                                                 
Manifest   2.0.0.0    TrustedPlatformModule               {Get-Tpm, Initialize-Tpm, Clear-Tpm, Unblock-Tpm...}                                                                                  
Binary     2.1.639.0  UEV                                 {Clear-UevConfiguration, Clear-UevAppxPackage, Restore-UevBackup, Set-UevTemplateProfile...}                                          
Manifest   2.0.0.0    VpnClient                           {Add-VpnConnection, Set-VpnConnection, Remove-VpnConnection, Get-VpnConnection...}                                                    
Manifest   1.0.0.0    Wdac                                {Get-OdbcDriver, Set-OdbcDriver, Get-OdbcDsn, Add-OdbcDsn...}                                                                         
Manifest   1.0.0.0    WebAdministration                   {Start-WebCommitDelay, Stop-WebCommitDelay, Get-WebConfigurationLock, Remove-WebConfigurationLock...}                                 
Manifest   1.0.0.0    WindowsDeveloperLicense             {Get-WindowsDeveloperLicense, Unregister-WindowsDeveloperLicense, Show-WindowsDeveloperLicenseRegistration}                           
Script     1.0        WindowsErrorReporting               {Enable-WindowsErrorReporting, Disable-WindowsErrorReporting, Get-WindowsErrorReporting}                                              
Manifest   1.0.0.0    WindowsSearch                       {Get-WindowsSearchSetting, Set-WindowsSearchSetting}                                                                                  
Manifest   1.0.0.0    WindowsUpdate                       Get-WindowsUpdateLog                                                                                                                  
Manifest   1.0.0.2    WindowsUpdateProvider               {Get-WUAVersion, Get-WULastInstallationDate, Get-WULastScanSuccessDate, Get-WUIsPendingReboot...}                                     

    Directory: C:\Program Files (x86)\Microsoft SQL Server\130\Tools\PowerShell\Modules

ModuleType Version    Name                                ExportedCommands                                                                                                                      
---------- -------    ----                                ----------------                                                                                                                      
Manifest   1.0        SQLPS                               {Backup-SqlDatabase, Save-SqlMigrationReport, Add-SqlAvailabilityDatabase, Add-SqlAvailabilityGroupListenerStaticIp...}               

    Directory: C:\Users\mkharitonov\.vscode\extensions\ms-vscode.powershell-2019.12.0\modules

ModuleType Version    Name                                ExportedCommands                                                                                                                      
---------- -------    ----                                ----------------                                                                                                                      
Script     1.1.3      Plaster                             {Invoke-Plaster, New-PlasterManifest, Get-PlasterTemplate, Test-PlasterManifest}                                                      
Script     1.13.2     PowerShellEditorServices            {Start-EditorServicesHost, Get-PowerShellEditorServicesVersion, Compress-LogDir}                                                      
Script     0.2.0      PowerShellEditorServices.VSCode     {New-VSCodeHtmlContentView, Show-VSCodeHtmlContentView, Close-VSCodeHtmlContentView, Set-VSCodeHtmlContentView...}                    
Script     1.18.3     PSScriptAnalyzer                    {Get-ScriptAnalyzerRule, Invoke-ScriptAnalyzer, Invoke-Formatter}

Debug output

C:\Dayforce\fintech [master ≡]> $SourceFile = 'C:/Users/MKHARI~1/AppData/Local/Temp/a10392_20200619142422/plan-data-resources.tfplan'
C:\Dayforce\fintech [master ≡]> $ContainerName = 'app508-dfpg-dev2-logs'
C:\Dayforce\fintech [master ≡]> $BlobName = 'fintech-infrastructure\0.0.20171.32924\0\dev2\plan-data-resources.tfplan'
C:\Dayforce\fintech [master ≡]> $DebugPreference='Continue'
C:\Dayforce\fintech [master ≡]> Set-AzStorageBlobContent -File $SourceFile -Container $ContainerName -Context $ctx.Context -Blob $BlobName -Force
DEBUG: 2:42:19 PM - Init Operation Context for 'SetAzureBlobContentCommand' with client request id  Azure-Storage-PowerShell-658fda7a-ed10-413c-9c71-9f6055a9b326. If you want to get more
 details, please add "-Debug" to your command.
DEBUG: 2:42:19 PM - Use storage account 'app508cerlogs' from storage context.
DEBUG: 2:42:19 PM - SetAzureBlobContentCommand end processing, Start 0 remote calls. Finish 0 remote calls. Elapsed time 31382.88 ms. Client operation id:
Azure-Storage-PowerShell-658fda7a-ed10-413c-9c71-9f6055a9b326.
DEBUG: 2:42:19 PM - SetAzureBlobContentCommand end processing.
Set-AzStorageBlobContent : Failed to open file C:\Users\mkharitonov\AppData\Local\Temp\a10392_20200619142422\plan-data-resources.tfplan: Illegal characters in path..
At line:1 char:1
+ Set-AzStorageBlobContent -File $SourceFile -Container $ContainerName  ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : CloseError: (:) [Set-AzStorageBlobContent], TransferException
    + FullyQualifiedErrorId : TransferException,Microsoft.WindowsAzure.Commands.Storage.Blob.SetAzureBlobContentCommand

C:\Dayforce\fintech [master ≡]>

So, there is this SO question - https://stackoverflow.com/questions/54522744/set-azstorageblobcontent-throws-exception-illegal-characters-in-path. The proposals are:

  1. Modify powershell.exe.config - not applicable when there is no control over the environment.
  2. Use Azure.Storage module instead of Az.Storage. We are using other Az modules. Do we need to move away from all the Az modules? Because it does not seem possible to use Az modules for all except Storage and use Azure.Storage. Please, correct me if I am wrong.

Now I was trying to check the latest preview version of the Az.Storage module, but it does not seem possible with Powershell 5.1:

PS C:\Users\mkharitonov> install-module Az.Storage -Repository PSGallery -AllowPrerelease -Force
PackageManagement\Install-Package : The version '4.0.2' of the module 'Az.Storage' being installed is not catalog signed. Ensure that the version '4.0.2' of the module 'Az.Storage'
has the catalog file 'Az.Storage.cat' and signed with the same publisher 'CN=Microsoft Corporation, O=Microsoft Corporation, L=Redmond, S=Washington, C=US' as the
previously-installed module 'Az.Storage' with version '2.1.0' under the directory 'C:\Users\mkharitonov\Documents\WindowsPowerShell\Modules\Az.Storage\2.1.0'. If you still want to
install or update, use -SkipPublisherCheck parameter.
At C:\Users\mkharitonov\Documents\WindowsPowerShell\Modules\powershellget\2.2.4.1\PSModule.psm1:9709 char:34
+ ... talledPackages = PackageManagement\Install-Package @PSBoundParameters
+                      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (Microsoft.Power....InstallPackage:InstallPackage) [Install-Package], Exception
    + FullyQualifiedErrorId : ModuleIsNotCatalogSigned,Validate-ModuleAuthenticodeSignature,Microsoft.PowerShell.PackageManagement.Cmdlets.InstallPackage

PS C:\Users\mkharitonov>

Please, advise.

ghost commented 4 years ago

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @xgithubtriage.

dingmeng-xue commented 4 years ago

Storage team, please help to look into this question.

blueww commented 4 years ago

@MarkKharitonov

Thanks for reporting this issue! This is the same issue of https://github.com/Azure/azure-powershell/issues/8531 Do you run this cmdlet in Azure automation?

To answer your question:

  1. Az.Storage 4.0.2 is not for normal Powershell user. If you just want to use common scenarios of Storage, please install the latest stable version 2.1.0 without "-AllowPrerelease" parameter.
  2. Please don't use Azure.Storage together with Az module on same machine.
  3. The latest Az.Storage still can repro this issue. The root cause is the API "System.IO.Path.GetFullPath()" not work with UNC path on your machine. The workaround is change Powershell config to make the API work. Upgrade OS might can also make it work.
  4. If you can't take the workaround, you might can consider to upgrade the blob with storage API directly if your blob is not too big. Following is a sample. 
    PS C:\> $container = Get-AzStorageContainer $containerName -Context $ctx
PS C:\> $blob = $container.CloudBlobContainer.GetBlockBlobReference($blobName)
PS C:\> $blob.UploadFromFile($localDestFile)
MarkKharitonov commented 4 years ago

Please, let me repeat the first sentence in this issue:

Related to #8531, only happens locally. Not running any Azure Runbook.

About item (4) - Could you provide a fully functional sample? Because the blob can be quite large and we need to deal with asynchronous nature of the call, i.e. wait for its completion. Please, do not spare the vital details, since this issue is very annoying to say the least.

Thank you.

lukeb1961 commented 4 years ago

"The latest Az.Storage still can repro this issue. The root cause is the API "System.IO.Path.GetFullPath()" not work with UNC path on your machine. The workaround is change Powershell config to make the API work. Upgrade OS might can also make it work."

I am running Win10 20150 and AZ 4.2.0 and Az.Storage 2.1.0- this is BROKEN. To suggest I alter the Powershell config of my systems is INSANE.

Would someone in the AZ team acknowledge that a previously functional cmdlet is now BROKEN? Please get it FIXED.

blueww commented 4 years ago

@MarkKharitonov @lukeb1961

Thanks for the responds!

For fix this in PSH, We have discussed this for a long time before, there's no easy fix in PSH code. As the issue happens in Azure runbook before, so we fixed runbook to make it work with the API. Per before discussion, upgrade to Windows 10 Anniversary, this issue should not appear. Would you please run [environment]::OSVersion in Powershell console which can repro this issue, and give the detail output. Then I can check related person to see why the API not work in your OS. (I know you use Win10 20150, but would like to get more info.)

For the script, would you please share your upload scenario, like the blob Type (block/page/append), the average size of the blob? And we can see if there's a more stable way for the blob upload by call SDK API. If your blob is really large, you might can also try to call AzCopy command in PSH to upload blob.

Besides that, please share the detail requirements that you need to write the script. Do you start upload asynchronously, and wait for it complete with a task? You can share your original script to use this cmdlets (hide credential), and I can see how to switch it to SDK API.

MarkKharitonov commented 4 years ago

Here are all the details (mind the prompt, it is from within VS Code debugger):

 [DBG]> [environment]::OSVersion

Platform ServicePack Version      VersionString
-------- ----------- -------      -------------
 Win32NT             10.0.17763.0 Microsoft Windows NT 10.0.17763.0

 [DBG]> $BlobName
fintech-infrastructure\0.0.20176.25730\0\dev2\plan-data-resources.tfplan
 [DBG]> dir $SourceFile

    Directory: C:\Users\mkharitonov\AppData\Local\Temp\a8440_20200624101820

Mode                LastWriteTime         Length Name
----                -------------         ------ ----
-a----        6/24/2020  10:19 AM          12404 plan-data-resources.tfplan

 [DBG]> [io.path]::GetFullPath($SourceFile)
C:\Users\mkharitonov\AppData\Local\Temp\a8440_20200624101820\plan-data-resources.tfplan
 [DBG]> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      5.1.17763.1007
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.17763.1007
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1

 [DBG]> Set-AzStorageBlobContent -File $SourceFile -Container $ContainerName -Context $ctx.Context -Blob $BlobName -Force
Failed to open file C:\Users\mkharitonov\AppData\Local\Temp\a8440_20200624101820\plan-data-resources.tfplan: Illegal characters in path..
At line:1 char:1
+ Set-AzStorageBlobContent -File $SourceFile -Container $ContainerName  ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : CloseError: (:) [Set-AzStorageBlobContent], TransferException
    + FullyQualifiedErrorId : TransferException,Microsoft.WindowsAzure.Commands.Storage.Blob.SetAzureBlobContentCommand
 [DBG]>

As you can see:

  1. No UNC paths are involved.
  2. [io.path]::GetFullPath() works fine.

The files we upload are Terraform plans (small) and terraform verbose logs (megabytes)

The problem is that the Az.Storage module authors convert the given path to UNC themselves by utilizing the \\? notation, which does not work out of the box:

 [DBG]> dir "\\?\c:\temp"
Illegal characters in path.
At line:1 char:1
+ dir "\\?\c:\temp"
+ ~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Get-ChildItem], ArgumentException
    + FullyQualifiedErrorId : System.ArgumentException,Microsoft.PowerShell.Commands.GetChildItemCommand
 [DBG]>

It does work on a machine where the powershell.exe.config was modified:

C:\> dir "\\?\c:\temp"

    Directory: \\?\c:

Mode                LastWriteTime         Length Name
----                -------------         ------ ----
d-----        6/19/2020   5:10 PM                temp

C:\>

Now I understand the rationale behind this conversion - to bypass the maximum path length imposed by Windows on the standard paths which does not apply to \\? paths. But come on, you have to be smarter than that and do it ONLY if the given path is actually over the limit, because the current implementation is just broken for ALL paths.

And the advice to modify the powershell.exe.config is awful - have you actually tried it yourself? The only way I could do it is first change the ownership of the file from TrustedInstaller to myself and only then I could add myself to the ACL and give me the write permissions. And what would happen on the next update of powershell? Will the installer be able to overwrite it? In short, this is a miserable workaround that does not scale.

You have to fix your mess. You have to take into account that the code is broken for many people out there. Be professional, apply the \\? conversion only if you really need it as a fallback mechanism in case the max path error has actually occurred. I do not believe I need to give such an obvious advice here.

Please, take responsibility and fix the Az.Storage module already.

blueww commented 4 years ago

@MarkKharitonov

Thanks for the detail!

  1. The issue is caused by the .net API System.IO.Path.GetFullPath(), it should support UNC path from .net 4.6.2 framework per link. But currently seems not and cause the issue. As you have shared your OS/PSH version, I will contact the .net team to see what should be the correct behavior, and the correct way to handle path.

  2. PowerShell based on DMlib to do the data transfer, and the related code for UNC path is in DMlib. I have opened an issue in DMlib. After DMlib fix this issue, Powershell can fix it by upgrade to new version of DMlib. I have discussed with DMlib team for this, current all long/short file path will be convert to UNC path in DMlib since: 1. .net API declared to support UNC path; 2. separate code path to handle long/short file path will make code difficult to maintain. DMlib team will evaluate if/how to fix it in DMlib , depends on the reply from .net team.

  3. Per you scenario, the files to upload looks are small ( "small" or "megabytes" per your above comments). Before the issue is fixed, If you don't want to apply the workaround by modify the powershell.exe.config, Would you like to upload it with SDK? 

    
# Get blob object to upload with
$container = Get-AzStorageContainer $containerName -Context $ctx
$blob = $container.CloudBlobContainer.GetBlockBlobReference($blobName)

Upload sync, will return when upload finish

$blob.UploadFromFile($localSrcFile)

Start upload async and create a task, then wait for task finish.

$t = $blob.UploadFromFileAsync($localSrcFile) $t.Wait()

MarkKharitonov commented 4 years ago

We will try to upload with SDK. We work in sprints, so it may take a few weeks for us to get there.

blueww commented 4 years ago

@MarkKharitonov Thanks for the update! Let us know if you need any assistance on upload with SDK.

We have contact .net team, and get their comment "If PowerShell doesn’t have a problem with the setup logic, the most likely scenario is that something installed has mussed with PowerShell’s config file. "

So to investigate why the issue happens on your machine (it should not happen after Windows 10 Anniversary):

  1. Could you share the powershell.exe.config file content (hide credential if any), on the machine which can repro the issue and before you do any change to workaround this issue. Especially , does it contains any setting of "Switch.System.IO.UseLegacyPathHandling"?

  2. We can't repro this issue locally on same OS. If you can share how you setup such a repro machine from clean OS, it might can help us to setup a repro environment and investigate the root cause.

MarkKharitonov commented 4 years ago

Hi, The machines where the issue reproduces 100% are our developer workstations. On the build agents the issue is intermittent. On the workstations the powershell.exe.config is:

<configuration>
  <uri>
    <schemeSettings>
      <add name="http" genericUriParserOptions="DontUnescapePathDotsAndSlashes"/>
      <add name="https" genericUriParserOptions="DontUnescapePathDotsAndSlashes"/>
    </schemeSettings>
  </uri>
</configuration>

At least on the 4 workstations I saw it. Adding

  <runtime>
    <AppContextSwitchOverrides value="Switch.System.IO.UseLegacyPathHandling=false" />
  </runtime>

removes the problem, but it is not a solution in general. The workstations are imaged from a set of "gold" images used by our IT. I do not know what other information I can provide on it. If you have some command to collect more stats, then I can run it and share the results.

I have not gotten to apply the SDK solution yet.

blueww commented 4 years ago

@MarkKharitonov Thanks for the reply! I have shared the above information with .net team, and will update you if get any responds from them.

shellby1 commented 4 years ago

i use https://docs.microsoft.com/da-dk/archive/blogs/jeremykuhne/new-net-path-handling-sneak-peek as a work around

MarkKharitonov commented 4 years ago

We ended up with the following code based on your recommendation:

    $ctx = Get-StorageContext $ResourceGroupName $AccountName
    $container = Get-AzStorageContainer $ContainerName -Context $ctx
    $blob = $container.CloudBlobContainer.GetBlockBlobReference($blobName)
    $blob.UploadFromFile($SourceFile)

I will report on how it works for us.

blueww commented 4 years ago

@MarkKharitonov Thanks for the update! Let me know if anything need our assistance.

jusnitlive commented 4 years ago

We ended up with the following code based on your recommendation:

    $ctx = Get-StorageContext $ResourceGroupName $AccountName
    $container = Get-AzStorageContainer $ContainerName -Context $ctx
    $blob = $container.CloudBlobContainer.GetBlockBlobReference($blobName)
    $blob.UploadFromFile($SourceFile)

I will report on how it works for us.

@MarkKharitonov is this working for you without the modifications to powershell.exe.config? Or does it require setting Switch.System.IO.UseLegacyPathHandling=false? I am still dealing with having to reset this for hybrid runbook workers.

MarkKharitonov commented 4 years ago

@jusnitlive - this code does not require messing with powershell.exe.config

jusnitlive commented 4 years ago

https://github.com/Azure/azure-powershell/issues/8473#issuecomment-609847693

I can confirm that this fixes my issue. Add string value: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\.NETFramework\AppContext Value name: Switch.System.IO.UseLegacyPathHandling Value data: false

@MarkKharitonov Set-AzStorageBlobContent works with no code modifications

blueww commented 4 years ago

We are working closely with DMlib team to fix this issue asap. We expect the fix will be out in 1-2 months.

blueww commented 4 years ago

The Powershell fix PR is merged :https://github.com/Azure/azure-powershell/pull/12882, target to release on 9/22. Code change is upgrade to the latest DMlib release with the fix.

blueww commented 4 years ago

The fixed build have released in Az 4.7.0 (Az.Storage 2.6.0)

You can get the release here:

Download the install package from Github: https://github.com/Azure/azure-powershell/releases/tag/v4.7.0-September2020 Install the package from PSGallery: https://www.powershellgallery.com/packages/Az/4.7.0 WW

akashsawant1992 commented 2 years ago

@blueww , I got same issue of "illegal path" while uploading .json file into blob. I was using az.storage 1.5.0 , as suggested above try installing az.storage 2.6.0 ( also tried latest 4.2.0 ) and imported module from there while I was uploading and didnt fix the issue.

Only issue got fixed after registry hack and az.storage 1.5.0: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft.NETFramework\AppContext Value name: Switch.System.IO.UseLegacyPathHandling Value data: false

Any idea is this issue resolved by Microsoft in which version of az.storage ? as didnt work with 2.6.0 az.storage

blueww commented 2 years ago

@akashsawant1992 This issue is fixed in Az.Storage 2.6.0 and later. (You can use the latest Az.Storage) Please make sure you have close and reopen the Powershell session, the correct version of Az.Storage is loaded, the test again.

The workaround from you also works, since the root cause is: the .net API System.IO.Path.GetFullPath(), it should support UNC path from .net 4.6.2 framework per link but currently seems not. And your registry make this API works with UNC path.