search

Azure / azure-powershell

Microsoft Azure PowerShell
Other
4.26k stars 3.86k forks source link

Unable to deserialize response for get-AzDataFactoryV2 #14162

Closed BeneHa closed 3 years ago

BeneHa commented 3 years ago

Description

Hi, @NowinskiK discovered this bug here: https://github.com/SQLPlayer/azure.datafactory.tools/issues/85 When I run Get-AzDataFactoryV2LinkedService against my ADF, I get an error due to the linked service below. If you need any more information, please let me know! Thanks

{
    "name": "REST_AUTHBASIC_GEN",
    "type": "Microsoft.DataFactory/factories/linkedservices",
    "properties": {
        "parameters": {
            "baseUrl": {
                "type": "string",
                "defaultValue": "*********"
            },
            "authSecret": {
                "type": "string",
                "defaultValue": "none"
            }
        },
        "annotations": [],
        "type": "RestService",
        "typeProperties": {
            "url": "@{linkedService().baseUrl}",
            "enableServerCertificateValidation": true,
            "authenticationType": "Basic",
            "userName": "*********",
            "password": "@{linkedService().authSecret}"
        }
    }
}

Environment data

2021-02-09T12:37:01.4005246Z Name                           Value                                                                                   
2021-02-09T12:37:01.4019914Z ----                           -----                                                                                   
2021-02-09T12:37:01.4036660Z PSVersion                      5.1.14393.3866                                                                          
2021-02-09T12:37:01.4037395Z PSEdition                      Desktop                                                                                 
2021-02-09T12:37:01.4038159Z PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}                                                                 
2021-02-09T12:37:01.4040107Z BuildVersion                   10.0.14393.3866                                                                         
2021-02-09T12:37:01.4040758Z CLRVersion                     4.0.30319.42000                                                                         
2021-02-09T12:37:01.4041403Z WSManStackVersion              3.0                                                                                     
2021-02-09T12:37:01.4042100Z PSRemotingProtocolVersion      2.3                                                                                     
2021-02-09T12:37:01.4042734Z SerializationVersion           1.1.0.1

Module versions

I am running this from DevOps, Microsoft hosted agent, the only thing I install is:

Install-Module -Name azure.datafactory.tools -Scope CurrentUser -Force
Import-Module -Name azure.datafactory.tools

Debug output

2021-02-08T18:28:58.5176788Z Azure Data Factory (instance) loaded.
2021-02-08T18:28:59.7374802Z DataSets: 20 object(s) loaded.
2021-02-08T18:29:00.8439176Z IntegrationRuntimes: 2 object(s) loaded.
2021-02-08T18:29:01.1717909Z ##[debug]Error record:
2021-02-08T18:29:01.2492297Z ##[debug]Get-AzDataFactoryV2LinkedService : Unable to deserialize the response.
2021-02-08T18:29:01.2504029Z ##[debug]At C:\Users\VssAdministrator\Documents\WindowsPowerShell\Modules\azure.datafactory.tools\0.50.0\public\Get-AdfFromService.ps1:44 char:27
2021-02-08T18:29:01.2516885Z ##[debug]+ ... dServices = Get-AzDataFactoryV2LinkedService -ResourceGroupName "$Res ...
2021-02-08T18:29:01.2528934Z ##[debug]+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2021-02-08T18:29:01.2540544Z ##[debug]    + CategoryInfo          : CloseError: (:) [Get-AzDataFactoryV2LinkedService], SerializationException
2021-02-08T18:29:01.2552972Z ##[debug]    + FullyQualifiedErrorId : Microsoft.Azure.Commands.DataFactoryV2.GetAzureDataFactoryLinkedServiceCommand
2021-02-08T18:29:01.2563554Z ##[debug] 
2021-02-08T18:29:01.2580886Z ##[debug]Script stack trace:
2021-02-08T18:29:01.2614328Z ##[debug]at Get-AdfFromService, C:\Users\VssAdministrator\Documents\WindowsPowerShell\Modules\azure.datafactory.tools\0.50.0\public\Get-AdfFromService.ps1: line 44
2021-02-08T18:29:01.2627206Z ##[debug]at <ScriptBlock>, D:\a\_temp\397591c2-b806-460c-9108-abec7135dcd6.ps1: line 39
2021-02-08T18:29:01.2640417Z ##[debug]at <ScriptBlock>, <No file>: line 1
2021-02-08T18:29:01.2657880Z ##[debug]Exception:
2021-02-08T18:29:01.2745660Z ##[debug]Microsoft.Rest.SerializationException: Unable to deserialize the response. ---> Newtonsoft.Json.JsonReaderException: Error reading JObject from JsonReader. Current JsonReader item is not an object: String. Path 'typeProperties.password', line 1, position 2808.
2021-02-08T18:29:01.2756374Z ##[debug]   at Newtonsoft.Json.Linq.JObject.Load(JsonReader reader, JsonLoadSettings settings)
2021-02-08T18:29:01.2767925Z ##[debug]   at Microsoft.Rest.Serialization.PolymorphicDeserializeJsonConverter`1.ReadJson(JsonReader reader, Type objectType, Object existingValue, JsonSerializer serializer)
2021-02-08T18:29:01.2781510Z ##[debug]   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.DeserializeConvertable(JsonConverter converter, JsonReader reader, Type objectType, Object existingValue)
2021-02-08T18:29:01.2792862Z ##[debug]   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)
2021-02-08T18:29:01.2802888Z ##[debug]   at Newtonsoft.Json.Linq.JToken.ToObject(Type objectType, JsonSerializer jsonSerializer)
2021-02-08T18:29:01.2813976Z ##[debug]   at Microsoft.Rest.Serialization.PolymorphicDeserializeJsonConverter`1.ReadJson(JsonReader reader, Type objectType, Object existingValue, JsonSerializer serializer)
2021-02-08T18:29:01.2830678Z ##[debug]   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.DeserializeConvertable(JsonConverter converter, JsonReader reader, Type objectType, Object existingValue)
2021-02-08T18:29:01.2860423Z ##[debug]   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty containerProperty, JsonReader reader, Object target)
2021-02-08T18:29:01.2870877Z ##[debug]   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id)
2021-02-08T18:29:01.2881562Z ##[debug]   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
2021-02-08T18:29:01.2892034Z ##[debug]   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateList(IList list, JsonReader reader, JsonArrayContract contract, JsonProperty containerProperty, String id)
2021-02-08T18:29:01.2911488Z ##[debug]   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateList(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, Object existingValue, String id)
2021-02-08T18:29:01.2936270Z ##[debug]   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty containerProperty, JsonReader reader, Object target)
2021-02-08T18:29:01.2949225Z ##[debug]   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id)
2021-02-08T18:29:01.2962256Z ##[debug]   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
2021-02-08T18:29:01.2972620Z ##[debug]   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)
2021-02-08T18:29:01.2986614Z ##[debug]   at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType)
2021-02-08T18:29:01.2998535Z ##[debug]   at Microsoft.Rest.Serialization.SafeJsonConvert.DeserializeObject[T](String json, JsonSerializerSettings settings)
2021-02-08T18:29:01.3009068Z ##[debug]   at Microsoft.Azure.Management.DataFactory.LinkedServicesOperations.<ListByFactoryWithHttpMessagesAsync>d__5.MoveNext()
2021-02-08T18:29:01.3019700Z ##[debug]   --- End of inner exception stack trace ---
2021-02-08T18:29:01.3030327Z ##[debug]   at Microsoft.Azure.Management.DataFactory.LinkedServicesOperations.<ListByFactoryWithHttpMessagesAsync>d__5.MoveNext()
2021-02-08T18:29:01.3040510Z ##[debug]--- End of stack trace from previous location where exception was thrown ---
2021-02-08T18:29:01.3051684Z ##[debug]   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
2021-02-08T18:29:01.3062202Z ##[debug]   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
2021-02-08T18:29:01.3074609Z ##[debug]   at Microsoft.Azure.Management.DataFactory.LinkedServicesOperationsExtensions.<ListByFactoryAsync>d__1.MoveNext()
2021-02-08T18:29:01.3085232Z ##[debug]--- End of stack trace from previous location where exception was thrown ---
2021-02-08T18:29:01.3097779Z ##[debug]   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
2021-02-08T18:29:01.3107934Z ##[debug]   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
2021-02-08T18:29:01.3119011Z ##[debug]   at Microsoft.Azure.Management.DataFactory.LinkedServicesOperationsExtensions.ListByFactory(ILinkedServicesOperations operations, String resourceGroupName, String factoryName)
2021-02-08T18:29:01.3129323Z ##[debug]   at Microsoft.Azure.Commands.DataFactoryV2.DataFactoryClient.ListLinkedServices(AdfEntityFilterOptions filterOptions)
2021-02-08T18:29:01.3140276Z ##[debug]   at Microsoft.Azure.Commands.DataFactoryV2.DataFactoryClient.FilterPSLinkedServices(AdfEntityFilterOptions filterOptions)
2021-02-08T18:29:01.3152830Z ##[debug]   at Microsoft.Azure.Commands.DataFactoryV2.GetAzureDataFactoryLinkedServiceCommand.ExecuteCmdlet()
2021-02-08T18:29:01.3161391Z ##[debug]   at Microsoft.WindowsAzure.Commands.Utilities.Common.AzurePSCmdlet.ProcessRecord()
2021-02-08T18:29:01.3391470Z ##[error]Unable to deserialize the response.
2021-02-08T18:29:01.3405939Z ##[debug]Processed: ##vso[task.logissue type=error]Unable to deserialize the response.
2021-02-08T18:29:01.4281449Z ##[debug]Exit code: 1
2021-02-08T18:29:01.4755596Z ##[debug]Leaving Invoke-VstsTool.
2021-02-08T18:29:01.4756553Z ##[error]PowerShell exited with code '1'.
2021-02-08T18:29:01.4757537Z ##[debug]Processed: ##vso[task.logissue type=error]PowerShell exited with code '1'.
2021-02-08T18:29:01.4790415Z ##[debug]Processed: ##vso[task.complete result=Failed]Error detected
2021-02-08T18:29:01.4808687Z ##[debug]Loading module from path 'D:\a\_tasks\AzurePowerShell_72a1931b-effb-4d2e-8fd8-f8472a07cb62\5.179.0\ps_modules\VstsAzureHelpers_\VstsAzureHelpers_.psm1'.
2021-02-08T18:29:01.5217853Z ##[debug]$OVERRIDING $global:DebugPreference from 'Continue' to 'SilentlyContinue'.
2021-02-08T18:29:01.5636401Z ##[debug]Loading resource strings from: D:\a\_tasks\AzurePowerShell_72a1931b-effb-4d2e-8fd8-f8472a07cb62\5.179.0\ps_modules\VstsAzureHelpers_/module.json
2021-02-08T18:29:01.5980430Z ##[debug]Loaded 13 strings.
2021-02-08T18:29:01.5981026Z ##[debug]SYSTEM_CULTURE: 'en-US'
2021-02-08T18:29:01.6022620Z ##[debug]Loading resource strings from: D:\a\_tasks\AzurePowerShell_72a1931b-effb-4d2e-8fd8-f8472a07cb62\5.179.0\ps_modules\VstsAzureHelpers_\Strings\resources.resjson\en-US\resources.resjson
2021-02-08T18:29:01.6311941Z ##[debug]Loaded 13 strings.

Error output

2021-02-09T12:51:48.9094113Z WARNING: Upcoming breaking changes in the cmdlet 'Resolve-AzError' :
2021-02-09T12:51:48.9094619Z 
2021-02-09T12:51:48.9095217Z The `Resolve-Error` alias will be removed in a future release.  Please change any scripts that use this alias to use 
2021-02-09T12:51:48.9096337Z `Resolve-AzError` instead.
2021-02-09T12:51:48.9096645Z 
2021-02-09T12:51:48.9097272Z Note : Go to https://aka.ms/azps-changewarnings for steps to suppress this breaking change warning, and other 
2021-02-09T12:51:48.9098046Z information on breaking changes in Azure PowerShell.
2021-02-09T12:51:48.9554248Z 
2021-02-09T12:51:48.9564390Z 
2021-02-09T12:51:48.9565441Z    HistoryId: 1
2021-02-09T12:51:48.9565914Z 
2021-02-09T12:51:48.9566594Z 
2021-02-09T12:51:48.9616029Z Message        : Unable to deserialize the response.
2021-02-09T12:51:48.9638723Z StackTrace     :    at Microsoft.Azure.Management.DataFactory.LinkedServicesOperations.<ListByFactoryWithHttpMessagesAs
2021-02-09T12:51:48.9639612Z                  ync>d__5.MoveNext()
2021-02-09T12:51:48.9640302Z                  --- End of stack trace from previous location where exception was thrown ---
2021-02-09T12:51:48.9640927Z                     at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
2021-02-09T12:51:48.9641569Z                     at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
2021-02-09T12:51:48.9657480Z                     at Microsoft.Azure.Management.DataFactory.LinkedServicesOperationsExtensions.<ListByFactoryAsync>d_
2021-02-09T12:51:48.9658150Z                  _1.MoveNext()
2021-02-09T12:51:48.9659595Z                  --- End of stack trace from previous location where exception was thrown ---
2021-02-09T12:51:48.9660244Z                     at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
2021-02-09T12:51:48.9661522Z                     at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
2021-02-09T12:51:48.9676475Z                     at Microsoft.Azure.Management.DataFactory.LinkedServicesOperationsExtensions.ListByFactory(ILinkedS
2021-02-09T12:51:48.9677481Z                  ervicesOperations operations, String resourceGroupName, String factoryName)
2021-02-09T12:51:48.9678008Z                     at 
2021-02-09T12:51:48.9678488Z                  Microsoft.Azure.Commands.DataFactoryV2.DataFactoryClient.ListLinkedServices(AdfEntityFilterOptions 
2021-02-09T12:51:48.9679654Z                  filterOptions)
2021-02-09T12:51:48.9698301Z                     at Microsoft.Azure.Commands.DataFactoryV2.DataFactoryClient.FilterPSLinkedServices(AdfEntityFilterO
2021-02-09T12:51:48.9698864Z                  ptions filterOptions)
2021-02-09T12:51:48.9699948Z                     at Microsoft.Azure.Commands.DataFactoryV2.GetAzureDataFactoryLinkedServiceCommand.ExecuteCmdlet()
2021-02-09T12:51:48.9700965Z                     at Microsoft.WindowsAzure.Commands.Utilities.Common.AzurePSCmdlet.ProcessRecord()
2021-02-09T12:51:48.9701494Z Exception      : Microsoft.Rest.SerializationException
2021-02-09T12:51:48.9701922Z InvocationInfo : {Get-AzDataFactoryV2LinkedService}
2021-02-09T12:51:48.9702497Z Line           :     $adf.LinkedServices = Get-AzDataFactoryV2LinkedService -ResourceGroupName "$ResourceGroupName" 
2021-02-09T12:51:48.9703807Z                  -DataFactoryName "$FactoryName" | ToArray
2021-02-09T12:51:48.9704194Z                  
2021-02-09T12:51:48.9725050Z Position       : At C:\Users\VssAdministrator\Documents\WindowsPowerShell\Modules\azure.datafactory.tools\0.50.0\public
2021-02-09T12:51:48.9725683Z                  \Get-AdfFromService.ps1:44 char:27
2021-02-09T12:51:48.9726650Z                  + ... dServices = Get-AzDataFactoryV2LinkedService -ResourceGroupName "$Res ...
2021-02-09T12:51:48.9727611Z                  +                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2021-02-09T12:51:48.9728404Z HistoryId      : 1
2021-02-09T12:51:48.9728867Z 
2021-02-09T12:51:48.9761844Z Message        : Error reading JObject from JsonReader. Current JsonReader item is not an object: String. Path 
2021-02-09T12:51:48.9762602Z                  'typeProperties.password', line 1, position 2808.
2021-02-09T12:51:48.9763157Z StackTrace     :    at Newtonsoft.Json.Linq.JObject.Load(JsonReader reader, JsonLoadSettings settings)
2021-02-09T12:51:48.9763827Z                     at Microsoft.Rest.Serialization.PolymorphicDeserializeJsonConverter`1.ReadJson(JsonReader reader, 
2021-02-09T12:51:48.9765487Z                  Type objectType, Object existingValue, JsonSerializer serializer)
2021-02-09T12:51:48.9766163Z                     at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.DeserializeConvertable(JsonConverter 
2021-02-09T12:51:48.9767263Z                  converter, JsonReader reader, Type objectType, Object existingValue)
2021-02-09T12:51:48.9767915Z                     at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type 
2021-02-09T12:51:48.9769016Z                  objectType, Boolean checkAdditionalContent)
2021-02-09T12:51:48.9769592Z                     at Newtonsoft.Json.Linq.JToken.ToObject(Type objectType, JsonSerializer jsonSerializer)
2021-02-09T12:51:48.9770967Z                     at Microsoft.Rest.Serialization.PolymorphicDeserializeJsonConverter`1.ReadJson(JsonReader reader, 
2021-02-09T12:51:48.9771583Z                  Type objectType, Object existingValue, JsonSerializer serializer)
2021-02-09T12:51:48.9773008Z                     at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.DeserializeConvertable(JsonConverter 
2021-02-09T12:51:48.9773594Z                  converter, JsonReader reader, Type objectType, Object existingValue)
2021-02-09T12:51:48.9774162Z                     at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty 
2021-02-09T12:51:48.9774923Z                  property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty 
2021-02-09T12:51:48.9775475Z                  containerProperty, JsonReader reader, Object target)
2021-02-09T12:51:48.9776073Z                     at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, 
2021-02-09T12:51:48.9777108Z                  JsonReader reader, JsonObjectContract contract, JsonProperty member, String id)
2021-02-09T12:51:48.9777710Z                     at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type 
2021-02-09T12:51:48.9779400Z                  objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, 
2021-02-09T12:51:48.9780116Z                  JsonProperty containerMember, Object existingValue)
2021-02-09T12:51:48.9780660Z                     at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateList(IList list, JsonReader 
2021-02-09T12:51:48.9781268Z                  reader, JsonArrayContract contract, JsonProperty containerProperty, String id)
2021-02-09T12:51:48.9781857Z                     at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateList(JsonReader reader, Type 
2021-02-09T12:51:48.9782463Z                  objectType, JsonContract contract, JsonProperty member, Object existingValue, String id)
2021-02-09T12:51:48.9783049Z                     at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty 
2021-02-09T12:51:48.9783660Z                  property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty 
2021-02-09T12:51:48.9784214Z                  containerProperty, JsonReader reader, Object target)
2021-02-09T12:51:48.9784838Z                     at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, 
2021-02-09T12:51:48.9786199Z                  JsonReader reader, JsonObjectContract contract, JsonProperty member, String id)
2021-02-09T12:51:48.9787306Z                     at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type 
2021-02-09T12:51:48.9787994Z                  objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, 
2021-02-09T12:51:48.9788564Z                  JsonProperty containerMember, Object existingValue)
2021-02-09T12:51:48.9789129Z                     at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type 
2021-02-09T12:51:48.9790112Z                  objectType, Boolean checkAdditionalContent)
2021-02-09T12:51:48.9790627Z                     at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType)
2021-02-09T12:51:48.9791202Z                     at Microsoft.Rest.Serialization.SafeJsonConvert.DeserializeObject[T](String json, 
2021-02-09T12:51:48.9791688Z                  JsonSerializerSettings settings)
2021-02-09T12:51:48.9794958Z                     at Microsoft.Azure.Management.DataFactory.LinkedServicesOperations.<ListByFactoryWithHttpMessagesAs
2021-02-09T12:51:48.9795499Z                  ync>d__5.MoveNext()
2021-02-09T12:51:48.9796331Z Exception      : Newtonsoft.Json.JsonReaderException
2021-02-09T12:51:48.9796757Z InvocationInfo : {Get-AzDataFactoryV2LinkedService}
2021-02-09T12:51:48.9797557Z Line           :     $adf.LinkedServices = Get-AzDataFactoryV2LinkedService -ResourceGroupName "$ResourceGroupName" 
2021-02-09T12:51:48.9798064Z                  -DataFactoryName "$FactoryName" | ToArray
2021-02-09T12:51:48.9799159Z                  
2021-02-09T12:51:48.9815793Z Position       : At C:\Users\VssAdministrator\Documents\WindowsPowerShell\Modules\azure.datafactory.tools\0.50.0\public
2021-02-09T12:51:48.9816368Z                  \Get-AdfFromService.ps1:44 char:27
2021-02-09T12:51:48.9817622Z                  + ... dServices = Get-AzDataFactoryV2LinkedService -ResourceGroupName "$Res ...
2021-02-09T12:51:48.9818480Z                  +                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2021-02-09T12:51:48.9819192Z HistoryId      : 1
2021-02-09T12:51:48.9819621Z 
2021-02-09T12:51:48.9845016Z Message        : Cannot find a variable with the name 'ADF_FOLDERS'.
2021-02-09T12:51:48.9845483Z StackTrace     : 
2021-02-09T12:51:48.9846654Z Exception      : System.Management.Automation.ItemNotFoundException
2021-02-09T12:51:48.9847045Z InvocationInfo : {Get-Variable}
2021-02-09T12:51:48.9847478Z Line           : if (!(Get-Variable ADF_FOLDERS -ErrorAction:SilentlyContinue)) {
2021-02-09T12:51:48.9847891Z                  
2021-02-09T12:51:48.9865388Z Position       : At C:\Users\VssAdministrator\Documents\WindowsPowerShell\Modules\azure.datafactory.tools\0.50.0\privat
2021-02-09T12:51:48.9866003Z                  e\AdfObject.class.ps1:75 char:7
2021-02-09T12:51:48.9866994Z                  + if (!(Get-Variable ADF_FOLDERS -ErrorAction:SilentlyContinue)) {
2021-02-09T12:51:48.9867893Z                  +       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2021-02-09T12:51:48.9868602Z HistoryId      : 1
ghost commented 3 years ago

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @shawnxzq, @lmy269.

Issue Details
## Description Hi, @NowinskiK discovered this bug here: https://github.com/SQLPlayer/azure.datafactory.tools/issues/85 When I run `Get-AzDataFactoryV2LinkedService` against my ADF, I get an error due to the linked service below. If you need any more information, please let me know! Thanks ``` { "name": "REST_AUTHBASIC_GEN", "type": "Microsoft.DataFactory/factories/linkedservices", "properties": { "parameters": { "baseUrl": { "type": "string", "defaultValue": "*********" }, "authSecret": { "type": "string", "defaultValue": "none" } }, "annotations": [], "type": "RestService", "typeProperties": { "url": "@{linkedService().baseUrl}", "enableServerCertificateValidation": true, "authenticationType": "Basic", "userName": "*********", "password": "@{linkedService().authSecret}" } } } ``` ## Environment data ``` 2021-02-09T12:37:01.4005246Z Name Value 2021-02-09T12:37:01.4019914Z ---- ----- 2021-02-09T12:37:01.4036660Z PSVersion 5.1.14393.3866 2021-02-09T12:37:01.4037395Z PSEdition Desktop 2021-02-09T12:37:01.4038159Z PSCompatibleVersions {1.0, 2.0, 3.0, 4.0...} 2021-02-09T12:37:01.4040107Z BuildVersion 10.0.14393.3866 2021-02-09T12:37:01.4040758Z CLRVersion 4.0.30319.42000 2021-02-09T12:37:01.4041403Z WSManStackVersion 3.0 2021-02-09T12:37:01.4042100Z PSRemotingProtocolVersion 2.3 2021-02-09T12:37:01.4042734Z SerializationVersion 1.1.0.1 ``` ## Module versions I am running this from DevOps, Microsoft hosted agent, the only thing I install is: ```powershell: Install-Module -Name azure.datafactory.tools -Scope CurrentUser -Force Import-Module -Name azure.datafactory.tools ``` ## Debug output ``` 2021-02-08T18:28:58.5176788Z Azure Data Factory (instance) loaded. 2021-02-08T18:28:59.7374802Z DataSets: 20 object(s) loaded. 2021-02-08T18:29:00.8439176Z IntegrationRuntimes: 2 object(s) loaded. 2021-02-08T18:29:01.1717909Z ##[debug]Error record: 2021-02-08T18:29:01.2492297Z ##[debug]Get-AzDataFactoryV2LinkedService : Unable to deserialize the response. 2021-02-08T18:29:01.2504029Z ##[debug]At C:\Users\VssAdministrator\Documents\WindowsPowerShell\Modules\azure.datafactory.tools\0.50.0\public\Get-AdfFromService.ps1:44 char:27 2021-02-08T18:29:01.2516885Z ##[debug]+ ... dServices = Get-AzDataFactoryV2LinkedService -ResourceGroupName "$Res ... 2021-02-08T18:29:01.2528934Z ##[debug]+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2021-02-08T18:29:01.2540544Z ##[debug] + CategoryInfo : CloseError: (:) [Get-AzDataFactoryV2LinkedService], SerializationException 2021-02-08T18:29:01.2552972Z ##[debug] + FullyQualifiedErrorId : Microsoft.Azure.Commands.DataFactoryV2.GetAzureDataFactoryLinkedServiceCommand 2021-02-08T18:29:01.2563554Z ##[debug] 2021-02-08T18:29:01.2580886Z ##[debug]Script stack trace: 2021-02-08T18:29:01.2614328Z ##[debug]at Get-AdfFromService, C:\Users\VssAdministrator\Documents\WindowsPowerShell\Modules\azure.datafactory.tools\0.50.0\public\Get-AdfFromService.ps1: line 44 2021-02-08T18:29:01.2627206Z ##[debug]at , D:\a\_temp\397591c2-b806-460c-9108-abec7135dcd6.ps1: line 39 2021-02-08T18:29:01.2640417Z ##[debug]at , : line 1 2021-02-08T18:29:01.2657880Z ##[debug]Exception: 2021-02-08T18:29:01.2745660Z ##[debug]Microsoft.Rest.SerializationException: Unable to deserialize the response. ---> Newtonsoft.Json.JsonReaderException: Error reading JObject from JsonReader. Current JsonReader item is not an object: String. Path 'typeProperties.password', line 1, position 2808. 2021-02-08T18:29:01.2756374Z ##[debug] at Newtonsoft.Json.Linq.JObject.Load(JsonReader reader, JsonLoadSettings settings) 2021-02-08T18:29:01.2767925Z ##[debug] at Microsoft.Rest.Serialization.PolymorphicDeserializeJsonConverter`1.ReadJson(JsonReader reader, Type objectType, Object existingValue, JsonSerializer serializer) 2021-02-08T18:29:01.2781510Z ##[debug] at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.DeserializeConvertable(JsonConverter converter, JsonReader reader, Type objectType, Object existingValue) 2021-02-08T18:29:01.2792862Z ##[debug] at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent) 2021-02-08T18:29:01.2802888Z ##[debug] at Newtonsoft.Json.Linq.JToken.ToObject(Type objectType, JsonSerializer jsonSerializer) 2021-02-08T18:29:01.2813976Z ##[debug] at Microsoft.Rest.Serialization.PolymorphicDeserializeJsonConverter`1.ReadJson(JsonReader reader, Type objectType, Object existingValue, JsonSerializer serializer) 2021-02-08T18:29:01.2830678Z ##[debug] at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.DeserializeConvertable(JsonConverter converter, JsonReader reader, Type objectType, Object existingValue) 2021-02-08T18:29:01.2860423Z ##[debug] at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty containerProperty, JsonReader reader, Object target) 2021-02-08T18:29:01.2870877Z ##[debug] at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id) 2021-02-08T18:29:01.2881562Z ##[debug] at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue) 2021-02-08T18:29:01.2892034Z ##[debug] at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateList(IList list, JsonReader reader, JsonArrayContract contract, JsonProperty containerProperty, String id) 2021-02-08T18:29:01.2911488Z ##[debug] at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateList(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, Object existingValue, String id) 2021-02-08T18:29:01.2936270Z ##[debug] at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty containerProperty, JsonReader reader, Object target) 2021-02-08T18:29:01.2949225Z ##[debug] at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id) 2021-02-08T18:29:01.2962256Z ##[debug] at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue) 2021-02-08T18:29:01.2972620Z ##[debug] at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent) 2021-02-08T18:29:01.2986614Z ##[debug] at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType) 2021-02-08T18:29:01.2998535Z ##[debug] at Microsoft.Rest.Serialization.SafeJsonConvert.DeserializeObject[T](String json, JsonSerializerSettings settings) 2021-02-08T18:29:01.3009068Z ##[debug] at Microsoft.Azure.Management.DataFactory.LinkedServicesOperations.d__5.MoveNext() 2021-02-08T18:29:01.3019700Z ##[debug] --- End of inner exception stack trace --- 2021-02-08T18:29:01.3030327Z ##[debug] at Microsoft.Azure.Management.DataFactory.LinkedServicesOperations.d__5.MoveNext() 2021-02-08T18:29:01.3040510Z ##[debug]--- End of stack trace from previous location where exception was thrown --- 2021-02-08T18:29:01.3051684Z ##[debug] at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() 2021-02-08T18:29:01.3062202Z ##[debug] at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) 2021-02-08T18:29:01.3074609Z ##[debug] at Microsoft.Azure.Management.DataFactory.LinkedServicesOperationsExtensions.d__1.MoveNext() 2021-02-08T18:29:01.3085232Z ##[debug]--- End of stack trace from previous location where exception was thrown --- 2021-02-08T18:29:01.3097779Z ##[debug] at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() 2021-02-08T18:29:01.3107934Z ##[debug] at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) 2021-02-08T18:29:01.3119011Z ##[debug] at Microsoft.Azure.Management.DataFactory.LinkedServicesOperationsExtensions.ListByFactory(ILinkedServicesOperations operations, String resourceGroupName, String factoryName) 2021-02-08T18:29:01.3129323Z ##[debug] at Microsoft.Azure.Commands.DataFactoryV2.DataFactoryClient.ListLinkedServices(AdfEntityFilterOptions filterOptions) 2021-02-08T18:29:01.3140276Z ##[debug] at Microsoft.Azure.Commands.DataFactoryV2.DataFactoryClient.FilterPSLinkedServices(AdfEntityFilterOptions filterOptions) 2021-02-08T18:29:01.3152830Z ##[debug] at Microsoft.Azure.Commands.DataFactoryV2.GetAzureDataFactoryLinkedServiceCommand.ExecuteCmdlet() 2021-02-08T18:29:01.3161391Z ##[debug] at Microsoft.WindowsAzure.Commands.Utilities.Common.AzurePSCmdlet.ProcessRecord() 2021-02-08T18:29:01.3391470Z ##[error]Unable to deserialize the response. 2021-02-08T18:29:01.3405939Z ##[debug]Processed: ##vso[task.logissue type=error]Unable to deserialize the response. 2021-02-08T18:29:01.4281449Z ##[debug]Exit code: 1 2021-02-08T18:29:01.4755596Z ##[debug]Leaving Invoke-VstsTool. 2021-02-08T18:29:01.4756553Z ##[error]PowerShell exited with code '1'. 2021-02-08T18:29:01.4757537Z ##[debug]Processed: ##vso[task.logissue type=error]PowerShell exited with code '1'. 2021-02-08T18:29:01.4790415Z ##[debug]Processed: ##vso[task.complete result=Failed]Error detected 2021-02-08T18:29:01.4808687Z ##[debug]Loading module from path 'D:\a\_tasks\AzurePowerShell_72a1931b-effb-4d2e-8fd8-f8472a07cb62\5.179.0\ps_modules\VstsAzureHelpers_\VstsAzureHelpers_.psm1'. 2021-02-08T18:29:01.5217853Z ##[debug]$OVERRIDING $global:DebugPreference from 'Continue' to 'SilentlyContinue'. 2021-02-08T18:29:01.5636401Z ##[debug]Loading resource strings from: D:\a\_tasks\AzurePowerShell_72a1931b-effb-4d2e-8fd8-f8472a07cb62\5.179.0\ps_modules\VstsAzureHelpers_/module.json 2021-02-08T18:29:01.5980430Z ##[debug]Loaded 13 strings. 2021-02-08T18:29:01.5981026Z ##[debug]SYSTEM_CULTURE: 'en-US' 2021-02-08T18:29:01.6022620Z ##[debug]Loading resource strings from: D:\a\_tasks\AzurePowerShell_72a1931b-effb-4d2e-8fd8-f8472a07cb62\5.179.0\ps_modules\VstsAzureHelpers_\Strings\resources.resjson\en-US\resources.resjson 2021-02-08T18:29:01.6311941Z ##[debug]Loaded 13 strings. ``` ## Error output ``` 2021-02-09T12:51:48.9094113Z WARNING: Upcoming breaking changes in the cmdlet 'Resolve-AzError' : 2021-02-09T12:51:48.9094619Z 2021-02-09T12:51:48.9095217Z The `Resolve-Error` alias will be removed in a future release. Please change any scripts that use this alias to use 2021-02-09T12:51:48.9096337Z `Resolve-AzError` instead. 2021-02-09T12:51:48.9096645Z 2021-02-09T12:51:48.9097272Z Note : Go to https://aka.ms/azps-changewarnings for steps to suppress this breaking change warning, and other 2021-02-09T12:51:48.9098046Z information on breaking changes in Azure PowerShell. 2021-02-09T12:51:48.9554248Z 2021-02-09T12:51:48.9564390Z 2021-02-09T12:51:48.9565441Z HistoryId: 1 2021-02-09T12:51:48.9565914Z 2021-02-09T12:51:48.9566594Z 2021-02-09T12:51:48.9616029Z Message : Unable to deserialize the response. 2021-02-09T12:51:48.9638723Z StackTrace : at Microsoft.Azure.Management.DataFactory.LinkedServicesOperations.d__5.MoveNext() 2021-02-09T12:51:48.9640302Z --- End of stack trace from previous location where exception was thrown --- 2021-02-09T12:51:48.9640927Z at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() 2021-02-09T12:51:48.9641569Z at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) 2021-02-09T12:51:48.9657480Z at Microsoft.Azure.Management.DataFactory.LinkedServicesOperationsExtensions.d_ 2021-02-09T12:51:48.9658150Z _1.MoveNext() 2021-02-09T12:51:48.9659595Z --- End of stack trace from previous location where exception was thrown --- 2021-02-09T12:51:48.9660244Z at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() 2021-02-09T12:51:48.9661522Z at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) 2021-02-09T12:51:48.9676475Z at Microsoft.Azure.Management.DataFactory.LinkedServicesOperationsExtensions.ListByFactory(ILinkedS 2021-02-09T12:51:48.9677481Z ervicesOperations operations, String resourceGroupName, String factoryName) 2021-02-09T12:51:48.9678008Z at 2021-02-09T12:51:48.9678488Z Microsoft.Azure.Commands.DataFactoryV2.DataFactoryClient.ListLinkedServices(AdfEntityFilterOptions 2021-02-09T12:51:48.9679654Z filterOptions) 2021-02-09T12:51:48.9698301Z at Microsoft.Azure.Commands.DataFactoryV2.DataFactoryClient.FilterPSLinkedServices(AdfEntityFilterO 2021-02-09T12:51:48.9698864Z ptions filterOptions) 2021-02-09T12:51:48.9699948Z at Microsoft.Azure.Commands.DataFactoryV2.GetAzureDataFactoryLinkedServiceCommand.ExecuteCmdlet() 2021-02-09T12:51:48.9700965Z at Microsoft.WindowsAzure.Commands.Utilities.Common.AzurePSCmdlet.ProcessRecord() 2021-02-09T12:51:48.9701494Z Exception : Microsoft.Rest.SerializationException 2021-02-09T12:51:48.9701922Z InvocationInfo : {Get-AzDataFactoryV2LinkedService} 2021-02-09T12:51:48.9702497Z Line : $adf.LinkedServices = Get-AzDataFactoryV2LinkedService -ResourceGroupName "$ResourceGroupName" 2021-02-09T12:51:48.9703807Z -DataFactoryName "$FactoryName" | ToArray 2021-02-09T12:51:48.9704194Z 2021-02-09T12:51:48.9725050Z Position : At C:\Users\VssAdministrator\Documents\WindowsPowerShell\Modules\azure.datafactory.tools\0.50.0\public 2021-02-09T12:51:48.9725683Z \Get-AdfFromService.ps1:44 char:27 2021-02-09T12:51:48.9726650Z + ... dServices = Get-AzDataFactoryV2LinkedService -ResourceGroupName "$Res ... 2021-02-09T12:51:48.9727611Z + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2021-02-09T12:51:48.9728404Z HistoryId : 1 2021-02-09T12:51:48.9728867Z 2021-02-09T12:51:48.9761844Z Message : Error reading JObject from JsonReader. Current JsonReader item is not an object: String. Path 2021-02-09T12:51:48.9762602Z 'typeProperties.password', line 1, position 2808. 2021-02-09T12:51:48.9763157Z StackTrace : at Newtonsoft.Json.Linq.JObject.Load(JsonReader reader, JsonLoadSettings settings) 2021-02-09T12:51:48.9763827Z at Microsoft.Rest.Serialization.PolymorphicDeserializeJsonConverter`1.ReadJson(JsonReader reader, 2021-02-09T12:51:48.9765487Z Type objectType, Object existingValue, JsonSerializer serializer) 2021-02-09T12:51:48.9766163Z at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.DeserializeConvertable(JsonConverter 2021-02-09T12:51:48.9767263Z converter, JsonReader reader, Type objectType, Object existingValue) 2021-02-09T12:51:48.9767915Z at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type 2021-02-09T12:51:48.9769016Z objectType, Boolean checkAdditionalContent) 2021-02-09T12:51:48.9769592Z at Newtonsoft.Json.Linq.JToken.ToObject(Type objectType, JsonSerializer jsonSerializer) 2021-02-09T12:51:48.9770967Z at Microsoft.Rest.Serialization.PolymorphicDeserializeJsonConverter`1.ReadJson(JsonReader reader, 2021-02-09T12:51:48.9771583Z Type objectType, Object existingValue, JsonSerializer serializer) 2021-02-09T12:51:48.9773008Z at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.DeserializeConvertable(JsonConverter 2021-02-09T12:51:48.9773594Z converter, JsonReader reader, Type objectType, Object existingValue) 2021-02-09T12:51:48.9774162Z at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty 2021-02-09T12:51:48.9774923Z property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty 2021-02-09T12:51:48.9775475Z containerProperty, JsonReader reader, Object target) 2021-02-09T12:51:48.9776073Z at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, 2021-02-09T12:51:48.9777108Z JsonReader reader, JsonObjectContract contract, JsonProperty member, String id) 2021-02-09T12:51:48.9777710Z at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type 2021-02-09T12:51:48.9779400Z objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, 2021-02-09T12:51:48.9780116Z JsonProperty containerMember, Object existingValue) 2021-02-09T12:51:48.9780660Z at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateList(IList list, JsonReader 2021-02-09T12:51:48.9781268Z reader, JsonArrayContract contract, JsonProperty containerProperty, String id) 2021-02-09T12:51:48.9781857Z at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateList(JsonReader reader, Type 2021-02-09T12:51:48.9782463Z objectType, JsonContract contract, JsonProperty member, Object existingValue, String id) 2021-02-09T12:51:48.9783049Z at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty 2021-02-09T12:51:48.9783660Z property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty 2021-02-09T12:51:48.9784214Z containerProperty, JsonReader reader, Object target) 2021-02-09T12:51:48.9784838Z at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, 2021-02-09T12:51:48.9786199Z JsonReader reader, JsonObjectContract contract, JsonProperty member, String id) 2021-02-09T12:51:48.9787306Z at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type 2021-02-09T12:51:48.9787994Z objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, 2021-02-09T12:51:48.9788564Z JsonProperty containerMember, Object existingValue) 2021-02-09T12:51:48.9789129Z at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type 2021-02-09T12:51:48.9790112Z objectType, Boolean checkAdditionalContent) 2021-02-09T12:51:48.9790627Z at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType) 2021-02-09T12:51:48.9791202Z at Microsoft.Rest.Serialization.SafeJsonConvert.DeserializeObject[T](String json, 2021-02-09T12:51:48.9791688Z JsonSerializerSettings settings) 2021-02-09T12:51:48.9794958Z at Microsoft.Azure.Management.DataFactory.LinkedServicesOperations.d__5.MoveNext() 2021-02-09T12:51:48.9796331Z Exception : Newtonsoft.Json.JsonReaderException 2021-02-09T12:51:48.9796757Z InvocationInfo : {Get-AzDataFactoryV2LinkedService} 2021-02-09T12:51:48.9797557Z Line : $adf.LinkedServices = Get-AzDataFactoryV2LinkedService -ResourceGroupName "$ResourceGroupName" 2021-02-09T12:51:48.9798064Z -DataFactoryName "$FactoryName" | ToArray 2021-02-09T12:51:48.9799159Z 2021-02-09T12:51:48.9815793Z Position : At C:\Users\VssAdministrator\Documents\WindowsPowerShell\Modules\azure.datafactory.tools\0.50.0\public 2021-02-09T12:51:48.9816368Z \Get-AdfFromService.ps1:44 char:27 2021-02-09T12:51:48.9817622Z + ... dServices = Get-AzDataFactoryV2LinkedService -ResourceGroupName "$Res ... 2021-02-09T12:51:48.9818480Z + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2021-02-09T12:51:48.9819192Z HistoryId : 1 2021-02-09T12:51:48.9819621Z 2021-02-09T12:51:48.9845016Z Message : Cannot find a variable with the name 'ADF_FOLDERS'. 2021-02-09T12:51:48.9845483Z StackTrace : 2021-02-09T12:51:48.9846654Z Exception : System.Management.Automation.ItemNotFoundException 2021-02-09T12:51:48.9847045Z InvocationInfo : {Get-Variable} 2021-02-09T12:51:48.9847478Z Line : if (!(Get-Variable ADF_FOLDERS -ErrorAction:SilentlyContinue)) { 2021-02-09T12:51:48.9847891Z 2021-02-09T12:51:48.9865388Z Position : At C:\Users\VssAdministrator\Documents\WindowsPowerShell\Modules\azure.datafactory.tools\0.50.0\privat 2021-02-09T12:51:48.9866003Z e\AdfObject.class.ps1:75 char:7 2021-02-09T12:51:48.9866994Z + if (!(Get-Variable ADF_FOLDERS -ErrorAction:SilentlyContinue)) { 2021-02-09T12:51:48.9867893Z + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2021-02-09T12:51:48.9868602Z HistoryId : 1 ```
Author: BeneHa
Assignees: -
Labels: `Data Factory`, `Service Attention`, `customer-reported`, `needs-triage`, `question`
Milestone: -
shawnxzq commented 3 years ago

@BeneHa In your linked service, the way to define "password" property in your linked service is not correct, it must be a credential and please don't use parameter for credentials. Suggest you either put raw credential as the value and the backend will encrypt it or you can put your credential into AKV and you can define it as a AKV credential, please refer to the document below for details, thanks! https://docs.microsoft.com/en-us/azure/data-factory/store-credentials-in-key-vault

shawnxzq commented 3 years ago

@BeneHa I am going to close the issue for now and feel free to let me know if you still have any question on this, thanks!

NowinskiK commented 3 years ago

@shawnxzq, I do not agree. In the example - a password is provided not directly but via parameter (dynamic content), which IS ABSOLUTELY POSSIBLE IN UX. image

Are you trying to say that the option should not exist there? Why parameter should not be used for the password??? Storing credentials in Azure Key Vault is another option, but it's not possible if you want to make LS generic (like in this example). This is clearly BUG - please reopen the issue.

BeneHa commented 3 years ago

@shawnxzq I agree with @NowinskiK obviously. If it is possible to configure ADF this way and everything is working (which it is) in the ADF, then if another tool cannot handle this it is a bug in the other tool, not a misconfiguration in the ADF itself.

We can work store everything in the Key Vault for now, but other customers will stumble upon the same issue sooner or later if it is not fixed.

NowinskiK commented 3 years ago

@shawnxzq hasn't had a chance to answer as closed threads are not monitoring by their system. I spoke to him and here is more clarification on this issue:

Using AKV credential and parameterize the AKV credential name is the recommended way to parameterize credentials in ADF, the guideline is not to parameterize the credential directly in ADF from security perspective. However, the UX supports it due to some historic reasons and we need to keep it there for compatibility reasons, the PS/SDK never supports it. So, I strongly suggest you using AKV if you really need credential parameterization. From our side, we will check whether we can add reminders in document or UX first.

jashwood commented 3 months ago

I just had a painful time recovering from this, including disconnecting adf from git in order to delete the offending linked services.