Executable Does Not Return Error in Powershell

[mawarren642]

I'm using this program to automate synchronization of our clusters to source control. I am finding that deltakusto.exe does not actually return an error to the console when it encounters an error.

If it logs an error, the $LASTEXITCODE of powershell is not modified nor does it return any error codes. The only way to determine if it ran successfully or not seems to be to print its output to a file and scan it for error codes.

[vplauzon]

Interesting. The main method (in C#) returns 0 on success, 1 on failure.

That isn't reflected in PowerShell?

[mawarren642]

Thanks for checking this. Here is what I found, and I can reproduce it for you and link you to it in the pipeline tomorrow

If the argument provided for database is invalid, it returns a long list of exceptions to the console but doesn't actually produce an error or exit code.

[vplauzon]

Odd since even with an empty command line it gives me:

delta-kusto 0.8.2

ERROR(S): Required option 'p, parameter' is missing.

-v, --verbose Set output to verbose messages.

-p, --parameter Required. Set parameter file path.

-o, --override Parameter path overrides (list).

--help Display this help screen.

[...]\delta-kusto.exe (process 19740) exited with code 1.

[mawarren642]

Here is an instance I came across several times in my error testing. I'll share a link to the logs and also repost the logs below, in case you come up against missing ADO access:

1. Pipeline run of DK script to sync clusters (Kusto Synchronization, line 170): https://msazure.visualstudio.com/One/_releaseProgress?_a=release-environment-logs&releaseId=6300015&environmentId=22911188
2. This is the script that calls DK in our pipeline (Line 77) : https://msazure.visualstudio.com/One/_git/Compute-Insights-Gandalf?path=/Kusto/kusto-download.ps1&version=GBautomated/kusto-staging&line=77&lineEnd=78&lineStartColumn=1&lineEndColumn=1&lineStyle=plain&_a=contents
3. This is the yaml we reference, but it's all overriden so I wouldn't pay it much mind: https://msazure.visualstudio.com/One/_git/Compute-Insights-Gandalf?path=/Kusto/download-dev.yaml&version=GBautomated/kusto-staging&_a=contents

Code Snippets:

1. 2022-02-14T18:30:35.1597906Z 1 jobs
   2022-02-14T18:30:35.1644957Z Job 'download-dev':
   2022-02-14T18:30:35.1645533Z 
   2022-02-14T18:30:36.0651256Z Exception registered with Session ID '24791137-a7fc-483b-b392-efa0ab5139b6'
   2022-02-14T18:30:36.0669490Z Error:  Issue in running job 'download-dev'
   2022-02-14T18:30:36.0670426Z   Error:  Issue running Kusto script '.show database schema as csl script' on cluster 'https://balrogdevkusto.eastus.kusto.windows.net/' / database 'banana'
   2022-02-14T18:30:36.0672909Z     Exception encountered:  Kusto.Data.Exceptions.EntityNotFoundException ; Entity ID '[DB banana v?.?]' of kind 'Database' was not found.
   2022-02-14T18:30:36.0675595Z     Stack trace:     at Kusto.Cloud.Platform.Http.KustoHttpClient.ThrowKustoExceptionFromResponseMessageAsync(KustoExceptionContext exceptionContext, HttpResponseMessage responseMessage, ClientRequestProperties properties, Boolean shouldBuffer, Action`2 notify)
   2022-02-14T18:30:36.0677104Z    at Kusto.Data.Net.Client.RestClient2.MakeHttpRequestAsyncImpl(String address, String csl, String ns, String databaseName, Boolean streaming, ClientRequestProperties properties, ServiceModelTimeoutKind timeoutKind, String clientRequestId, Stream body, StreamProperties streamProperties)
   2022-02-14T18:30:36.0679402Z    at Kusto.Cloud.Platform.Utils.MonitoredActivity.InvokeAsync[TActivityType,TResult](TActivityType activityType, Func`1 func, String clientRequestId)
   2022-02-14T18:30:36.0684997Z    at Kusto.Cloud.Platform.Utils.MonitoredActivity.InvokeAsync[TActivityType,TResult](TActivityType activityType, Func`1 func, String clientRequestId)
   2022-02-14T18:30:36.0686510Z    at Kusto.Data.Net.Client.RestClient2.MakeHttpRequestAsync(ActivityType activityType, String baseAddress, String relativeAddress, String clientRequestIdPrefix, String ns, String databaseName, String csl, String addr, Boolean streaming, ClientRequestProperties properties, ServiceModelTimeoutKind timeoutKind, StreamProperties streamProperties)
   2022-02-14T18:30:36.0692856Z    at Kusto.Data.Net.Client.RestClient2.ExecuteControlCommandAsync(String databaseName, String command, String addr, ClientRequestProperties properties)
   2022-02-14T18:30:36.0693819Z    at DeltaKustoIntegration.Kusto.KustoManagementGateway.<>c__DisplayClass10_0.<<ExecuteCommandAsync>b__0>d.MoveNext()
   2022-02-14T18:30:36.0694418Z --- End of stack trace from previous location ---
   2022-02-14T18:30:36.0695522Z    at Polly.Retry.AsyncRetryEngine.ImplementationAsync[TResult](Func`3 action, Context context, CancellationToken cancellationToken, ExceptionPredicates shouldRetryExceptionPredicates, ResultPredicates`1 shouldRetryResultPredicates, Func`5 onRetryAsync, Int32 permittedRetryCount, IEnumerable`1 sleepDurationsEnumerable, Func`4 sleepDurationProvider, Boolean continueOnCapturedContext)
   2022-02-14T18:30:36.0696796Z    at Polly.AsyncPolicy.ExecuteAsync[TResult](Func`3 action, Context context, CancellationToken cancellationToken, Boolean continueOnCapturedContext)
   2022-02-14T18:30:36.0697823Z    at DeltaKustoIntegration.Kusto.KustoManagementGateway.ExecuteCommandAsync(String commandScript, CancellationToken ct)
   2022-02-14T18:30:36.0698703Z   Exception encountered:  DeltaKustoLib.DeltaException ; Issue running Kusto script '.show database schema as csl script' on cluster 'https://balrogdevkusto.eastus.kusto.windows.net/' / database 'banana'
   2022-02-14T18:30:36.0699614Z   Stack trace:     at DeltaKustoIntegration.Kusto.KustoManagementGateway.ExecuteCommandAsync(String commandScript, CancellationToken ct)
   2022-02-14T18:30:36.0700459Z    at DeltaKustoIntegration.Kusto.KustoManagementGateway.DeltaKustoIntegration.Kusto.IKustoManagementGateway.ReverseEngineerDatabaseAsync(CancellationToken ct)
   2022-02-14T18:30:36.0701328Z    at DeltaKustoIntegration.Database.KustoDatabaseProvider.DeltaKustoIntegration.Database.IDatabaseProvider.RetrieveDatabaseAsync(CancellationToken ct)
   2022-02-14T18:30:36.0702816Z    at delta_kusto.DeltaOrchestration.RetrieveDatabaseAsync(IDatabaseProvider currentDbProvider, String db)
   2022-02-14T18:30:36.0704888Z    at delta_kusto.DeltaOrchestration.ProcessJobAsync(MainParameterization parameters, IKustoManagementGatewayFactory kustoGatewayFactory, IFileGateway localFileGateway, String jobName, JobParameterization job)
   2022-02-14T18:30:36.0707795Z     Exception encountered:  Kusto.Data.Exceptions.EntityNotFoundException ; Entity ID '[DB banana v?.?]' of kind 'Database' was not found.
   2022-02-14T18:30:36.0713705Z     Stack trace:     at Kusto.Cloud.Platform.Http.KustoHttpClient.ThrowKustoExceptionFromResponseMessageAsync(KustoExceptionContext exceptionContext, HttpResponseMessage responseMessage, ClientRequestProperties properties, Boolean shouldBuffer, Action`2 notify)
   2022-02-14T18:30:36.0716228Z    at Kusto.Data.Net.Client.RestClient2.MakeHttpRequestAsyncImpl(String address, String csl, String ns, String databaseName, Boolean streaming, ClientRequestProperties properties, ServiceModelTimeoutKind timeoutKind, String clientRequestId, Stream body, StreamProperties streamProperties)
   2022-02-14T18:30:36.0717344Z    at Kusto.Cloud.Platform.Utils.MonitoredActivity.InvokeAsync[TActivityType,TResult](TActivityType activityType, Func`1 func, String clientRequestId)
   2022-02-14T18:30:36.0718188Z    at Kusto.Cloud.Platform.Utils.MonitoredActivity.InvokeAsync[TActivityType,TResult](TActivityType activityType, Func`1 func, String clientRequestId)
   2022-02-14T18:30:36.0719349Z    at Kusto.Data.Net.Client.RestClient2.MakeHttpRequestAsync(ActivityType activityType, String baseAddress, String relativeAddress, String clientRequestIdPrefix, String ns, String databaseName, String csl, String addr, Boolean streaming, ClientRequestProperties properties, ServiceModelTimeoutKind timeoutKind, StreamProperties streamProperties)
   2022-02-14T18:30:36.0720666Z    at Kusto.Data.Net.Client.RestClient2.ExecuteControlCommandAsync(String databaseName, String command, String addr, ClientRequestProperties properties)
   2022-02-14T18:30:36.0721454Z    at DeltaKustoIntegration.Kusto.KustoManagementGateway.<>c__DisplayClass10_0.<<ExecuteCommandAsync>b__0>d.MoveNext()
   2022-02-14T18:30:36.0722032Z --- End of stack trace from previous location ---
   2022-02-14T18:30:36.0723216Z    at Polly.Retry.AsyncRetryEngine.ImplementationAsync[TResult](Func`3 action, Context context, CancellationToken cancellationToken, ExceptionPredicates shouldRetryExceptionPredicates, ResultPredicates`1 shouldRetryResultPredicates, Func`5 onRetryAsync, Int32 permittedRetryCount, IEnumerable`1 sleepDurationsEnumerable, Func`4 sleepDurationProvider, Boolean continueOnCapturedContext)
   2022-02-14T18:30:36.0724438Z    at Polly.AsyncPolicy.ExecuteAsync[TResult](Func`3 action, Context context, CancellationToken cancellationToken, Boolean continueOnCapturedContext)
   2022-02-14T18:30:36.0725229Z    at DeltaKustoIntegration.Kusto.KustoManagementGateway.ExecuteCommandAsync(String commandScript, CancellationToken ct)
   2022-02-14T18:30:36.0857889Z ### Adding file to git: [download-balrogdevkusto-demo.kql]
   2022-02-14T18:30:36.1210964Z $### Checking last exit code: [0]
   2022-02-14T18:30:36.1213942Z $### Current error count: 0
   2022-02-14T18:30:36.1290789Z ### Finished with cluster download attempt...

2. (I promise our database is not a banana)

   try {
           $LASTEXITCODE = 0
           Write-Output $"### Current error count: $($Error.Count)"
           $name = $cluster.Name
           $fileName = "download-$name-$database.kql"
           $URI = $cluster.URI
           Write-Output "### Working $database in $name at [$fileName]"
           .\delta-kusto.exe -p download-dev.yaml -o tokenProvider.login.tenantId=$tenantId tokenProvider.login.clientId=$clientId tokenProvider.login.secret=$secret jobs.download-dev.target.adx.clusterUri=$URI jobs.download-dev.target.adx.database="banana" jobs.download-dev.action.filePath="$fileName"
           # Try to add resultant files to git, in case they are untracked
           Write-Output "### Adding file to git: [$fileName]"
           git add $fileName
           Write-Output $"### Checking last exit code: [$LASTEXITCODE]"
           Write-Output $"### Current error count: $($Error.Count)"
           if ($LASTEXITCODE -ne 0) {
               Write-Output "### Failed to download $cluster : $name"
               $successfulKusto = 1
               $LASTEXITCODE = 0
           }
       }
       catch {
           Write-Output "### Failed to download $cluster : $name"
           $successfulKusto = 1
       }
       finally {
           Write-Output "### Finished with cluster download attempt..."
       }

3. jobs:
   download-dev:
   target:
       adx:
           clusterUri:  to-be-overriden
           database:  to-be-overriden
   action:
       filePath:  to-be-overriden
   tokenProvider:
   login:
       tenantId:  to-be-overriden
       clientId:  to-be-overriden
       secret:  to-be-overriden