Evaluate stages & Artifacts in RouteStage

[shaiguitar]

It looks like there is incompatibility between the plugin expecting a full/newly defined embedded artifact and the pipeline’s needs to use an evaluate stage type artifact - ie user input for it. Full details:

• A pipeline definition needs to have user input in an Artifact to define the payload used in the RouteStage - so we need to use SPEL & evaluate stage ( similar approach to TFvar files defined in the evaluate stage docs ).

• To use the eval’ed files in later stages those are referenced by the ID of the eval’ed file. As such, the content/reference of the file is not passed in. ( The content from eval is saved in the internals of spinnaker, is fetched only by ID w/out reference content and other fields ).

• The plugin tries to resolvePipelineArtifact over here - and is expecting full attributes of a newly defined artifact and not the trimmed attributes of an eval'ed file - including fields that are not passed in when you use a eval file. As such, NULL pointer exception is raised and the pipeline/stage fails.

Reference attributes in both scenarios ( eval artifacts and newly defined embedded ones ):

• using a newly defined artifact includes:

  # the contents here above can't be spel, and we need the eval'ed content
  
  "payloadArtifact": {
  "account": "embedded-artifact",
  "artifact": {
    "artifactAccount": "embedded-artifact",
    "id": "68bdc44e-585e-4b74-825c-1321c2cca47f",
    "name": "foobar.json",
    "reference": "eyJwYXlsb2FkIjogMX0=",
    "type": "embedded/base64"
  },
  "id": null
  },

• Fields of using an already eval’ed artifact:

# using the evaled ID:

  "payloadArtifact": {
    "artifact": null,
    "id": "ab3a5b2d6755e4b9e769d44859d21a15"
  },

An example trace:

2021-06-01 15:08:34.328  WARN 1 --- [     handlers-6] c.n.s.o.e.DefaultExceptionHandler        : [XXX] Error occurred during task lambdaTrafficUpdateTask
java.lang.NullPointerException: null
    at com.amazon.aws.spinnaker.plugin.lambda.utils.LambdaCloudDriverUtils.resolvePipelineArtifact(LambdaCloudDriverUtils.java:397)
    at com.amazon.aws.spinnaker.plugin.lambda.utils.LambdaCloudDriverUtils.lambda$getPipelinesArtifactContent$3(LambdaCloudDriverUtils.java:416)
    at com.netflix.spinnaker.kork.core.RetrySupport.retry(RetrySupport.java:34)
    at com.netflix.spinnaker.kork.core.RetrySupport.retry(RetrySupport.java:27)
    at com.amazon.aws.spinnaker.plugin.lambda.utils.LambdaCloudDriverUtils.getPipelinesArtifactContent(LambdaCloudDriverUtils.java:413)
    at com.amazon.aws.spinnaker.plugin.lambda.traffic.BlueGreenDeploymentStrategy.verifyResults(BlueGreenDeploymentStrategy.java:99)
    at com.amazon.aws.spinnaker.plugin.lambda.traffic.BlueGreenDeploymentStrategy.deploy(BlueGreenDeploymentStrategy.java:54)
    at com.amazon.aws.spinnaker.plugin.lambda.traffic.BlueGreenDeploymentStrategy.deploy(BlueGreenDeploymentStrategy.java:38)
    at com.amazon.aws.spinnaker.plugin.lambda.traffic.LambdaTrafficUpdateTask.execute(LambdaTrafficUpdateTask.java:69)
    at com.netflix.spinnaker.orca.q.handler.RunTaskHandler$handle$1$1$1.invoke(RunTaskHandler.kt:143)
    at com.netflix.spinnaker.orca.q.handler.RunTaskHandler$handle$1$1$1.invoke(RunTaskHandler.kt:75)
    at com.netflix.spinnaker.orca.q.handler.RunTaskHandler.withLoggingContext(RunTaskHandler.kt:416)
    at com.netflix.spinnaker.orca.q.handler.RunTaskHandler.access$withLoggingContext(RunTaskHandler.kt:75)
    at com.netflix.spinnaker.orca.q.handler.RunTaskHandler$handle$1$1.invoke(RunTaskHandler.kt:105)
    at com.netflix.spinnaker.orca.q.handler.RunTaskHandler$handle$1$1.invoke(RunTaskHandler.kt:75)
    at com.netflix.spinnaker.orca.q.handler.AuthenticationAware$sam$java_util_concurrent_Callable$0.call(AuthenticationAware.kt)
    at com.netflix.spinnaker.security.AuthenticatedRequest.lambda$wrapCallableForPrincipal$0(AuthenticatedRequest.java:272)
    at com.netflix.spinnaker.orca.q.handler.AuthenticationAware$DefaultImpls.withAuth(AuthenticationAware.kt:53)
    at com.netflix.spinnaker.orca.q.handler.RunTaskHandler.withAuth(RunTaskHandler.kt:75)
    at com.netflix.spinnaker.orca.q.handler.RunTaskHandler$handle$1.invoke(RunTaskHandler.kt:104)
    at com.netflix.spinnaker.orca.q.handler.RunTaskHandler$handle$1.invoke(RunTaskHandler.kt:75)
    at com.netflix.spinnaker.orca.q.handler.RunTaskHandler$withTask$1.invoke(RunTaskHandler.kt:244)
    at com.netflix.spinnaker.orca.q.handler.RunTaskHandler$withTask$1.invoke(RunTaskHandler.kt:75)
    at com.netflix.spinnaker.orca.q.handler.OrcaMessageHandler$withTask$1.invoke(OrcaMessageHandler.kt:68)
    at com.netflix.spinnaker.orca.q.handler.OrcaMessageHandler$withTask$1.invoke(OrcaMessageHandler.kt:46)
    at com.netflix.spinnaker.orca.q.handler.OrcaMessageHandler$withStage$1.invoke(OrcaMessageHandler.kt:85)
    at com.netflix.spinnaker.orca.q.handler.OrcaMessageHandler$withStage$1.invoke(OrcaMessageHandler.kt:46)
    at com.netflix.spinnaker.orca.q.handler.OrcaMessageHandler$DefaultImpls.withExecution(OrcaMessageHandler.kt:95)
    at com.netflix.spinnaker.orca.q.handler.RunTaskHandler.withExecution(RunTaskHandler.kt:75)
    at com.netflix.spinnaker.orca.q.handler.OrcaMessageHandler$DefaultImpls.withStage(OrcaMessageHandler.kt:74)
    at com.netflix.spinnaker.orca.q.handler.RunTaskHandler.withStage(RunTaskHandler.kt:75)
    at com.netflix.spinnaker.orca.q.handler.OrcaMessageHandler$DefaultImpls.withTask(OrcaMessageHandler.kt:60)
    at com.netflix.spinnaker.orca.q.handler.RunTaskHandler.withTask(RunTaskHandler.kt:75)
    at com.netflix.spinnaker.orca.q.handler.RunTaskHandler.withTask(RunTaskHandler.kt:233)
    at com.netflix.spinnaker.orca.q.handler.RunTaskHandler.handle(RunTaskHandler.kt:101)
    at com.netflix.spinnaker.orca.q.handler.RunTaskHandler.handle(RunTaskHandler.kt:75)
    at com.netflix.spinnaker.q.MessageHandler$DefaultImpls.invoke(MessageHandler.kt:36)
    at com.netflix.spinnaker.orca.q.handler.OrcaMessageHandler$DefaultImpls.invoke(OrcaMessageHandler.kt)
    at com.netflix.spinnaker.orca.q.handler.RunTaskHandler.invoke(RunTaskHandler.kt:75)
    at com.netflix.spinnaker.orca.q.audit.ExecutionTrackingMessageHandlerPostProcessor$ExecutionTrackingMessageHandlerProxy.invoke(ExecutionTrackingMessageHandlerPostProcessor.kt:72)
    at com.netflix.spinnaker.q.QueueProcessor$callback$1$1.run(QueueProcessor.kt:90)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:834)
2021-06-01 15:08:34.328 ERROR 1 --- [     handlers-6] c.n.s.orca.q.handler.RunTaskHandler      : [XXX] Error running LambdaTrafficUpdateTask for pipeline[01F741ME5020VW2WDY7QWW3KCC]
java.lang.NullPointerException: null
    at com.amazon.aws.spinnaker.plugin.lambda.utils.LambdaCloudDriverUtils.resolvePipelineArtifact(LambdaCloudDriverUtils.java:397)
    at com.amazon.aws.spinnaker.plugin.lambda.utils.LambdaCloudDriverUtils.lambda$getPipelinesArtifactContent$3(LambdaCloudDriverUtils.java:416)
    at com.netflix.spinnaker.kork.core.RetrySupport.retry(RetrySupport.java:34)
    at com.netflix.spinnaker.kork.core.RetrySupport.retry(RetrySupport.java:27)
    at com.amazon.aws.spinnaker.plugin.lambda.utils.LambdaCloudDriverUtils.getPipelinesArtifactContent(LambdaCloudDriverUtils.java:413)
    at com.amazon.aws.spinnaker.plugin.lambda.traffic.BlueGreenDeploymentStrategy.verifyResults(BlueGreenDeploymentStrategy.java:99)
    at com.amazon.aws.spinnaker.plugin.lambda.traffic.BlueGreenDeploymentStrategy.deploy(BlueGreenDeploymentStrategy.java:54)
    at com.amazon.aws.spinnaker.plugin.lambda.traffic.BlueGreenDeploymentStrategy.deploy(BlueGreenDeploymentStrategy.java:38)
    at com.amazon.aws.spinnaker.plugin.lambda.traffic.LambdaTrafficUpdateTask.execute(LambdaTrafficUpdateTask.java:69)
    at com.netflix.spinnaker.orca.q.handler.RunTaskHandler$handle$1$1$1.invoke(RunTaskHandler.kt:143)
    at com.netflix.spinnaker.orca.q.handler.RunTaskHandler$handle$1$1$1.invoke(RunTaskHandler.kt:75)
    at com.netflix.spinnaker.orca.q.handler.RunTaskHandler.withLoggingContext(RunTaskHandler.kt:416)
    at com.netflix.spinnaker.orca.q.handler.RunTaskHandler.access$withLoggingContext(RunTaskHandler.kt:75)
    at com.netflix.spinnaker.orca.q.handler.RunTaskHandler$handle$1$1.invoke(RunTaskHandler.kt:105)
    at com.netflix.spinnaker.orca.q.handler.RunTaskHandler$handle$1$1.invoke(RunTaskHandler.kt:75)
    at com.netflix.spinnaker.orca.q.handler.AuthenticationAware$sam$java_util_concurrent_Callable$0.call(AuthenticationAware.kt)
    at com.netflix.spinnaker.security.AuthenticatedRequest.lambda$wrapCallableForPrincipal$0(AuthenticatedRequest.java:272)
    at com.netflix.spinnaker.orca.q.handler.AuthenticationAware$DefaultImpls.withAuth(AuthenticationAware.kt:53)
    at com.netflix.spinnaker.orca.q.handler.RunTaskHandler.withAuth(RunTaskHandler.kt:75)
    at com.netflix.spinnaker.orca.q.handler.RunTaskHandler$handle$1.invoke(RunTaskHandler.kt:104)
    at com.netflix.spinnaker.orca.q.handler.RunTaskHandler$handle$1.invoke(RunTaskHandler.kt:75)
    at com.netflix.spinnaker.orca.q.handler.RunTaskHandler$withTask$1.invoke(RunTaskHandler.kt:244)
    at com.netflix.spinnaker.orca.q.handler.RunTaskHandler$withTask$1.invoke(RunTaskHandler.kt:75)
    at com.netflix.spinnaker.orca.q.handler.OrcaMessageHandler$withTask$1.invoke(OrcaMessageHandler.kt:68)
    at com.netflix.spinnaker.orca.q.handler.OrcaMessageHandler$withTask$1.invoke(OrcaMessageHandler.kt:46)
    at com.netflix.spinnaker.orca.q.handler.OrcaMessageHandler$withStage$1.invoke(OrcaMessageHandler.kt:85)
    at com.netflix.spinnaker.orca.q.handler.OrcaMessageHandler$withStage$1.invoke(OrcaMessageHandler.kt:46)
    at com.netflix.spinnaker.orca.q.handler.OrcaMessageHandler$DefaultImpls.withExecution(OrcaMessageHandler.kt:95)
    at com.netflix.spinnaker.orca.q.handler.RunTaskHandler.withExecution(RunTaskHandler.kt:75)
    at com.netflix.spinnaker.orca.q.handler.OrcaMessageHandler$DefaultImpls.withStage(OrcaMessageHandler.kt:74)
    at com.netflix.spinnaker.orca.q.handler.RunTaskHandler.withStage(RunTaskHandler.kt:75)
    at com.netflix.spinnaker.orca.q.handler.OrcaMessageHandler$DefaultImpls.withTask(OrcaMessageHandler.kt:60)
    at com.netflix.spinnaker.orca.q.handler.RunTaskHandler.withTask(RunTaskHandler.kt:75)
    at com.netflix.spinnaker.orca.q.handler.RunTaskHandler.withTask(RunTaskHandler.kt:233)
    at com.netflix.spinnaker.orca.q.handler.RunTaskHandler.handle(RunTaskHandler.kt:101)
    at com.netflix.spinnaker.orca.q.handler.RunTaskHandler.handle(RunTaskHandler.kt:75)
    at com.netflix.spinnaker.q.MessageHandler$DefaultImpls.invoke(MessageHandler.kt:36)
    at com.netflix.spinnaker.orca.q.handler.OrcaMessageHandler$DefaultImpls.invoke(OrcaMessageHandler.kt)
    at com.netflix.spinnaker.orca.q.handler.RunTaskHandler.invoke(RunTaskHandler.kt:75)
    at com.netflix.spinnaker.orca.q.audit.ExecutionTrackingMessageHandlerPostProcessor$ExecutionTrackingMessageHandlerProxy.invoke(ExecutionTrackingMessageHandlerPostProcessor.kt:72)
    at com.netflix.spinnaker.q.QueueProcessor$callback$1$1.run(QueueProcessor.kt:90)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:834)

Let me know if I can provide any other needed information.

[mad-briller]

was any solution found to this? still encountering the same issue

thanks for your time