Error: Exactly one of `[container]` or `[containers]` must be provided

pawelprazak commented 6 months ago

What happened?

The same Pulumi program that succeeds in TS fails in Java:

Diagnostics:
  pulumi:pulumi:Stack (karol-issue-2-java-dev):
    Error: Exactly one of [container] or [containers] must be provided: Error: Exactly one of [container] or [containers] must be provided
        at normalizeTaskDefinitionContainers (/snapshot/awsx/bin/ecs/containers.js:37:15)
        at new FargateTaskDefinition (/snapshot/awsx/bin/ecs/fargateTaskDefinition.js:34:79)
        at new FargateService (/snapshot/awsx/bin/ecs/fargateService.js:40:30)
        at awsx:ecs:FargateService (/snapshot/awsx/bin/resources.js:25:45)
        at construct (/snapshot/awsx/bin/resources.js:43:12)
        at Provider.construct (/snapshot/awsx/bin/index.js:39:52)
        at Server.<anonymous> (/snapshot/awsx/node_modules/@pulumi/pulumi/provider/server.js:315:52)
        at Generator.next (<anonymous>)
        at fulfilled (/snapshot/awsx/node_modules/@pulumi/pulumi/provider/server.js:18:58)
        at processTicksAndRejections (node:internal/process/task_queues:96:5)

I believe this might be because TS (and all other SDKs use the new serializer path, the keepOutputValues (specialOutputValueSig) and Java uses the old output serializer.

We can also see this is Besom Scala SDK, that uses the old serializer.

Example

The failing Java program:

package myproject;

import com.pulumi.Context;
import com.pulumi.Pulumi;
import com.pulumi.aws.ecs.Cluster;
import com.pulumi.aws.lb.LoadBalancer;
import com.pulumi.awsx.ecs.FargateService;
import com.pulumi.awsx.ecs.FargateServiceArgs;
import com.pulumi.awsx.ecs.inputs.FargateServiceTaskDefinitionArgs;
import com.pulumi.awsx.ecs.inputs.TaskDefinitionContainerDefinitionArgs;
import com.pulumi.awsx.ecs.inputs.TaskDefinitionPortMappingArgs;
import com.pulumi.awsx.lb.ApplicationLoadBalancer;
import com.pulumi.core.Output;

public class App {
    public static void main(String[] args) {
        Pulumi.run(App::stack);
    }

    private static void stack(Context ctx) {
        var cluster = new Cluster("cluster");

        var loadBalancer = new ApplicationLoadBalancer("lb");

        var service = new FargateService("app-service", FargateServiceArgs.builder()
                .cluster(cluster.arn())
                .assignPublicIp(true)
                .taskDefinitionArgs(FargateServiceTaskDefinitionArgs.builder()
                        .container(TaskDefinitionContainerDefinitionArgs.builder()
                                .name("test")
                                .image("nginx:latest")
                                .cpu(256)
                                .memory(512)
                                .portMappings(
                                        TaskDefinitionPortMappingArgs.builder()
                                                .containerPort(80)
                                                .targetGroup(loadBalancer.defaultTargetGroup())
                                                .build()
                                )
                                .build()
                        ).build()
                )
                .desiredCount(1)
                .build()
        );

        ctx.export("frontendURL",
                Output.format("http://%s", loadBalancer.loadBalancer().apply(LoadBalancer::dnsName))
        );
        ctx.export("url", service.urn());
    }
}

And the working TypeScript:

import * as aws from "@pulumi/aws";
import * as awsx from "@pulumi/awsx";
import * as pulumi from "@pulumi/pulumi";

// Create an ECS cluster
const cluster = new aws.ecs.Cluster("my-cluster", {});

// Create a load balancer to listen for requests and route them to the container.
const loadBalancer = new awsx.lb.ApplicationLoadBalancer("loadbalancer", {});

// Create a Fargate service running on the cluster, referencing the task definition
const service = new awsx.ecs.FargateService("app-service", {
  cluster: cluster.arn,
  assignPublicIp: true,
  taskDefinitionArgs: {
      container: {
        name: "test",
        image: "nginx:latest",
        cpu: 256,
        memory: 512,
        portMappings: [{
            containerPort: 80,
            targetGroup: loadBalancer.defaultTargetGroup,
        }],
      }
    },
  desiredCount: 1, // Number of tasks to run
});

// Export the URL of the service
export const frontendURL = pulumi.interpolate `http://${loadBalancer.loadBalancer.dnsName}`;
export const url = service.urn;

Output of `pulumi about`

TypeScript

pulumi about
CLI
Version      3.107.0
Go Version   go1.22.0
Go Compiler  gc

Plugins
NAME    VERSION
aws     6.24.0
awsx    2.5.0
docker  4.5.1
docker  3.6.1
nodejs  unknown

Host
OS       darwin
Version  14.2.1
Arch     arm64

This project is written in nodejs: executable='/opt/homebrew/bin/node' version='v21.6.2'

Backend
Name           pulumi.com
URL            https://app.pulumi.com/pprazak
User           pprazak
Organizations  pprazak, besom
Token type     personal

Dependencies:
NAME            VERSION
typescript      4.9.5
@pulumi/aws     6.24.0
@pulumi/awsx    2.5.0
@pulumi/docker  4.5.1
@pulumi/pulumi  3.107.0
@types/node     18.19.19

Pulumi locates its logs in /var/folders/n6/bg1skjsj3k7_1tjvn7n65zl40000gn/T/ by default
warning: Failed to get information about the current stack: No current stack

Java

pulumi about
CLI
Version      3.107.0
Go Version   go1.22.0
Go Compiler  gc

Plugins
NAME    VERSION
aws     6.24.0
awsx    2.5.0
docker  4.5.0
java    unknown

Host
OS       darwin
Version  14.2.1
Arch     arm64

This project is written in java: executable='/opt/homebrew/opt/sdkman-cli/libexec/candidates/java/current/bin/java' version='openjdk 21.0.1 2023-10-17
OpenJDK Runtime Environment GraalVM CE 21.0.1+12.1 (build 21.0.1+12-jvmci-23.1-b19)
OpenJDK 64-Bit Server VM GraalVM CE 21.0.1+12.1 (build 21.0.1+12-jvmci-23.1-b19, mixed mode, sharing)' java='/opt/homebrew/opt/sdkman-cli/libexec/candidates/java/current/bin/java' javac='21.0.1' maven='Apache Maven 3.9.6 (bc0240f3c744dd6b6ec2920b3cd08dcc295161ae)' gradle='8.6'

Current Stack: pprazak/karol-issue-2-java/dev

Found no resources associated with dev

Found no pending operations associated with dev

Backend
Name           pulumi.com
URL            https://app.pulumi.com/pprazak
User           pprazak
Organizations  pprazak, besom
Token type     personal

No dependencies found

Pulumi locates its logs in /var/folders/n6/bg1skjsj3k7_1tjvn7n65zl40000gn/T/ by default

Additional context

None.

Contributing

Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

mjeffryes commented 6 months ago

Thanks for reporting this @pawelprazak. I'm going to transfer this to pulumi-java as I think the fix probably lies there.

pawelprazak commented 5 months ago

I believe this is the same issue: https://github.com/pulumi/pulumi-awsx/issues/820

pawelprazak commented 5 months ago

What is interesting, is that doing this:

var tg = loadBalancer.defaultTargetGroup()
                .apply(
                        t -> t.name().applyValue(
                                n -> TargetGroup.get(n, t.id(), null, null)
                        )
                );

Results in the following error:

  aws:lb:TargetGroup (lb-515c2c0):
    error: Preview failed: diffing urn:pulumi:dev::karol-issue-2-java::aws:lb/targetGroup:TargetGroup::lb-515c2c0: Invalid Attribute Combination

    Attribute "port" must be specified when "target_type" is "instance".
    target_type

Not sure if this is related but I've noticed this while trying to find a workaround, and if it's related it could help pinpoint the exact cause.

pawelprazak commented 5 months ago

found another problem while working on a workaround, this is a promising workaround:

var cluster = new Cluster("cluster");

        var loadBalancer = new ApplicationLoadBalancer("lb");
        var taskDef = new FargateTaskDefinition("app-task", FargateTaskDefinitionArgs.builder()
                .containers(Map.of(
                        "test", TaskDefinitionContainerDefinitionArgs.builder()
                                .name("test")
                                .image("nginx:latest")
                                .cpu(256)
                                .memory(512)
                                .portMappings(
                                        Output.of(loadBalancer.defaultTargetGroup().apply(t -> Output.of(List.of(
                                                TaskDefinitionPortMappingArgs.builder()
                                                        .containerPort(80)
                                                        .targetGroup(t)
                                                        .build()
                                        )))).apply(o -> o)
                                )
                                .build()
                ))
                .build()
        );
        var service = new FargateService("app-service", FargateServiceArgs.builder()
                .cluster(cluster.arn())
                .assignPublicIp(true)
                .taskDefinition(taskDef.taskDefinition().apply(TaskDefinition::arn))
                .desiredCount(1)
                .build(),
                ComponentResourceOptions.builder()
                        .customTimeouts(CustomTimeouts.builder()
                                .create(Duration.ofMinutes(60))
                                .build()
                        )
                        .build()
        );

but results in:

    error: Running program [PID: 71677](/opt/homebrew/opt/sdkman-cli/libexec/candidates/java/current/bin/java -classpath /Users/pprazak/repos/karol-issue-2/karol-issue-2-java/.mvn/wrapper/maven-wrapper.jar -Dmaven.home=/Users/pprazak/repos/karol-issue-2 -Dmaven.multiModuleProjectDirectory=/Users/pprazak/repos/karol-issue-2/karol-issue-2-java org.apache.maven.wrapper.MavenWrapperMain -Dorg.slf4j.simpleLogger.defaultLogLevel=warn --no-transfer-progress compile exec:java) failed with an unhandled exception:
    java.lang.UnsupportedOperationException: Convert [com.pulumi.aws.lb.TargetGroup.targetHealthStates]: Error converting 'java.util.Collections$UnmodifiableRandomAccessList' to 'TypeShape{type=interface java.util.List, parameters=[TypeShape{type=class com.pulumi.aws.lb.outputs.TargetGroupTargetHealthState, parameters=[]}]}'. null
        at com.pulumi.serialization.internal.Converter.convertObjectUntyped(Converter.java:119)
        at com.pulumi.serialization.internal.Converter.convertValue(Converter.java:86)
        at com.pulumi.core.internal.OutputCompletionSource.setValue(OutputCompletionSource.java:95)
        at com.pulumi.deployment.internal.DeploymentImpl$ReadOrRegisterResourceInternal.lambda$completeResourceAsync$0(DeploymentImpl.java:1187)
        at java.base/java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:646)
        at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483)
        at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387)
        at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312)
        at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843)
        at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808)
        at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188)
    Caused by: java.lang.NullPointerException
        at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:903)
        at com.google.common.collect.ImmutableList$Builder.add(ImmutableList.java:815)
        at com.pulumi.serialization.internal.Converter.tryConvertList(Converter.java:642)
        at com.pulumi.serialization.internal.Converter.tryConvertObjectInner(Converter.java:277)
        at com.pulumi.serialization.internal.Converter.convertObjectUntyped(Converter.java:115)
        ... 10 more

But it looks lika separate issue in the serde implementation on the Java SDK side.

pulumi / pulumi-java