search

aws / aws-cdk

The AWS Cloud Development Kit is a framework for defining cloud infrastructure in code
https://aws.amazon.com/cdk
Apache License 2.0
11.52k stars 3.86k forks source link

CodeBuild PipelineProject LinuxBuildImage.STANDARD_6_0 somehow got mixed up with aws-cdk-lib.aws_codebuild.LinuxBuildImage.AMAZON_LINUX_2_ARM #29528

Open icelava opened 6 months ago

icelava commented 6 months ago

Describe the bug

We have a .NET CDK project that handles deployment of CodePipleline/CodeBuild and has been working based on Amazon.CDK.Lib 2.94.0.

For our React application pipelines, the CodeBuild setup is like

this.Build = new PipelineProject(this, codeBuildName, new PipelineProjectProps
{
    Environment = new BuildEnvironment
    {
        BuildImage = LinuxBuildImage.STANDARD_6_0,
        ComputeType = ComputeType.MEDIUM
    },
    Description = $"{_props.ProductName} React CI CD {_props.Environment}",
    ProjectName = codeBuildName,
    Timeout = Duration.Minutes(10),
    EnvironmentVariables = envVars,
    Badge = false,
    Cache = Cache.Local(LocalCacheMode.CUSTOM),
    Role = this._props.CodebuildRole
});

Upon updating to CDK 2.130.0, attempting cdk list would result in strange error.

PS C:\projects\dev-tools-cdk> cdk list Warning NETSDK1174: The abbreviation of -p for --project is deprecated. Please use --project.

Unhandled exception. System.TypeInitializationException: The type initializer for 'Amazon.CDK.AWS.CodeBuild.LinuxBuildImage' threw an exception. ---> System.Exception: Of static property aws-cdk-lib.aws_codebuild.LinuxBuildImage.AMAZON_LINUX_2_ARM: Unable to serialize value as aws-cdk-lib.aws_codebuild.IBuildImage ├── ?? Failing value is undefined ?── ?? Failure reason(s): ?─ A value is required (type is non-optional) at Amazon.JSII.Runtime.Services.Client.TryDeserialize[TResponse](String responseJson) at Amazon.JSII.Runtime.Services.Client.ReceiveResponse[TResponse]() at Amazon.JSII.Runtime.Services.Client.Send[TRequest,TResponse](TRequest requestObject) at Amazon.JSII.Runtime.Services.Client.StaticGet(StaticGetRequest request) at Amazon.JSII.Runtime.Services.Client.StaticGet(String fullyQualifiedName, String property) at Amazon.JSII.Runtime.Deputy.DeputyBase.<>c__DisplayClass8_01.<GetStaticProperty>b__0(IClient client) at Amazon.JSII.Runtime.Deputy.DeputyBase.GetPropertyCore[T](JsiiPropertyAttribute propertyAttribute, Func2 getFunc) at Amazon.JSII.Runtime.Deputy.DeputyBase.GetStaticProperty[T](Type type, String propertyName) at Amazon.CDK.AWS.CodeBuild.LinuxBuildImage..cctor() --- End of inner exception stack trace --- at Amazon.CDK.AWS.CodeBuild.LinuxBuildImage.get_STANDARD_6_0() at DevToolsCdk.Constructs.ReactPipeline.Initialize() in C:\projects\dev-tools-cdk\src\DevToolsCdk\Constructs\ReactPipeline.cs:line 85 at DevToolsCdk.Constructs.ReactPipeline..ctor(Construct scope, String id, ArtifactStore artifactStore, ReactPipelineProps props) in C:\projects\dev-tools-cdk\src\DevToolsCdk\Constructs\ReactPipeline.cs:line 48 at DevToolsCdk.Stacks.EvoBuildStack.EstablishReactFrontEnd(ReactFrontEndConfig frontEndConfig) in C:\projects\dev-tools-cdk\src\DevToolsCdk\Stacks\EvoBuildStack.cs:line 229 at DevToolsCdk.Stacks.EvoBuildStack..ctor(Construct scope, String id, EvoBuildConfig config, DevToolsStack sharedStack, DevToolsConfig sharedConfig, IStackProps props) in C:\projects\dev-tools-cdk\src\DevToolsCdk\Stacks\EvoBuildStack.cs:line 72 at DevToolsCdk.Program.Main(String[] args) in C:\projects\dev-tools-cdk\src\DevToolsCdk\Program.cs:line 56

Why is there involvement with LinuxBuildImage.AMAZON_LINUX_2_ARM? Our definition is for STANDARD_6_0 (Ubuntu).

Expected Behavior

CodeBuild continue to use LinuxBuildImage.STANDARD_6_0 like previously.

Current Behavior

LinuxBuildImage.STANDARD_6_0 somehow ends up as LinuxBuildImage.AMAZON_LINUX_2_ARM and not properly defined under the covers?

Reproduction Steps

  1. Update from Amazon.CDK.Lib 2.94.0 to 2.130.0.
  2. Build existing CDK project as per above code expectations.
  3. run cdk list

Possible Solution

No response

Additional Information/Context

No response

CDK CLI Version

2.130.0

Framework Version

.NET 6.0

Node.js Version

18.9.0

OS

Windows 11

Language

.NET

Language Version

C# 10

Other information

No response

icelava commented 6 months ago

Even if other build images are used, STANDARD_7_0 or AMAZON_LINUX_2_5 the same error still appears.

icelava commented 6 months ago

We have a separate library project that does something similar but this one works correctly.

var buildProject = new PipelineProject(this, this.codeBuildName, new PipelineProjectProps
{
    Environment = new BuildEnvironment
    {
        BuildImage = LinuxBuildImage.STANDARD_7_0,
        ComputeType = ComputeType.MEDIUM
    },
    Description = $"{this._props.ProductName} {this._props.PackageName} publisher {this._props.Environment}",
    ProjectName = this.codeBuildName,
    Timeout = Duration.Minutes(10),
    Badge = false
});

Can't tell what is the difference that causes proper behaviour.

icelava commented 6 months ago

Update to CDK 2.133.0 did not solve the problem. Probably because the same underlying Amazon.JSII.Runtime 1.94.0 is used.

UPDATE

Force Amazon.JSII.Runtime update to 1.95.0 at top propject level still yields same result.

icelava commented 6 months ago

Following the inner libraries' stack the lowest I can get to is Amazon.JSII.Runtime.Services.ServiceContainer

namespace Amazon.JSII.Runtime.Services;

public static class ServiceContainer
{
    private static readonly Lazy<IServiceProvider> _serviceProvider = new Lazy<IServiceProvider>(() => BuildServiceProvider(), LazyThreadSafetyMode.ExecutionAndPublication);

    public static IServiceProvider? ServiceProviderOverride { get; set; }

    internal static IServiceProvider ServiceProvider => ServiceProviderOverride ?? _serviceProvider.Value;

    public static ServiceProvider BuildServiceProvider(ILoggerFactory? loggerFactoryOverride = null)
    {
        IServiceCollection services = new ServiceCollection();
        if (loggerFactoryOverride == null)
        {
            services.AddLogging(delegate (ILoggingBuilder builder)
            {
                builder.AddConsole();
            });
        }
        else
        {
            services.AddSingleton(loggerFactoryOverride);
        }

        services.AddSingleton<IFileSystem, FileSystem>();
        services.AddSingleton<IJsiiRuntimeProvider, JsiiRuntimeProvider>();
        services.AddSingleton<ILoadedPackageSet, LoadedPackageSet>();
        services.AddSingleton<ITypeCache, TypeCache>();
        services.AddSingleton<IJsiiToFrameworkConverter, JsiiToFrameworkConverter>();
        services.AddSingleton<IFrameworkToJsiiConverter, FrameworkToJsiiConverter>();
        services.AddSingleton<IReferenceMap, ReferenceMap>();
        services.AddSingleton<INodeProcess, NodeProcess>();
        services.AddSingleton<IRuntime, Runtime>();
        services.AddSingleton<IResourceExtractor, ResourceExtractor>();
        services.AddTransient<IClient, Client>();
        ServiceProvider serviceProvider = services.BuildServiceProvider();
        serviceProvider.GetRequiredService<IClient>().Hello();
        return serviceProvider;
    }

Client relies on Runtime to perform requests and responses for the enum property values but Runtime cannot be decompiled/navigated to thus cannot see how its performing these value references.

icelava commented 6 months ago

Value caching in play?

More details about our stack, and where the "culprit" is.

Our system consists for five separate applications and the constructor of the stack defines a CodePipeline for each individual app.

this.EstablishContainerPipeline1();
this.EstablishContainerPipeline2();
this.EstablishContainerPipeline3();
this.EstablishReactPipeline1();
this.EstablishReactPipeline2();

The first three use a different pipeline construct to build and push container images to ECR. CDK has no problem synthesizing those. It's only on the fourth (React) pipeline where the error happens, wihch made us wonder what's wrong with the React pipeline construct.

However on later testing by commenting out the container pipelines did CDK finally synthesize the React pipelines as expected. Reversing the order of pipelines (React before container) also works.

The former pipeline construct's CodeBuild makes use of LinuxArmBuildImage.AMAZON_LINUX_2_STANDARD_3_0

this.buildProject = new PipelineProject(this, codeBuildName, new PipelineProjectProps
{
    Environment = new BuildEnvironment
    {
        BuildImage = LinuxArmBuildImage.AMAZON_LINUX_2_STANDARD_3_0,
        ComputeType = ComputeType.LARGE,
        Privileged = true
    },
    Description = $"{this._props.ProductName} {this._props.AppShorthand} CI CD {_props.Environment}",
    ProjectName = codeBuildName,
    BuildSpec = BuildSpec.FromSourceFilename(this._props.Buildspec),
    Timeout = Duration.Minutes(20),
    EnvironmentVariables = envVars,
    Badge = false,
    Cache = Cache.Local(LocalCacheMode.CUSTOM, LocalCacheMode.DOCKER_LAYER),
    Role = this._props.CodebuildRole
});

Those somehow have a lingering effect on the latter fourth (React) pipeline to reference LinuxBuildImage.AMAZON_LINUX_2_ARM instead?

I can't track any relation between LinuxArmBuildImage and LinuxBuildImage.

pahud commented 6 months ago

Hi

I am not .NET expert but this works for me in cdk v2.126.0

using Amazon.CDK;
using Amazon.CDK.AWS.CodeBuild;
using Constructs;

namespace Dotnet
{
    public class DotnetStack : Stack
    {
        internal DotnetStack(Construct scope, string id, IStackProps props = null) : base(scope, id, props)
        {
            // The code that defines your stack goes here
            new PipelineProject(this, "foo", new PipelineProjectProps
            {
                Environment = new BuildEnvironment
                {
                    BuildImage = LinuxBuildImage.STANDARD_6_0,
                    ComputeType = ComputeType.MEDIUM
                },
                ProjectName = "foo",
                Timeout = Duration.Minutes(10),
                Badge = false,
                Cache = Cache.Local(LocalCacheMode.CUSTOM),
            });
        }
    }
}

and cdk diff shows

Resources
[+] AWS::IAM::Role foo/Role fooRole65AC3ADB 
[+] AWS::IAM::Policy foo/Role/DefaultPolicy fooRoleDefaultPolicy357EC2C3 
[+] AWS::CodeBuild::Project foo foo6445C170

LinuxArmBuildImage is for ARM-based images

https://github.com/aws/aws-cdk/blob/0fee99b473c17210a156d74263227cdf3274d253/packages/aws-cdk-lib/aws-codebuild/lib/project.ts#L1791-L1799

LinuxArmBuildImage.AMAZON_LINUX_2_STANDARD_3_0 is defined here.

LinuxBuildImage.STANDARD_7_0 is defined here for AMD64 standard images.

Update from Amazon.CDK.Lib 2.94.0 to 2.130.0.

How did you upgrade from 2.94.0 to 2.130.0 ?

icelava commented 6 months ago

We have our own L3 library assembly which depended on 2.130.0, now 2.133.0.

The pipeline constructs in play however are not part of that library but with the top-level CDK project. Synth works when the CodePipeline/Codebuild depending on LinuxBuildImage.STANDARD_6_0 are instantiated first before those depending LinuxArmBuildImage.AMAZON_LINUX_2_STANDARD_3_0.

WIth the original reverse order, that's when the strange mis-reference happens. Something that I cannot see with the Runtime class in use (from Client.StaticGet()).

icelava commented 5 months ago

So who exactly is in charge of the .NET packages? (Amazon.JSII.Runtime)

AleksandrsJakovlevsVisma commented 2 months ago

I am having the same issue. It's annoying that I had to refactor code because of this silly little bug