CodeBuild buildspec should support YAML FileAsset and project file name

[costleya]

A CodeBuild project currently calls JSON.stringify on the contents of a FileAsset to make CodeBuild recognize the buildspec as inline. However, CodeBuild uses YAML files exclusively in the documentation. Attempting to use JSON also does not work.

[eladb]

I think the issue not that you can't use JSON, because YAML is a superset of JSON, but because codebuild.Project currently assumes that you are providing the buildspec as an inline object (the typing is any, but technically it should be a strongly typed schema for the buildspec, see #1135).

I'll rename this issue to add support for assets, as it seems like a useful feature. We will probably end up with something like something like lambda.Code:

buildspec: codebuild.BuildSpec.asset('path/to/asset')
// or
buildspec: codebuild.BuildSpec.inline({
  <strong-typed-schema>
})
// or (this is also supported today by CFN):
buildspec: codebuild.BuildSpec.projectFile('my-buildspec.yaml')

[jesterhazy]

this would be a good feature

[eladb]

Please reopen if this is still desirable

[jostick]

I still think this is useful and couldn't find any solution to easily use an asset as CodeBuild buildspec. Did I overlook something? Or is it just not possible? How am I supposed to put buildspecs in a CDK library?

[skinny85]

@jostick you're right - we had that functionality before, but we removed it.

Can you describe your use case for this? I'll be re-opening the issue either way.

Thanks, Adam

[jostick]

We are using CDK to provide resources and build steps common in all our CodePipeline projects, e.g. cloning code from a version control system (no CodeCommit unfortunately). Cloning the repository is implemented using CodeBuild. It would be nice if it would be possible to include the buildspec for this step as an asset in the library. There are other shared build steps (like self-updating the pipeline) that would also be easier if the buildspec could be provide das FileAsset.

Our current workaround is parse the buildspec.yml and then inline it with BuildSpec.fromObject(YAML.parse(fs.readFileSync('buildspec.yaml', 'utf8'))). This currently works fine but is not nice and unnecessarily increases the template size (which is limited in Cloudformation).

[skinny85]

That's helpful, thanks @jostick !

[soverbosch]

This is what I need, currently trying to work with cdk and one of the problems I have now is using inline buildspecs.

[prasnitt]

True, I also need this feature.

[whereisaaron]

I have a buildspec YAML file that I just what to inline into the CDK .NET generated stacks at synth time. What I am looking for but not finding in BuildSpec is BuildSpec.FromPath(string) or BuildSpec.FromFile(File) or BuildSpec.FromStream(Stream) that will accept YAML text and inline it as YAML in the stack.

The closest I found is BuildSpec.FromObjectToYaml(IDictionary<String, Object>) which requires first converting your YAML file asset to a large collection of nested dictionaries, only to then convert it straight back to YAML text. That's even less workable than any and also a lot harder that not using CDK 😀

Is there a supporting .NET library people use that can parse/deserialize YAML files into this IDictionary<String, Object> structure? I looked but nothing sprung out that didn't involve a lot of messing around to get it into that form 😭

(We need to inline the buildspec because CodeBuild doesn't support the buildspec being in a secondary git source, so BuildSpec.FromSourceFilename(String) can't help us here. In our idiom the git webhook triggers the primary application code source, but all of the build support files come from a secondary git source.)

@markreyndev in https://github.com/aws/aws-cdk/issues/17178 had a similar request to have a better wait to create an inline BuildSpec than having to transform YAML to IDictionary<String, Object> but the discussion only address specifying a file in Git source, with doesn't allow my use case for loading a file from CDK project to inline in stack.

[skinny85]

@whereisaaron I believe what you want is to read the file with a YAML parser, get it in the form of a bunch of nested Dictionaries, and then use BuildSpec.FromObject().

[whereisaaron]

Thanks @skinny85, that would be a workaround, but I don't think that - ideally - CDK users should have to do all that? The canonical type of a CodeBuild buildspec is YAML. CDK should support YAML for both in-source file and inline.

The different language versions of CDK have different, loosely typed, internal representations whether any or IDictionary<String, Object> or Map.of(). None are strongly typed YAML. For .Net CDK users my opinion is that the interface should be either/both an actual YAML string/stream or a proper .Net YAML type such as defined by YamlDotNet. I've seen comments that strongly typed (buildspec-specific) YAML is the end goal for CDK but I am not sure if project is willing to include something like YamlDotNet to achieve that?

And since I haven't rocked up with a pull request, I'll do the workaround 😄

[skinny85]

For .Net CDK users my opinion is that the interface should be either/both an actual YAML string/stream or a proper .Net YAML type such as defined by YamlDotNet. I've seen comments that strongly typed (buildspec-specific) YAML is the end goal for CDK but I am not sure if project is willing to include something like YamlDotNet to achieve that?

Implementing something like that would require deep changes in JSII, the technology CDK uses for multi-language support - it's definitely not just a simple PR. So, I would suggest going with the workaround for now 😉.

[madeline-k]

See https://github.com/aws/aws-cdk/issues/17178 for pipelines use-case.

[github-actions[bot]]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.