Missing support for submodules?

[marko-team-ai]

Hi there,

Our project has a submodule and we need to have a way to fetch submodule content before the build starts.

Is there a way to configure this plugin to execute the command: git submodule update --init --recursive during DOWNLOAD_SOURCE action?

or we need to that with Git plugin?

Thanks.

[leoherran-aws]

This plugin actually doesn't do any source downloading. You can configure it two ways:

1. Use Project source option: just start a build for your CodeBuild project. CodeBuild will download the source configured in your CodeBuild project, which can contain contain submodules.
2. Use Jenkins source option: this will zip up anything in a directory in your Jenkins workspace, upload it to S3, and start a CodeBuild build with that uploaded zip as the source. In this case you will want to use one or more plugins that download your source and its submodules into your workspace before the CodeBuild step.

[marko-team-ai]

Hey, @leoherran-aws it seems that I am missing something obvious.

This is our Project source setup, which doesn't pull submodules, as when we check the folder where submodule content should be, it's empty. Did we set up something wrong?

this is the error message: Child module /codebuild/output/src753569504/src/github.com/markosproject/web/pom.xml of /codebuild/output/src753569504/src/github.com/markosproject/pom.xml does not exist @ so /web is empty (it's a submodule folder)

We are on the version 0.45 and the latest one is 0.49 is that a problem?

[marko-team-ai]

This plugin actually doesn't do any source downloading. You can configure it two ways:

1. Use Project source option: just start a build for your CodeBuild project. CodeBuild will download the source configured in your CodeBuild project, which can contain contain submodules.

Not sure if I am following this. It's not downloading the source, but it will download a configured source?

[marko-team-ai]

2. Use Jenkins source option: this will zip up anything in a directory in your Jenkins workspace, upload it to S3, and start a CodeBuild build with that uploaded zip as the source. In this case, you will want to use one or more plugins that download your source and its submodules into your workspace before the CodeBuild step.

Also, I have tried this method. I used Git to pull everything and the submodule was also pulled fine. But I experienced this error:

[AWS CodeBuild Plugin] Please select a project with S3 source type

Not sure where I need to set S3 source type? @leoherran-aws

[leoherran-aws]

This plugin actually doesn't do any source downloading. You can configure it two ways:

1. Use Project source option: just start a build for your CodeBuild project. CodeBuild will download the source configured in your CodeBuild project, which can contain contain submodules.

Not sure if I am following this. It's not downloading the source, but it will download a configured source?

With this option configured the plugin will just start a build. CodeBuild (the AWS service, not the plugin) will download the source that's configured in your CodeBuild project inside your build container.

[leoherran-aws]

1. Use Jenkins source option: this will zip up anything in a directory in your Jenkins workspace, upload it to S3, and start a CodeBuild build with that uploaded zip as the source. In this case, you will want to use one or more plugins that download your source and its submodules into your workspace before the CodeBuild step.

Also, I have tried this method. I used Git to pull everything and the submodule was also pulled fine. But I experienced this error:

[AWS CodeBuild Plugin] Please select a project with S3 source type

Not sure where I need to set S3 source type? @leoherran-aws

Your CodeBuild project needs to be configured with S3 source. This is so the CodeBuild plugin can upload your Jenkins workspace there to be used as source in the build.

[marko-team-ai]

This plugin actually doesn't do any source downloading. You can configure it two ways:

1. Use Project source option: just start a build for your CodeBuild project. CodeBuild will download the source configured in your CodeBuild project, which can contain contain submodules.

Not sure if I am following this. It's not downloading the source, but it will download a configured source?

With this option configured the plugin will just start a build. CodeBuild (the AWS service, not the plugin) will download the source that's configured in your CodeBuild project inside your build container.

Hey @leoherran-aws, thanks for trying to help.

Maybe I am missing something big. Let try final shot :)

• If Jenkins Source Code Management is NONE
• if Project configuration is -> Use Project Source
• and when you say this: will download the source that's configured in your CodeBuild project inside your build container.

Who is responsible to download/store data here -> the source that's configured in your CodeBuild project inside your build container. ?

1. Jenkins?
2. AWS directly from GIT?
3. Something else?

Additional I have tried this:

And I have got this error: `[AWS CodeBuild Plugin] Build TeamAITest:53120ecd-b2f4-418b-9984-65b9f9cf6832 failed

Submodule error repository not found for primary source and source version 8cbab211495373de5c109d16857f4a7142e7290e (status code: CLIENT_ERROR)`

Many thanks!

[leoherran-aws]

Who is responsible to download/store data here

AWS CodeBuild is responsible for the source download in this case. The source download will happen in the build container configured for your CodeBuild project, which runs in AWS.

What is the source type for your CodeBuild project? If you are connected to GitHub or Bitbucket via OAuth, can you make sure that the connected account has access to the submodule repository?

[marko-team-ai]

Thanks for the help @leoherran-aws! Use definitely had permissions on both repositories as it's working in Jenkins from source management plugin.

We can close this one, but just to mention that checking: Enable git submodules didn't work for me.