How exactly do you disable git submodules?

[sgammon]

I have searched everywhere, and found like 4 environment variables, and one agent configuration flag, which disable auto-clone of git submodules.

It breaks our project to have this be the default. We have other projects where it would be fine (making a build agent flag a suboptimal option, but nonetheless, with forced git submodules, we can't use Buildkite).

So we ventured to disable it. After finding the aforementioned env vars and agent flag, it took about an hour to figure out how that fit in, and it also seems to just not work:

1) We tried adding it via the pipeline environment variables. Obviously, you are now chuckling to yourself, Buildkite dev, because that wasn't what you meant when you said it was configurable via environment variables! We ran face first into:

(Okay. Firstly, why? Is this a seriously sensitive flag? It seems like this should work on a Pipeline-level, anyway, so this was confusing.)

2) We realize that security is important, and so on, so, great. Let's add it properly. We dig in to find the systemd service file for the agent (this is on Ubuntu), and modify it over several change-build-frown cycles until it looks like this:

/lib/systemd/system/buildkite-agent@.service:

[Unit]
Description=Buildkite Agent (%i)
Documentation=https://buildkite.com/agent
After=syslog.target
After=network.target

[Service]
Type=simple
User=buildkite-agent
Environment=HOME=/var/lib/buildkite-agent
Environment=BUILDKITE_DISABLE_GIT_SUBMODULES=1
Environment=BUILDKITE_GIT_SUBMODULES=false
EnvironmentFile=/var/lib/buildkite-agent/ENVIRONMENT
ExecStart=BUILDKITE_GIT_SUBMODULES=false BUILDKITE_DISABLE_GIT_SUBMODULES=1 /usr/bin/buildkite-agent start --no-git-submodules
RestartSec=5
Restart=on-failure
RestartForceExitStatus=SIGPIPE
TimeoutStartSec=10
TimeoutStopSec=0
KillMode=process

[Install]
WantedBy=multi-user.target
DefaultInstance=1

/var/lib/buildkite-agent/ENVIRONMENT:

HOME=/var/lib/buildkite-agent
BUILDKITE_DISABLE_GIT_SUBMODULES=1
BUILDKITE_GIT_SUBMODULES=false

In those two samples, I count 7 different reasons why git submodules should not automatically and recursively sync all submodule code without asking the user. However, much to my dismay as an eager user of Buildkite, it barrels onward, automatically and recursively syncing all submodule code without asking the user:

For now, since the added code breaks our Bazel-based build, we have resorted to deleting the directories afterward, which is suboptimal (added build times, server activity, and disk usage...).

Is there something wrong with our approach here?

[lox]

Sorry you had such a rough time with this @sgammon. I agree it doesn't make much sense that it's a protected env, I'll discuss with @keithpitt and @sj26 to see what the context was there.

The config for disabling sub-modules should be no-git-submodules, and you should be able to set either BUILDKITE_NO_GIT_SUBMODULES, $BUILDKITE_DISABLE_GIT_SUBMODULES in systemd 🤔 Those should then be set correctly for each build that agent runs. I'm perplexed that you aren't seeing that.

Are you able to set debug: true in your agent config and then run a build and send us a build url to support@buildkite.com and we'll have a look?

[lox]

I just did some testing (an integration test and manually testing an agent with buildkite-agent start --no-git-submodules) and it seems to work 🤔

Did you do a systemctl daemon-reload after you made the unit changes?

For context, the best way to make config changes for the agent would be to set no-git-submodules=true in your /etc/buildkite-agent/buildkite-agent.cfg and then restart the agent.

As the error mentions, you can also set this in an agent hook, although it will be in inverse:

/etc/buildkite-agent/hooks/environment:

export BUILDKITE_GIT_SUBMODULES=false

[lox]

Did you have a chance to check out the above @sgammon?

[sgammon]

@lox it could very well be the daemon-reload, let me jump in and take a look. thank you for your help and advice on this

[sgammon]

@lox it still doesn't seem to work :( i can try to setup debug flags and send a log.

i presume the agent config is different from the build pipeline config? where would i find that on-disk?

[sgammon]

yoooo @lox @keithpitt are you guys ever going to fix this "supported feature"

[lox]

Mind dropping us an email to support@buildkite.com and we’ll help troubleshoot?

[keithduncan]

I’m going to close this issue as stale, if you’re still experiencing issues configuring no-git-submodules let us know and we can look into it :bow:

[jeffparsons]

@lox We'd really like to be able to configure this per-job using BUILDKITE_DISABLE_GIT_SUBMODULES as well. Seeing the same thing:

Warning: Ignored BUILDKITE_GIT_SUBMODULES

Is there a workaround available for configuring this per-job at the moment? Would it work to invent our own version of the environment variable and then use that in /etc/buildkite-agent/hooks/environment to set export BUILDKITE_GIT_SUBMODULES=false?

Or if this behavior is an accident, then maybe the agent could be updated to leave BUILDKITE_GIT_SUBMODULES alone? :slightly_smiling_face:

Cheers!