search

firecow / gitlab-ci-local

Tired of pushing to test your .gitlab-ci.yml?
MIT License
2.03k stars 115 forks source link

[FEATURE] #1217 Added feature to emulate gitlab hosted runner using the `--container-emulate` option with the name of one of the gitlab preset. #1219

Closed nicDamours closed 1 month ago

nicDamours commented 1 month ago

summary

Added an option to prefill memory, kernel-memory and cpus docker options based on the configuration of Gitlab's hosted runner. This is useful when attempting to debug performance issue. By default, no limit is applied, the host resources are all available.

When passing the option --container-emulate= with the name of a gitlab hosted runner, the docker resources is limited to the resource available for this class of hosted runner. As of this PR, those are the values

name memory CPU
saas-linux-small 8 GB 2
saas-linux-medium 16 GB 4
saas-linux-large 32 GB 8
saas-linux-xlarge 64 GB 16
saas-linux-2xlarge 128 GB 32

see the gitlab documentation here

Of course, you are also limited to the host resources. You cannot have 128 GB of RAM on a host that only has 16 GB.

As of right now, I've only added support for the base linux machine. This does not include runner with a GPUs. This could be a nice amelioration in the futur. The runners names does not contains the architecture, as this is based on the host CPU, and cannot be emulated.

Usage

You can pass an additional option --containter-emulate with one of the previously mentionned runner name. For example, using the name saas-linux-small will limit the container resource to 8 GB or RAM and 2 CPUs. 

gitlab-ci-local --cwd /path/to/.gitlab.yml --container-emulate=saas-linux-small

You can check the memory limit and CPU usage using the docker stats {container_id} command.

image

This PR Closes https://github.com/firecow/gitlab-ci-local/issues/1217

ANGkeith commented 1 month ago

thanks for the contribution !

firecow commented 1 month ago

@nicDamours I don't see the purpose of the amd64 postfix.

nicDamours commented 1 month ago

@nicDamours I don't see the purpose of the amd64 postfix.

@firecow it's the name given by gitlab for those specific runner: https://docs.gitlab.com/ee/ci/runners/hosted_runners/linux.html#machine-types-available-for-linux---x86-64

It's not essential for this feature but it helps linking gitlab's runner to the preset defined here.

firecow commented 1 month ago

@nicDamours I don't see the purpose of the amd64 postfix.

@firecow it's the name given by gitlab for those specific runner: https://docs.gitlab.com/ee/ci/runners/hosted_runners/linux.html#machine-types-available-for-linux---x86-64

It's not essential for this feature but it helps linking gitlab's runner to the preset defined here.

I can imagine people with ARM processors will be very confused, when they realise that using such a preset, actually works :smile:

nicDamours commented 1 month ago

I can see how that would be confusing. I will remove the amd64 prefix :slightly_smiling_face: