executor logs and concurrency issues

[Destrocamil]

Hey ! I'm so glad that gitlab-tart-executor exists, but currently I am facing 2 issues that could find some help 👍

I am having issues while trying to build 1 or more jobs simultaneously. The one that starts first, is all good. The second one keeps telling

The number of VMs exceeds the system limit (other running VMs: gitlab-8437130944, gitlab-8438646366)
Failed to retrieve IP address of VM "gitlab-8438646369" in 60 seconds: tart command returned non-zero exit code: "no IP address found, is your VM running?", will re-try...

My current configuration is

(...)
  [runners.custom]
    prepare_exec = "/opt/homebrew/bin/gitlab-tart-executor"
    prepare_args = ["prepare", "--concurrency", "4", "--cpu","auto", "--memory","auto"]
    run_exec = "/opt/homebrew/bin/gitlab-tart-executor"
    run_args = ["run"]
    cleanup_exec = "/opt/homebrew/bin/gitlab-tart-executor"
    cleanup_args = ["cleanup"]

Because I am having this issue, I was trying to find some logs but found none, at leas not aware where they are. Thanks!

[edigaryev]

The number of VMs exceeds the system limit

Are you running macOS VMs?

It's not possible to run more than 2 of such VMs due to the underlying Virtualization.Framework limitation.

It is possible, however, to run 2 macOS + many Linux VMs on a single host.

You can try decreasing the --concurrency to 2 to work around this.

[Destrocamil]

Thanks for the quick reply. I changed concurrency back to 2 but still facing the same issues. 2 jobs runnings, job 1:

2024/11/22 11:41:50 Cloning and configuring a new VM...
2024/11/22 11:41:50 Waiting for the VM to boot and be SSH-able...
2024/11/22 11:42:02 Was able to SSH!
2024/11/22 11:42:02 Installing GitLab Runner...

job 2:

2024/11/22 11:41:56 Cloning and configuring a new VM...
2024/11/22 11:41:56 Waiting for the VM to boot and be SSH-able...
The number of VMs exceeds the system limit (other running VMs: gitlab-8437130944, gitlab-8445654884)
Failed to retrieve IP address of VM "gitlab-8445655773" in 60 seconds: tart command returned non-zero exit code: "no IP address found, is your VM running?", will re-try...

Am I doing something wrong?!

[edigaryev]

The number of VMs exceeds the system limit (other running VMs: gitlab-8437130944, gitlab-8445654884)

You have more than 2 VMs running, not sure why.

Can try shutting down the GitLab Runner, and then stopping and deleting the existing gitlab-* VMs by hand (using tart list and tart delete)?

[Destrocamil]

You were right, seems some machine was hanging. Thanks 👍