search

actions / runner-images

GitHub Actions runner images
MIT License
9.17k stars 2.84k forks source link

Randomly getting 'Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend)' through multiple runs #1120

Closed erphunranjbar closed 3 years ago

erphunranjbar commented 3 years ago

Describe the bug When I attempt to run the Ubuntu 18.04 provisioner, I keep getting the E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it? error in random places. The image gets created nightly and i would say 90% of the time, it fails to create because of the lock... I added sleep 80 after the homebrew validation and it sometimes helps. I also added a sudo killall apt apt-get || true in the beginning of most scripts to attempt to combat this and it only sometimes helps... Anyone have any suggestions on what I can do or if there is a workaround?

Area for Triage:

Deployment/Release

Question, Bug, or Feature?:

Question/Bug

Virtual environments affected

Expected behavior I would expect the Ubuntu 18.04 image to get created

Actual behavior A description with steps to reproduce the issue. If your have a public example or repo to share, please provide the link.

  1. Run the Azure Pipeline
  2. Fails randomly on a apt-get installation. Mostly before clang-6.0, dnsutils or the dotnet-sdk installation.
Darleev commented 3 years ago

Hello @GWarrior5595, Thank you for the issue reported. Let's try to find out the reason of such behavior. I would like to ask you to add:

lsof /var/lib/dpkg/lock-frontend

before apt installation. It should help to understand which process takes the file.

The investigation is ongoing. I hope I will be able to return back with an answer or new ideas.

Darleev commented 3 years ago

Hello @GWarrior5595, Is this error still reproducing for you? We are looking forward to your reply.

erphunranjbar commented 3 years ago

@Darleev I am still running into the issue transiently. I tried the command but I don't get a response from that command before the failure happened

Darleev commented 3 years ago

Hello @GWarrior5595, I was able to reproduce the issue one time before Friday, but currently, the issue is not reproducing anymore on our side. Could you please provide us a log of mentioned command when the issue appears again?

Darleev commented 3 years ago

Hello @GWarrior5595, Let me clarify the current status of the issue: Is it still reproducing?

erphunranjbar commented 3 years ago

@Darleev I still get the issue when i try to run this nightly. It succeeds every 3-6 days

Here are the last few lines for the logs on failure

`

  azure-arm: Hit:3 http://ppa.launchpad.net/apt-fast/stable/ubuntu bionic InRelease
    azure-arm: Hit:4 http://azure.archive.ubuntu.com/ubuntu bionic InRelease
    azure-arm: Hit:5 http://azure.archive.ubuntu.com/ubuntu bionic-updates InRelease
    azure-arm: Hit:6 https://packages.microsoft.com/repos/azure-cli bionic InRelease
    azure-arm: Hit:7 http://azure.archive.ubuntu.com/ubuntu bionic-backports InRelease
    azure-arm: Hit:8 https://packages.microsoft.com/ubuntu/18.04/prod bionic InRelease
    azure-arm: Fetched 88.7 kB in 2s (51.9 kB/s)
    azure-arm: Reading package lists...
==> azure-arm: E: Could not get lock /var/lib/dpkg/lock-frontend - open (11: Resource temporarily unavailable)
==> azure-arm: E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?
==> azure-arm: --2020-07-06 00:14:35--  https://apt.llvm.org/llvm.sh
==> azure-arm: Resolving apt.llvm.org (apt.llvm.org)... 151.101.26.49, 2a04:4e42:6::561
==> azure-arm: Connecting to apt.llvm.org (apt.llvm.org)|151.101.26.49|:443... connected.
==> azure-arm: HTTP request sent, awaiting response... 200 OK
==> azure-arm: Length: 2730 (2.7K) [application/octet-stream]
==> azure-arm: Saving to: ‘llvm.sh’
==> azure-arm:
==> azure-arm:      0K ..                                                    100% 32.5M=0s
==> azure-arm:
==> azure-arm: 2020-07-06 00:14:35 (32.5 MB/s) - ‘llvm.sh’ saved [2730/2730]
==> azure-arm:
    azure-arm: Installing clang-6.0...
==> azure-arm: E: Could not get lock /var/lib/dpkg/lock-frontend - open (11: Resource temporarily unavailable)
==> azure-arm: E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?
    azure-arm: Testing to make sure that script performed as expected, and basic scenarios work
    azure-arm: clang-6.0 was not installed
==> azure-arm: Provisioning step had errors: Running the cleanup provisioner, if present...
==> azure-arm: Removing the created Deployment object: 'pkrdpx3uq2khu39'
==> azure-arm: 
==> azure-arm: Cleanup requested, deleting resource group ...
==> azure-arm: Resource group has been deleted.
Build 'azure-arm' errored: Script exited with non-zero exit status: 1.Allowed exit codes are: [0]

==> Some builds didn't complete successfully and had errors:
--> azure-arm: Script exited with non-zero exit status: 1.Allowed exit codes are: [0]

==> Builds finished but no artifacts were created.

`

miketimofeev commented 3 years ago

Hi @GWarrior5595! It seems we've found the root cause and implemented a fix. Could you please check does the issue persist on the latest master?

miketimofeev commented 3 years ago

@GWarrior5595 I'm going to close the issue since we haven't faced this issue any more after the fix but feel free to contact us if you have any concerns. Thank you!

miketimofeev commented 3 years ago

Have seen that during Ubuntu16 build again, reopening for additional investigation

erphunranjbar commented 3 years ago

@miketimofeev I have started running my packer builds again. I have pulled the latest main and I am still getting issues related to the locked files:

azure-arm: Reading package lists...
==> azure-arm: + apt-get install -y clang-9 lldb-9 lld-9 clangd-9
==> azure-arm: E: Could not get lock /var/lib/dpkg/lock-frontend - open (11: Resource temporarily unavailable)
==> azure-arm: E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?
==> azure-arm: E: Could not get lock /var/lib/dpkg/lock-frontend - open (11: Resource temporarily unavailable)
    azure-arm: Testing to make sure that script performed as expected, and basic scenarios work
    azure-arm: clang-9 was not installed
==> azure-arm: E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?
==> azure-arm: Provisioning step had errors: Running the cleanup provisioner, if present...
miketimofeev commented 3 years ago

@GWarrior5595 is it for ubuntu18? No changes being made for the template?

erphunranjbar commented 3 years ago

@miketimofeev Yes, for the Ubuntu1804. I am using the templates from "main" branch of this repo

miketimofeev commented 3 years ago

@GWarrior5595 could you please provide as many cases as possible or all of them failed on apt-get install -y clang-9 lldb-9 lld-9 clangd-9?

erphunranjbar commented 3 years ago

@miketimofeev It has failed on that several times but sometimes it gets through that and fails on other parts such as the docker installation

azure-arm: Reading package lists...
==> azure-arm: E: Could not get lock /var/lib/dpkg/lock-frontend - open (11: Resource temporarily unavailable)
==> azure-arm: E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?
==> azure-arm: E: Could not get lock /var/lib/dpkg/lock-frontend - open (11: Resource temporarily unavailable)
==> azure-arm: E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?
==> azure-arm: Failed to start docker.service: Unit docker.service not found.
==> azure-arm: Failed to get unit file state for docker.service: No such file or directory
    azure-arm: Testing to make sure that script performed as expected, and basic scenarios work
==> azure-arm: Failed to enable unit: Unit file docker.service does not exist.
    azure-arm: Checking the docker-moby and moby-buildx
    azure-arm: docker was not installed
==> azure-arm: Provisioning step had errors: Running the cleanup provisioner, if present...
miketimofeev commented 3 years ago

@GWarrior5595 So it's random. My assumption is that it can be some system update service. Would you mind adding apt-get remove unattended-upgrades to the apt.sh file here https://github.com/actions/virtual-environments/blob/e3cb3e32d496341d679cc9f2d68b2ae2053b7c55/images/linux/scripts/base/apt.sh#L9-L10 and trying to build a few images? I'm asking because, unfortunately, we don't have stable repro at the moment as I said before.

Threads with similar issues for reference: https://groups.google.com/forum/#!topic/packer-tool/NTvZP56DRqw https://github.com/geerlingguy/packer-boxes/issues/3 https://discuss.hashicorp.com/t/could-not-get-lock-var-lib-dpkg-lock-frontend/6001/2

miketimofeev commented 3 years ago

@GWarrior5595 @prashant6475 could you please try the suggested solution? I've built 6 Ubuntu 16.04 images without any issues with this change. If it helps we'll create a PR, if not — will work further on the investigation Here is the change I've made: https://github.com/actions/virtual-environments/commit/9ce59263d54214d65a9d5aaf5e276aeff8299c74

prashant6475 commented 3 years ago

@miketimofeev

We are testing on ubuntu 18.04 image will add that fix and let you know

miketimofeev commented 3 years ago

@prashant6475 @GWarrior5595 Have you been able to test it? We're looking forward to your reply, thank you!

erphunranjbar commented 3 years ago

@miketimofeev I ran the build four times and I still get a lock issue, even with the changes you made. Here is my images/linux/script/base/apt.sh:

#!/bin/bash

export DEBIAN_FRONTEND=noninteractive
apt-get -yq update
apt-get -yq dist-upgrade
apt-get remove unattended-upgrades
systemctl disable apt-daily.service
systemctl disable apt-daily.timer
systemctl disable apt-daily-upgrade.timer
systemctl disable apt-daily-upgrade.service

# Configure apt to always assume Y
echo "APT::Get::Assume-Yes \"true\";" > /etc/apt/apt.conf.d/90assumeyes

# Use apt-fast for parallel downloads
apt-get install aria2
add-apt-repository -y ppa:apt-fast/stable
apt-get update
apt-get -y install apt-fast

And here's logs from one of the errors:

azure-arm: Testing to make sure that script performed as expected, and basic scenarios work
    azure-arm: /usr/local/bin/docker-compose
    azure-arm: Documenting Docker Compose version
    azure-arm: - Docker Compose (docker-compose version 1.26.2, build eefe0d31)
==> azure-arm: Provisioning with shell script: /agent/_work/1/s/images/linux/scripts/installers/docker-moby.sh
    azure-arm: Determing if Docker (moby) is installed
    azure-arm: Docker (moby) was not found. Installing...
==> azure-arm: E: Could not get lock /var/lib/dpkg/lock-frontend - open (11: Resource temporarily unavailable)
==> azure-arm: E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?
    azure-arm: Hit:2 http://security.ubuntu.com/ubuntu bionic-security InRelease
    azure-arm: Hit:3 http://ppa.launchpad.net/ansible/ansible/ubuntu bionic InRelease
    azure-arm: Hit:4 http://ppa.launchpad.net/apt-fast/stable/ubuntu bionic InRelease
    azure-arm: Hit:5 http://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/xUbuntu_18.04  InRelease
    azure-arm: Hit:1 https://apt.llvm.org/bionic llvm-toolchain-bionic-9 InRelease
    azure-arm: Hit:6 https://packages.microsoft.com/repos/azure-cli bionic InRelease
    azure-arm: Hit:7 https://packages.microsoft.com/ubuntu/18.04/prod bionic InRelease
    azure-arm: Hit:8 http://azure.archive.ubuntu.com/ubuntu bionic InRelease
    azure-arm: Hit:9 http://azure.archive.ubuntu.com/ubuntu bionic-updates InRelease
    azure-arm: Hit:10 http://azure.archive.ubuntu.com/ubuntu bionic-backports InRelease
    azure-arm: Reading package lists...
==> azure-arm: E: Could not get lock /var/lib/dpkg/lock-frontend - open (11: Resource temporarily unavailable)
==> azure-arm: E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?
==> azure-arm: E: Could not get lock /var/lib/dpkg/lock-frontend - open (11: Resource temporarily unavailable)
==> azure-arm: E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?
==> azure-arm: Failed to start docker.service: Unit docker.service not found.
==> azure-arm: Failed to get unit file state for docker.service: No such file or directory
==> azure-arm: Failed to enable unit: Unit file docker.service does not exist.
    azure-arm: Testing to make sure that script performed as expected, and basic scenarios work
    azure-arm: Checking the docker-moby and moby-buildx
    azure-arm: docker was not installed
==> azure-arm: Provisioning step had errors: Running the cleanup provisioner, if present...
==> azure-arm: Removing the created Deployment object: 'pkrdp762dg9w5ew'
miketimofeev commented 3 years ago

@GWarrior5595 thanks for the update! Strangely we've not seen this for the last 2 months but anyway I'm going to try something like this and wrap all the executions into wait function https://github.com/geerlingguy/packer-boxes/issues/7#issuecomment-424485739

erphunranjbar commented 3 years ago

@miketimofeev I have also started getting this issue with the aws.sh, where the plugin command isn't found. Not sure if it's related to dpkg getting locked

    azure-arm: Note: AWS CLI version 2, the latest major version of the AWS CLI, is now stable and recommended for general use. For more information, see the AWS CLI version 2 installation instructions at: https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html
    azure-arm: Validate the installation
    azure-arm: /usr/local/bin/aws
==> azure-arm:   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
==> azure-arm:                                  Dload  Upload   Total   Spent    Left  Speed
==> azure-arm: 100 4830k  100 4830k    0     0  6052k      0 --:--:-- --:--:-- --:--:-- 6045k
==> azure-arm: dpkg: error: dpkg frontend is locked by another process
    azure-arm:
==> azure-arm: /tmp/script_1696.sh: line 38: session-manager-plugin: command not found
miketimofeev commented 3 years ago

@GWarrior5595 looks like the same issue since it uses dpkg to install the plugin.

Darleev commented 3 years ago

@erphunranjbar I have created a separate branch in the upstream repository where implemented retry logic for apt/dpkg, which should resolve the issue. Could you please try to build from the mentioned branch and provide a result?

We are looking forward to your reply.

Darleev commented 3 years ago

@erphunranjbar Could you please take a look at my reply above and test the provided solution?

We are looking forward to your reply.

ausfestivus commented 3 years ago

@Darleev ill use your branch for now and see if it helps.

ausfestivus commented 3 years ago

Morning @Darleev here is some output from a build with your branch. Failed in clang9.

2020-09-17T22:32:56.7747109Z     azure-arm: Testing to make sure that script performed as expected, and basic scenarios work
2020-09-17T22:32:56.7749270Z     azure-arm: /usr/bin/clang-8
2020-09-17T22:32:56.7755374Z     azure-arm: /usr/bin/clang++-8
2020-09-17T22:32:56.7758097Z     azure-arm: Documenting clang-8...
2020-09-17T22:32:56.8146951Z     azure-arm: - Clang 8 (8.0.0)
2020-09-17T22:32:56.8149612Z     azure-arm: Installing clang-9...
2020-09-17T22:32:56.8169544Z ==> azure-arm: + needed_binaries=(lsb_release wget add-apt-repository)
2020-09-17T22:32:56.8170190Z ==> azure-arm: + missing_binaries=()
2020-09-17T22:32:56.8178759Z ==> azure-arm: + for binary in "${needed_binaries[@]}"
2020-09-17T22:32:56.8192082Z ==> azure-arm: + which lsb_release
2020-09-17T22:32:56.8192948Z ==> azure-arm: + for binary in "${needed_binaries[@]}"
2020-09-17T22:32:56.8197576Z ==> azure-arm: + which wget
2020-09-17T22:32:56.8204917Z ==> azure-arm: + for binary in "${needed_binaries[@]}"
2020-09-17T22:32:56.8214179Z ==> azure-arm: + which add-apt-repository
2020-09-17T22:32:56.8217617Z ==> azure-arm: + [[ 0 -gt 0 ]]
2020-09-17T22:32:56.8221815Z ==> azure-arm: + LLVM_VERSION=11
2020-09-17T22:32:56.8222449Z ==> azure-arm: + '[' 1 -eq 1 ']'
2020-09-17T22:32:56.8222824Z ==> azure-arm: + LLVM_VERSION=9
2020-09-17T22:32:56.8223234Z ==> azure-arm: ++ lsb_release -is
2020-09-17T22:32:56.8689886Z ==> azure-arm: + DISTRO=Ubuntu
2020-09-17T22:32:56.8693793Z ==> azure-arm: ++ lsb_release -sr
2020-09-17T22:32:56.9175240Z ==> azure-arm: + VERSION=18.04
2020-09-17T22:32:56.9178898Z ==> azure-arm: + DIST_VERSION=Ubuntu_18.04
2020-09-17T22:32:56.9184331Z ==> azure-arm: + [[ 0 -ne 0 ]]
2020-09-17T22:32:56.9190590Z ==> azure-arm: + declare -A LLVM_VERSION_PATTERNS
2020-09-17T22:32:56.9205340Z ==> azure-arm: + LLVM_VERSION_PATTERNS[9]=-9
2020-09-17T22:32:56.9205778Z ==> azure-arm: + LLVM_VERSION_PATTERNS[10]=-10
2020-09-17T22:32:56.9206197Z ==> azure-arm: + LLVM_VERSION_PATTERNS[11]=-11
2020-09-17T22:32:56.9206573Z ==> azure-arm: + LLVM_VERSION_PATTERNS[12]=
2020-09-17T22:32:56.9211520Z ==> azure-arm: + '[' '!' _ ']'
2020-09-17T22:32:56.9217090Z ==> azure-arm: + LLVM_VERSION_STRING=-9
2020-09-17T22:32:56.9243016Z ==> azure-arm: + case "$DIST_VERSION" in
2020-09-17T22:32:56.9243777Z ==> azure-arm: + REPO_NAME='deb http://apt.llvm.org/bionic/   llvm-toolchain-bionic-9  main'
2020-09-17T22:32:56.9244661Z ==> azure-arm: + wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key
2020-09-17T22:32:56.9245271Z ==> azure-arm: + apt-key add -
2020-09-17T22:32:56.9245938Z ==> azure-arm: --2020-09-17 22:32:56--  https://apt.llvm.org/llvm-snapshot.gpg.key
2020-09-17T22:32:56.9274367Z ==> azure-arm: Resolving apt.llvm.org (apt.llvm.org)... 151.101.54.49, 2a04:4e42:2e::561
2020-09-17T22:32:56.9809722Z ==> azure-arm: Connecting to apt.llvm.org (apt.llvm.org)|151.101.54.49|:443... Warning: apt-key output should not be parsed (stdout is not a terminal)
2020-09-17T22:32:57.0818496Z ==> azure-arm: connected.
2020-09-17T22:32:57.5514911Z ==> azure-arm: HTTP request sent, awaiting response... 200 OK
2020-09-17T22:32:57.5517286Z ==> azure-arm: Length: 3145 (3.1K) [application/octet-stream]
2020-09-17T22:32:57.5519271Z ==> azure-arm: Saving to: ‘STDOUT’
2020-09-17T22:32:57.5522230Z ==> azure-arm:
2020-09-17T22:32:57.5522895Z ==> azure-arm:      0K ...                                                   100% 55.6M=0s
2020-09-17T22:32:57.5527154Z ==> azure-arm:
2020-09-17T22:32:57.5531805Z ==> azure-arm: 2020-09-17 22:32:57 (55.6 MB/s) - written to stdout [3145/3145]
2020-09-17T22:32:57.5534781Z ==> azure-arm:
2020-09-17T22:32:57.7901002Z     azure-arm: OK
2020-09-17T22:32:57.7973196Z ==> azure-arm: + add-apt-repository 'deb http://apt.llvm.org/bionic/   llvm-toolchain-bionic-9  main'
2020-09-17T22:32:58.5726383Z     azure-arm: Hit:1 http://azure.archive.ubuntu.com/ubuntu bionic InRelease
2020-09-17T22:32:58.5727254Z     azure-arm: Hit:2 http://azure.archive.ubuntu.com/ubuntu bionic-updates InRelease
2020-09-17T22:32:58.5728141Z     azure-arm: Hit:3 http://azure.archive.ubuntu.com/ubuntu bionic-backports InRelease
2020-09-17T22:32:58.9600020Z     azure-arm: Hit:4 http://security.ubuntu.com/ubuntu bionic-security InRelease
2020-09-17T22:32:58.9645425Z     azure-arm: Hit:5 http://ppa.launchpad.net/ansible/ansible/ubuntu bionic InRelease
2020-09-17T22:32:59.0423609Z     azure-arm: Hit:7 https://packages.microsoft.com/repos/azure-cli bionic InRelease
2020-09-17T22:32:59.1301774Z     azure-arm: Hit:8 https://packages.microsoft.com/ubuntu/18.04/prod bionic InRelease
2020-09-17T22:32:59.4552179Z     azure-arm: Hit:9 http://ppa.launchpad.net/apt-fast/stable/ubuntu bionic InRelease
2020-09-17T22:32:59.6651828Z     azure-arm: Get:6 https://apt.llvm.org/bionic llvm-toolchain-bionic-9 InRelease [4231 B]
2020-09-17T22:32:59.9766883Z     azure-arm: Get:10 https://apt.llvm.org/bionic llvm-toolchain-bionic-9/main amd64 Packages [8845 B]
2020-09-17T22:33:02.3573722Z     azure-arm: Fetched 13.1 kB in 2s (8591 B/s)
2020-09-17T22:33:03.5350680Z     azure-arm: Reading package lists...
2020-09-17T22:33:04.3078774Z ==> azure-arm: + apt-get update
2020-09-17T22:33:04.3079349Z     azure-arm: Hit:1 http://azure.archive.ubuntu.com/ubuntu bionic InRelease
2020-09-17T22:33:04.3079918Z     azure-arm: Hit:2 http://azure.archive.ubuntu.com/ubuntu bionic-updates InRelease
2020-09-17T22:33:04.3080503Z     azure-arm: Hit:3 http://azure.archive.ubuntu.com/ubuntu bionic-backports InRelease
2020-09-17T22:33:04.3081087Z     azure-arm: Hit:5 https://packages.microsoft.com/repos/azure-cli bionic InRelease
2020-09-17T22:33:04.3081689Z     azure-arm: Hit:6 http://ppa.launchpad.net/ansible/ansible/ubuntu bionic InRelease
2020-09-17T22:33:04.3082246Z     azure-arm: Hit:7 http://security.ubuntu.com/ubuntu bionic-security InRelease
2020-09-17T22:33:04.3083000Z     azure-arm: Hit:8 https://packages.microsoft.com/ubuntu/18.04/prod bionic InRelease
2020-09-17T22:33:04.6367972Z     azure-arm: Hit:4 https://apt.llvm.org/bionic llvm-toolchain-bionic-9 InRelease
2020-09-17T22:33:04.9684040Z     azure-arm: Hit:9 http://ppa.launchpad.net/apt-fast/stable/ubuntu bionic InRelease
2020-09-17T22:33:06.3231628Z     azure-arm: Reading package lists...
2020-09-17T22:33:06.3706560Z ==> azure-arm: + apt-get install -y clang-9 lldb-9 lld-9 clangd-9
2020-09-17T22:33:06.4206161Z ==> azure-arm: E: Could not get lock /var/lib/dpkg/lock-frontend - open (11: Resource temporarily unavailable)
2020-09-17T22:33:06.4207300Z ==> azure-arm: E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?
2020-09-17T22:33:07.5058069Z     azure-arm: Waiting for apt lock...
2020-09-17T22:33:07.7517123Z     azure-arm: Reading package lists...
2020-09-17T22:33:07.9454058Z     azure-arm: Building dependency tree...
2020-09-17T22:33:07.9454864Z     azure-arm: Reading state information...
2020-09-17T22:33:08.0681449Z     azure-arm: The following packages were automatically installed and are no longer required:
2020-09-17T22:33:08.0690146Z     azure-arm:   bc dns-root-data dnsmasq-base ebtables grub-pc-bin landscape-common
2020-09-17T22:33:08.0697895Z     azure-arm:   liblldb-6.0 liblxc-common liblxc1 libuv1 linux-headers-4.15.0-117 lxcfs lxd
2020-09-17T22:33:08.0714374Z     azure-arm:   lxd-client python3-attr python3-automat python3-click python3-colorama
2020-09-17T22:33:08.0715732Z     azure-arm:   python3-constantly python3-hyperlink python3-incremental python3-pam
2020-09-17T22:33:08.0718885Z     azure-arm:   python3-pyasn1 python3-pyasn1-modules python3-service-identity
2020-09-17T22:33:08.0725354Z     azure-arm:   python3-twisted python3-twisted-bin python3-zope.interface uidmap xdelta3
2020-09-17T22:33:08.0728398Z     azure-arm: Use 'sudo apt autoremove' to remove them.
2020-09-17T22:33:08.0729918Z     azure-arm: The following additional packages will be installed:
2020-09-17T22:33:08.0730589Z     azure-arm:   libclang-cpp9 libllvm9
2020-09-17T22:33:08.1039049Z     azure-arm: The following NEW packages will be installed:
2020-09-17T22:33:08.1050305Z     azure-arm:   clang-format-9 libclang-cpp9 libllvm9
2020-09-17T22:33:08.3690944Z     azure-arm: 0 upgraded, 3 newly installed, 0 to remove and 1 not upgraded.
2020-09-17T22:33:08.3691554Z     azure-arm: Need to get 40.0 MB of archives.
2020-09-17T22:33:08.3692226Z     azure-arm: After this operation, 213 MB of additional disk space will be used.
2020-09-17T22:33:08.3692918Z     azure-arm: Get:1 http://azure.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libllvm9 amd64 1:9-2~ubuntu18.04.2 [14.8 MB]
2020-09-17T22:33:08.9185117Z     azure-arm: Get:2 http://azure.archive.ubuntu.com/ubuntu bionic-updates/universe amd64 libclang-cpp9 amd64 1:9-2~ubuntu18.04.2 [25.1 MB]
2020-09-17T22:33:09.9890612Z     azure-arm: Get:3 http://azure.archive.ubuntu.com/ubuntu bionic-updates/universe amd64 clang-format-9 amd64 1:9-2~ubuntu18.04.2 [38.2 kB]
2020-09-17T22:33:10.3412782Z     azure-arm: Fetched 40.0 MB in 2s (21.3 MB/s)
2020-09-17T22:33:11.4150171Z     azure-arm: Selecting previously unselected package libllvm9:amd64.
2020-09-17T22:33:11.4734967Z     azure-arm: (Reading database ... 147955 files and directories currently installed.)
2020-09-17T22:33:11.4806802Z     azure-arm: Preparing to unpack .../libllvm9_1%3a9-2~ubuntu18.04.2_amd64.deb ...
2020-09-17T22:33:11.5196501Z     azure-arm: Unpacking libllvm9:amd64 (1:9-2~ubuntu18.04.2) ...
2020-09-17T22:33:14.8234280Z     azure-arm: Selecting previously unselected package libclang-cpp9.
2020-09-17T22:33:14.8373536Z     azure-arm: Preparing to unpack .../libclang-cpp9_1%3a9-2~ubuntu18.04.2_amd64.deb ...
2020-09-17T22:33:14.8586222Z     azure-arm: Unpacking libclang-cpp9 (1:9-2~ubuntu18.04.2) ...
2020-09-17T22:33:21.0387612Z     azure-arm: Selecting previously unselected package clang-format-9.
2020-09-17T22:33:21.0546831Z     azure-arm: Preparing to unpack .../clang-format-9_1%3a9-2~ubuntu18.04.2_amd64.deb ...
2020-09-17T22:33:21.0731164Z     azure-arm: Unpacking clang-format-9 (1:9-2~ubuntu18.04.2) ...
2020-09-17T22:33:21.3769904Z     azure-arm: Setting up libllvm9:amd64 (1:9-2~ubuntu18.04.2) ...
2020-09-17T22:33:21.4408908Z     azure-arm: Setting up libclang-cpp9 (1:9-2~ubuntu18.04.2) ...
2020-09-17T22:33:21.5074512Z     azure-arm: Setting up clang-format-9 (1:9-2~ubuntu18.04.2) ...
2020-09-17T22:33:21.5657335Z     azure-arm: Processing triggers for libc-bin (2.27-3ubuntu1.2) ...
2020-09-17T22:33:21.6377149Z     azure-arm: Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
2020-09-17T22:33:23.3497066Z     azure-arm: Testing to make sure that script performed as expected, and basic scenarios work
2020-09-17T22:33:23.3497712Z     azure-arm: clang-9 was not installed
2020-09-17T22:33:23.3498648Z ==> azure-arm: Provisioning step had errors: Running the cleanup provisioner, if present...
ausfestivus commented 3 years ago

Additionally, the following warnings are displayed at the end of a build:

==> azure-arm: Provisioning with shell script: /home/vsts/work/1/s/virtual-environments/images/linux/scripts/installers/cleanup.sh
==> azure-arm: /tmp/script_7457.sh: line 2: /install.sh: No such file or directory
==> azure-arm: /tmp/script_7457.sh: line 8: wait_for_apt_lock: command not found
Darleev commented 3 years ago

@ausfestivus thank you for the information. It's very strange since before branch publishing it was tested that build works as expected. A fix is underway. I will keep you informed.

ausfestivus commented 3 years ago

I managed to get a couple of good builds out of the branch today.

No other failures since the logs I posted earlier.

Will keep an eye out for the updates.

Darleev commented 3 years ago

@ausfestivus thank you for the update, I have prepared another workaround for the issue, and added it to separate branch in the forked repo. @erphunranjbar When you have a chance, could you please try to build from the following branch.

We are looking forward to your reply.

Darleev commented 3 years ago

@ausfestivus @erphunranjbar I have updated the branch with the latest image scripts. I believe this solution should help to avoid the issue with package locks. Could you please try to build and check that everything works as expected?

We are looking forward to your reply.

ausfestivus commented 3 years ago

Hey @Darleev. Appreciate the poke. Im currently in a launch run up this week and next. Im unlikely to get time allocated to review this till after that. Unless some time in my schedule becomes available in which I will look at it sooner.

Sorry mate.

ausfestivus commented 3 years ago

Hey @Darleev, some time did turn up. I had one scheduled run which failed in one landscape.

2020-10-04T03:11:34.4994859Z ==> azure-arm: + apt-get install -y clang-9 lldb-9 lld-9 clangd-9
2020-10-04T03:11:34.5487315Z ==> azure-arm: E: Could not get lock /var/lib/dpkg/lock-frontend - open (11: Resource temporarily unavailable)
2020-10-04T03:11:34.5488363Z ==> azure-arm: E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?
2020-10-04T03:11:35.6128264Z     azure-arm: Waiting for apt lock...
2020-10-04T03:11:36.6687428Z     azure-arm: Waiting for apt lock...
2020-10-04T03:11:37.7231896Z     azure-arm: Waiting for apt lock...
2020-10-04T03:11:38.7817786Z     azure-arm: Waiting for apt lock...
2020-10-04T03:11:39.8335140Z     azure-arm: Waiting for apt lock...
2020-10-04T03:11:40.8861940Z     azure-arm: Waiting for apt lock...
2020-10-04T03:11:41.9405142Z     azure-arm: Waiting for apt lock...
2020-10-04T03:11:42.9910032Z     azure-arm: Waiting for apt lock...
2020-10-04T03:11:44.0426424Z     azure-arm: Waiting for apt lock...
2020-10-04T03:11:45.1014813Z     azure-arm: Waiting for apt lock...
2020-10-04T03:11:46.1524807Z     azure-arm: Waiting for apt lock...
2020-10-04T03:11:46.3808946Z     azure-arm: Reading package lists...
2020-10-04T03:11:46.5719357Z     azure-arm: Building dependency tree...
2020-10-04T03:11:46.5729304Z     azure-arm: Reading state information...
2020-10-04T03:11:46.7005100Z     azure-arm: The following packages were automatically installed and are no longer required:
2020-10-04T03:11:46.7020268Z     azure-arm:   bc dns-root-data dnsmasq-base ebtables grub-pc-bin landscape-common
2020-10-04T03:11:46.7022830Z     azure-arm:   liblldb-6.0 liblxc-common liblxc1 libuv1 linux-headers-4.15.0-118 lxcfs lxd
2020-10-04T03:11:46.7026197Z     azure-arm:   lxd-client python3-attr python3-automat python3-click python3-colorama
2020-10-04T03:11:46.7029156Z     azure-arm:   python3-constantly python3-hyperlink python3-incremental python3-pam
2020-10-04T03:11:46.7056590Z     azure-arm:   python3-pyasn1 python3-pyasn1-modules python3-service-identity
2020-10-04T03:11:46.7060909Z     azure-arm:   python3-twisted python3-twisted-bin python3-zope.interface uidmap xdelta3
2020-10-04T03:11:46.7064301Z     azure-arm: Use 'sudo apt autoremove' to remove them.
2020-10-04T03:11:46.7066859Z     azure-arm: The following additional packages will be installed:
2020-10-04T03:11:46.7067726Z     azure-arm:   libclang-cpp9 libllvm9
2020-10-04T03:11:46.7418839Z     azure-arm: The following NEW packages will be installed:
2020-10-04T03:11:46.7425122Z     azure-arm:   clang-format-9 libclang-cpp9 libllvm9
2020-10-04T03:11:47.0041428Z     azure-arm: 0 upgraded, 3 newly installed, 0 to remove and 2 not upgraded.
2020-10-04T03:11:47.0054023Z     azure-arm: Need to get 40.0 MB of archives.
2020-10-04T03:11:47.0059278Z     azure-arm: After this operation, 213 MB of additional disk space will be used.
2020-10-04T03:11:47.0063879Z     azure-arm: Get:1 http://azure.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libllvm9 amd64 1:9-2~ubuntu18.04.2 [14.8 MB]
2020-10-04T03:11:47.7659621Z     azure-arm: Get:2 http://azure.archive.ubuntu.com/ubuntu bionic-updates/universe amd64 libclang-cpp9 amd64 1:9-2~ubuntu18.04.2 [25.1 MB]
2020-10-04T03:11:48.6939158Z     azure-arm: Get:3 http://azure.archive.ubuntu.com/ubuntu bionic-updates/universe amd64 clang-format-9 amd64 1:9-2~ubuntu18.04.2 [38.2 kB]
2020-10-04T03:11:49.0478150Z     azure-arm: Fetched 40.0 MB in 2s (20.6 MB/s)
2020-10-04T03:11:50.1665304Z     azure-arm: Selecting previously unselected package libllvm9:amd64.
2020-10-04T03:11:50.2307806Z     azure-arm: (Reading database ... 158262 files and directories currently installed.)
2020-10-04T03:11:50.2356552Z     azure-arm: Preparing to unpack .../libllvm9_1%3a9-2~ubuntu18.04.2_amd64.deb ...
2020-10-04T03:11:50.2784693Z     azure-arm: Unpacking libllvm9:amd64 (1:9-2~ubuntu18.04.2) ...
2020-10-04T03:11:54.1641221Z     azure-arm: Selecting previously unselected package libclang-cpp9.
2020-10-04T03:11:54.1769776Z     azure-arm: Preparing to unpack .../libclang-cpp9_1%3a9-2~ubuntu18.04.2_amd64.deb ...
2020-10-04T03:11:54.1936835Z     azure-arm: Unpacking libclang-cpp9 (1:9-2~ubuntu18.04.2) ...
2020-10-04T03:12:06.7597693Z     azure-arm: Selecting previously unselected package clang-format-9.
2020-10-04T03:12:06.7736443Z     azure-arm: Preparing to unpack .../clang-format-9_1%3a9-2~ubuntu18.04.2_amd64.deb ...
2020-10-04T03:12:06.7914733Z     azure-arm: Unpacking clang-format-9 (1:9-2~ubuntu18.04.2) ...
2020-10-04T03:12:08.0627786Z     azure-arm: Setting up libllvm9:amd64 (1:9-2~ubuntu18.04.2) ...
2020-10-04T03:12:08.1284605Z     azure-arm: Setting up libclang-cpp9 (1:9-2~ubuntu18.04.2) ...
2020-10-04T03:12:08.1878842Z     azure-arm: Setting up clang-format-9 (1:9-2~ubuntu18.04.2) ...
2020-10-04T03:12:08.2435808Z     azure-arm: Processing triggers for libc-bin (2.27-3ubuntu1.2) ...
2020-10-04T03:12:08.3098525Z     azure-arm: Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
2020-10-04T03:12:10.1247363Z     azure-arm: Testing to make sure that script performed as expected, and basic scenarios work
2020-10-04T03:12:10.1287969Z     azure-arm: clang-9 was not installed
2020-10-04T03:12:10.1291699Z ==> azure-arm: Provisioning step had errors: Running the cleanup provisioner, if present...
2020-10-04T03:12:10.1293997Z ==> azure-arm: Removing the created Deployment object: 'pkrdpw97gluc9fv'
2020-10-04T03:12:25.5353588Z ==> azure-arm: 
2020-10-04T03:12:25.5354246Z ==> azure-arm: Cleanup requested, deleting resource group ...
2020-10-04T03:19:11.3220526Z ==> azure-arm: Resource group has been deleted.
2020-10-04T03:19:11.3227942Z Build 'azure-arm' errored after 18 minutes 24 seconds: Script exited with non-zero exit status: 1.Allowed exit codes are: [0]
2020-10-04T03:19:11.3229209Z 
2020-10-04T03:19:11.3229875Z ==> Wait completed after 18 minutes 24 seconds
2020-10-04T03:19:11.3230425Z 
2020-10-04T03:19:11.3231266Z ==> Some builds didn't complete successfully and had errors:
2020-10-04T03:19:11.3231988Z --> azure-arm: Script exited with non-zero exit status: 1.Allowed exit codes are: [0]
2020-10-04T03:19:11.3232294Z 
2020-10-04T03:19:11.3232539Z ==> Builds finished but no artifacts were created.
2020-10-04T03:19:12.3102248Z

I re-ran the job and it finished fine. Here is the output from the job:

2020-10-04T06:42:16.2370488Z ==> azure-arm: + apt-get install -y clang-9 lldb-9 lld-9 clangd-9
2020-10-04T06:42:16.3168254Z     azure-arm: Reading package lists...
2020-10-04T06:42:16.5231071Z     azure-arm: Building dependency tree...
2020-10-04T06:42:16.5231805Z     azure-arm: Reading state information...
2020-10-04T06:42:16.6701229Z     azure-arm: The following packages were automatically installed and are no longer required:
2020-10-04T06:42:16.6702883Z     azure-arm:   bc dns-root-data dnsmasq-base ebtables grub-pc-bin landscape-common
2020-10-04T06:42:16.6712286Z     azure-arm:   liblldb-6.0 liblldb-8 liblxc-common liblxc1 libuv1 linux-headers-4.15.0-118
2020-10-04T06:42:16.6713621Z     azure-arm:   lxcfs lxd lxd-client python3-attr python3-automat python3-click
2020-10-04T06:42:16.6717536Z     azure-arm:   python3-colorama python3-constantly python3-hyperlink python3-incremental
2020-10-04T06:42:16.6718491Z     azure-arm:   python3-pam python3-pyasn1 python3-pyasn1-modules python3-service-identity
2020-10-04T06:42:16.6719246Z     azure-arm:   python3-twisted python3-twisted-bin python3-zope.interface uidmap xdelta3
2020-10-04T06:42:16.6719912Z     azure-arm: Use 'sudo apt autoremove' to remove them.
2020-10-04T06:42:16.6720567Z     azure-arm: The following additional packages will be installed:
2020-10-04T06:42:16.6721298Z     azure-arm:   libclang-common-9-dev libclang-cpp9 libclang1-9 liblldb-9 libllvm9
2020-10-04T06:42:16.6722059Z     azure-arm:   libomp-9-dev libomp5-9 libpfm4 llvm-9 llvm-9-dev llvm-9-runtime llvm-9-tools
2020-10-04T06:42:16.6722769Z     azure-arm:   python-chardet python-lldb-9 python-pygments python3-pygments
2020-10-04T06:42:16.6723328Z     azure-arm: Suggested packages:
2020-10-04T06:42:16.6724321Z     azure-arm:   clang-9-doc libomp-9-doc llvm-9-doc ttf-bitstream-vera
2020-10-04T06:42:16.7175373Z     azure-arm: The following packages will be REMOVED:
2020-10-04T06:42:16.7185709Z     azure-arm:   libomp-8-dev libomp5-8 lldb-8 python-lldb-8
2020-10-04T06:42:16.7187726Z     azure-arm: The following NEW packages will be installed:
2020-10-04T06:42:16.7209158Z     azure-arm:   clang-9 clangd-9 libclang-common-9-dev libclang-cpp9 libclang1-9 liblldb-9
2020-10-04T06:42:16.7215435Z     azure-arm:   libllvm9 libomp-9-dev libomp5-9 libpfm4 lld-9 lldb-9 llvm-9 llvm-9-dev
2020-10-04T06:42:16.7216793Z     azure-arm:   llvm-9-runtime llvm-9-tools python-chardet python-lldb-9 python-pygments
2020-10-04T06:42:16.7218092Z     azure-arm:   python3-pygments
2020-10-04T06:42:16.7377081Z     azure-arm: 0 upgraded, 20 newly installed, 4 to remove and 0 not upgraded.
2020-10-04T06:42:16.7388751Z     azure-arm: Need to get 108 MB of archives.
2020-10-04T06:42:16.7389731Z     azure-arm: After this operation, 568 MB of additional disk space will be used.
2020-10-04T06:42:16.7390583Z     azure-arm: Get:1 http://azure.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libllvm9 amd64 1:9-2~ubuntu18.04.2 [14.8 MB]
2020-10-04T06:42:17.1124156Z     azure-arm: Get:2 http://azure.archive.ubuntu.com/ubuntu bionic-updates/universe amd64 libclang-cpp9 amd64 1:9-2~ubuntu18.04.2 [25.1 MB]
2020-10-04T06:42:17.7331387Z     azure-arm: Get:3 http://azure.archive.ubuntu.com/ubuntu bionic-updates/universe amd64 libclang-common-9-dev amd64 1:9-2~ubuntu18.04.2 [3861 kB]
2020-10-04T06:42:17.8330525Z     azure-arm: Get:4 http://azure.archive.ubuntu.com/ubuntu bionic-updates/universe amd64 libclang1-9 amd64 1:9-2~ubuntu18.04.2 [6701 kB]
2020-10-04T06:42:18.0144119Z     azure-arm: Get:5 http://azure.archive.ubuntu.com/ubuntu bionic-updates/universe amd64 clang-9 amd64 1:9-2~ubuntu18.04.2 [1109 kB]
2020-10-04T06:42:18.0343025Z     azure-arm: Get:6 http://azure.archive.ubuntu.com/ubuntu bionic-updates/universe amd64 clangd-9 amd64 1:9-2~ubuntu18.04.2 [5885 kB]
2020-10-04T06:42:18.1437983Z     azure-arm: Get:7 http://azure.archive.ubuntu.com/ubuntu bionic-updates/universe amd64 liblldb-9 amd64 1:9-2~ubuntu18.04.2 [9023 kB]
2020-10-04T06:42:18.3232927Z     azure-arm: Get:8 http://azure.archive.ubuntu.com/ubuntu bionic-updates/universe amd64 libomp5-9 amd64 1:9-2~ubuntu18.04.2 [299 kB]
2020-10-04T06:42:18.3319238Z     azure-arm: Get:9 http://azure.archive.ubuntu.com/ubuntu bionic-updates/universe amd64 libomp-9-dev amd64 1:9-2~ubuntu18.04.2 [58.6 kB]
2020-10-04T06:42:18.3320531Z     azure-arm: Get:10 http://azure.archive.ubuntu.com/ubuntu bionic-updates/universe amd64 lld-9 amd64 1:9-2~ubuntu18.04.2 [915 kB]
2020-10-04T06:42:18.3525306Z     azure-arm: Get:11 http://azure.archive.ubuntu.com/ubuntu bionic-updates/universe amd64 llvm-9-runtime amd64 1:9-2~ubuntu18.04.2 [176 kB]
2020-10-04T06:42:18.3575442Z     azure-arm: Get:12 http://azure.archive.ubuntu.com/ubuntu bionic/main amd64 libpfm4 amd64 4.9.0-2 [225 kB]
2020-10-04T06:42:18.3642915Z     azure-arm: Get:13 http://azure.archive.ubuntu.com/ubuntu bionic-updates/universe amd64 llvm-9 amd64 1:9-2~ubuntu18.04.2 [4874 kB]
2020-10-04T06:42:18.4831765Z     azure-arm: Get:14 http://azure.archive.ubuntu.com/ubuntu bionic/main amd64 python-pygments all 2.2.0+dfsg-1 [577 kB]
2020-10-04T06:42:18.4976645Z     azure-arm: Get:15 http://azure.archive.ubuntu.com/ubuntu bionic/main amd64 python3-pygments all 2.2.0+dfsg-1 [574 kB]
2020-10-04T06:42:18.5105067Z     azure-arm: Get:16 http://azure.archive.ubuntu.com/ubuntu bionic-updates/universe amd64 llvm-9-tools amd64 1:9-2~ubuntu18.04.2 [250 kB]
2020-10-04T06:42:18.5172883Z     azure-arm: Get:17 http://azure.archive.ubuntu.com/ubuntu bionic-updates/universe amd64 llvm-9-dev amd64 1:9-2~ubuntu18.04.2 [26.6 MB]
2020-10-04T06:42:19.1661919Z     azure-arm: Get:18 http://azure.archive.ubuntu.com/ubuntu bionic-updates/universe amd64 python-lldb-9 amd64 1:9-2~ubuntu18.04.2 [4348 B]
2020-10-04T06:42:19.1669007Z     azure-arm: Get:19 http://azure.archive.ubuntu.com/ubuntu bionic-updates/universe amd64 lldb-9 amd64 1:9-2~ubuntu18.04.2 [7188 kB]
2020-10-04T06:42:19.3419037Z     azure-arm: Get:20 http://azure.archive.ubuntu.com/ubuntu bionic/main amd64 python-chardet all 3.0.4-1 [80.3 kB]
2020-10-04T06:42:20.9521419Z     azure-arm: Fetched 108 MB in 3s (41.4 MB/s)
2020-10-04T06:42:21.0611103Z     azure-arm: (Reading database ... 148102 files and directories currently installed.)
2020-10-04T06:42:21.0649292Z     azure-arm: Removing libomp-8-dev (1:8-3~ubuntu18.04.2) ...
2020-10-04T06:42:21.3032870Z     azure-arm: Removing libomp5-8:amd64 (1:8-3~ubuntu18.04.2) ...
2020-10-04T06:42:21.5662150Z     azure-arm: Removing lldb-8 (1:8-3~ubuntu18.04.2) ...
2020-10-04T06:42:21.8252268Z     azure-arm: Removing python-lldb-8 (1:8-3~ubuntu18.04.2) ...
2020-10-04T06:42:22.2242597Z     azure-arm: Selecting previously unselected package libllvm9:amd64.
2020-10-04T06:42:22.2906695Z     azure-arm: (Reading database ... 148032 files and directories currently installed.)
2020-10-04T06:42:22.2954062Z     azure-arm: Preparing to unpack .../00-libllvm9_1%3a9-2~ubuntu18.04.2_amd64.deb ...
2020-10-04T06:42:22.3210010Z     azure-arm: Unpacking libllvm9:amd64 (1:9-2~ubuntu18.04.2) ...
2020-10-04T06:42:25.6916766Z     azure-arm: Selecting previously unselected package libclang-cpp9.
2020-10-04T06:42:25.7035360Z     azure-arm: Preparing to unpack .../01-libclang-cpp9_1%3a9-2~ubuntu18.04.2_amd64.deb ...
2020-10-04T06:42:25.7225870Z     azure-arm: Unpacking libclang-cpp9 (1:9-2~ubuntu18.04.2) ...
2020-10-04T06:42:32.2490132Z     azure-arm: Selecting previously unselected package libclang-common-9-dev.
2020-10-04T06:42:32.2615938Z     azure-arm: Preparing to unpack .../02-libclang-common-9-dev_1%3a9-2~ubuntu18.04.2_amd64.deb ...
2020-10-04T06:42:32.2832026Z     azure-arm: Unpacking libclang-common-9-dev (1:9-2~ubuntu18.04.2) ...
2020-10-04T06:42:33.5231891Z     azure-arm: Selecting previously unselected package libclang1-9.
2020-10-04T06:42:33.5363758Z     azure-arm: Preparing to unpack .../03-libclang1-9_1%3a9-2~ubuntu18.04.2_amd64.deb ...
2020-10-04T06:42:33.5637958Z     azure-arm: Unpacking libclang1-9 (1:9-2~ubuntu18.04.2) ...
2020-10-04T06:42:35.0977348Z     azure-arm: Selecting previously unselected package clang-9.
2020-10-04T06:42:35.1086067Z     azure-arm: Preparing to unpack .../04-clang-9_1%3a9-2~ubuntu18.04.2_amd64.deb ...
2020-10-04T06:42:35.1261094Z     azure-arm: Unpacking clang-9 (1:9-2~ubuntu18.04.2) ...
2020-10-04T06:42:35.5078836Z     azure-arm: Selecting previously unselected package clangd-9.
2020-10-04T06:42:35.5194549Z     azure-arm: Preparing to unpack .../05-clangd-9_1%3a9-2~ubuntu18.04.2_amd64.deb ...
2020-10-04T06:42:35.5369091Z     azure-arm: Unpacking clangd-9 (1:9-2~ubuntu18.04.2) ...
2020-10-04T06:42:37.8062468Z     azure-arm: Selecting previously unselected package liblldb-9.
2020-10-04T06:42:37.8064839Z     azure-arm: Preparing to unpack .../06-liblldb-9_1%3a9-2~ubuntu18.04.2_amd64.deb ...
2020-10-04T06:42:37.8065732Z     azure-arm: Unpacking liblldb-9 (1:9-2~ubuntu18.04.2) ...
2020-10-04T06:42:38.9467163Z     azure-arm: Selecting previously unselected package libomp5-9:amd64.
2020-10-04T06:42:38.9578835Z     azure-arm: Preparing to unpack .../07-libomp5-9_1%3a9-2~ubuntu18.04.2_amd64.deb ...
2020-10-04T06:42:38.9759643Z     azure-arm: Unpacking libomp5-9:amd64 (1:9-2~ubuntu18.04.2) ...
2020-10-04T06:42:39.2061420Z     azure-arm: Selecting previously unselected package libomp-9-dev.
2020-10-04T06:42:39.2177900Z     azure-arm: Preparing to unpack .../08-libomp-9-dev_1%3a9-2~ubuntu18.04.2_amd64.deb ...
2020-10-04T06:42:39.2360821Z     azure-arm: Unpacking libomp-9-dev (1:9-2~ubuntu18.04.2) ...
2020-10-04T06:42:39.4226148Z     azure-arm: Selecting previously unselected package lld-9.
2020-10-04T06:42:39.4321259Z     azure-arm: Preparing to unpack .../09-lld-9_1%3a9-2~ubuntu18.04.2_amd64.deb ...
2020-10-04T06:42:39.4557644Z     azure-arm: Unpacking lld-9 (1:9-2~ubuntu18.04.2) ...
2020-10-04T06:42:39.8374943Z     azure-arm: Selecting previously unselected package llvm-9-runtime.
2020-10-04T06:42:39.8487293Z     azure-arm: Preparing to unpack .../10-llvm-9-runtime_1%3a9-2~ubuntu18.04.2_amd64.deb ...
2020-10-04T06:42:39.8684071Z     azure-arm: Unpacking llvm-9-runtime (1:9-2~ubuntu18.04.2) ...
2020-10-04T06:42:40.0912758Z     azure-arm: Selecting previously unselected package libpfm4:amd64.
2020-10-04T06:42:40.1031957Z     azure-arm: Preparing to unpack .../11-libpfm4_4.9.0-2_amd64.deb ...
2020-10-04T06:42:40.1252870Z     azure-arm: Unpacking libpfm4:amd64 (4.9.0-2) ...
2020-10-04T06:42:40.3331869Z     azure-arm: Selecting previously unselected package llvm-9.
2020-10-04T06:42:40.3469556Z     azure-arm: Preparing to unpack .../12-llvm-9_1%3a9-2~ubuntu18.04.2_amd64.deb ...
2020-10-04T06:42:40.3657829Z     azure-arm: Unpacking llvm-9 (1:9-2~ubuntu18.04.2) ...
2020-10-04T06:42:41.3131661Z     azure-arm: Selecting previously unselected package python-pygments.
2020-10-04T06:42:41.3250620Z     azure-arm: Preparing to unpack .../13-python-pygments_2.2.0+dfsg-1_all.deb ...
2020-10-04T06:42:41.3536304Z     azure-arm: Unpacking python-pygments (2.2.0+dfsg-1) ...
2020-10-04T06:42:41.6740665Z     azure-arm: Selecting previously unselected package python3-pygments.
2020-10-04T06:42:41.6877910Z     azure-arm: Preparing to unpack .../14-python3-pygments_2.2.0+dfsg-1_all.deb ...
2020-10-04T06:42:41.7072187Z     azure-arm: Unpacking python3-pygments (2.2.0+dfsg-1) ...
2020-10-04T06:42:42.0528350Z     azure-arm: Selecting previously unselected package llvm-9-tools.
2020-10-04T06:42:42.0599425Z     azure-arm: Preparing to unpack .../15-llvm-9-tools_1%3a9-2~ubuntu18.04.2_amd64.deb ...
2020-10-04T06:42:42.0844145Z     azure-arm: Unpacking llvm-9-tools (1:9-2~ubuntu18.04.2) ...
2020-10-04T06:42:42.4042358Z     azure-arm: Selecting previously unselected package llvm-9-dev.
2020-10-04T06:42:42.4151452Z     azure-arm: Preparing to unpack .../16-llvm-9-dev_1%3a9-2~ubuntu18.04.2_amd64.deb ...
2020-10-04T06:42:42.4365394Z     azure-arm: Unpacking llvm-9-dev (1:9-2~ubuntu18.04.2) ...
2020-10-04T06:42:49.8857840Z     azure-arm: Selecting previously unselected package python-lldb-9.
2020-10-04T06:42:49.8991587Z     azure-arm: Preparing to unpack .../17-python-lldb-9_1%3a9-2~ubuntu18.04.2_amd64.deb ...
2020-10-04T06:42:49.9166017Z     azure-arm: Unpacking python-lldb-9 (1:9-2~ubuntu18.04.2) ...
2020-10-04T06:42:50.0599908Z     azure-arm: Selecting previously unselected package lldb-9.
2020-10-04T06:42:50.0715658Z     azure-arm: Preparing to unpack .../18-lldb-9_1%3a9-2~ubuntu18.04.2_amd64.deb ...
2020-10-04T06:42:50.0903780Z     azure-arm: Unpacking lldb-9 (1:9-2~ubuntu18.04.2) ...
2020-10-04T06:42:51.4620089Z     azure-arm: Selecting previously unselected package python-chardet.
2020-10-04T06:42:51.4730350Z     azure-arm: Preparing to unpack .../19-python-chardet_3.0.4-1_all.deb ...
2020-10-04T06:42:51.4910955Z     azure-arm: Unpacking python-chardet (3.0.4-1) ...
2020-10-04T06:42:51.8810821Z     azure-arm: Setting up python-chardet (3.0.4-1) ...
2020-10-04T06:42:52.1563760Z     azure-arm: Setting up libomp5-9:amd64 (1:9-2~ubuntu18.04.2) ...
2020-10-04T06:42:52.2211995Z     azure-arm: Setting up libpfm4:amd64 (4.9.0-2) ...
2020-10-04T06:42:52.2875203Z     azure-arm: Setting up python-pygments (2.2.0+dfsg-1) ...
2020-10-04T06:42:52.7898885Z     azure-arm: Setting up libllvm9:amd64 (1:9-2~ubuntu18.04.2) ...
2020-10-04T06:42:52.8480711Z     azure-arm: Setting up lld-9 (1:9-2~ubuntu18.04.2) ...
2020-10-04T06:42:52.9221642Z     azure-arm: Setting up libomp-9-dev (1:9-2~ubuntu18.04.2) ...
2020-10-04T06:42:52.9960539Z     azure-arm: Setting up python3-pygments (2.2.0+dfsg-1) ...
2020-10-04T06:42:53.6046440Z     azure-arm: Setting up libclang-common-9-dev (1:9-2~ubuntu18.04.2) ...
2020-10-04T06:42:53.6628497Z     azure-arm: Setting up libclang1-9 (1:9-2~ubuntu18.04.2) ...
2020-10-04T06:42:53.7203210Z     azure-arm: Setting up libclang-cpp9 (1:9-2~ubuntu18.04.2) ...
2020-10-04T06:42:53.7859533Z     azure-arm: Setting up clangd-9 (1:9-2~ubuntu18.04.2) ...
2020-10-04T06:42:53.8440811Z     azure-arm: Setting up llvm-9-runtime (1:9-2~ubuntu18.04.2) ...
2020-10-04T06:42:53.9089195Z     azure-arm: Setting up liblldb-9 (1:9-2~ubuntu18.04.2) ...
2020-10-04T06:42:53.9644443Z     azure-arm: Setting up llvm-9-tools (1:9-2~ubuntu18.04.2) ...
2020-10-04T06:42:54.0242506Z     azure-arm: Setting up python-lldb-9 (1:9-2~ubuntu18.04.2) ...
2020-10-04T06:42:54.0845843Z     azure-arm: Setting up clang-9 (1:9-2~ubuntu18.04.2) ...
2020-10-04T06:42:54.1467317Z     azure-arm: Setting up llvm-9 (1:9-2~ubuntu18.04.2) ...
2020-10-04T06:42:54.2133536Z     azure-arm: Setting up llvm-9-dev (1:9-2~ubuntu18.04.2) ...
2020-10-04T06:42:54.2882809Z     azure-arm: Setting up lldb-9 (1:9-2~ubuntu18.04.2) ...
2020-10-04T06:42:54.3799694Z     azure-arm: Processing triggers for libc-bin (2.27-3ubuntu1.2) ...
2020-10-04T06:42:54.4532876Z     azure-arm: Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
2020-10-04T06:42:56.8180435Z     azure-arm: Reading package lists...
2020-10-04T06:42:57.0059390Z     azure-arm: Building dependency tree...
2020-10-04T06:42:57.0060032Z     azure-arm: Reading state information...
2020-10-04T06:42:57.1502734Z     azure-arm: The following packages were automatically installed and are no longer required:
2020-10-04T06:42:57.1503788Z     azure-arm:   bc dns-root-data dnsmasq-base ebtables grub-pc-bin landscape-common
2020-10-04T06:42:57.1504771Z     azure-arm:   liblldb-6.0 liblldb-8 liblxc-common liblxc1 libuv1 linux-headers-4.15.0-118
2020-10-04T06:42:57.1505551Z     azure-arm:   lxcfs lxd lxd-client python3-attr python3-automat python3-click
2020-10-04T06:42:57.1506309Z     azure-arm:   python3-colorama python3-constantly python3-hyperlink python3-incremental
2020-10-04T06:42:57.1507111Z     azure-arm:   python3-pam python3-pyasn1 python3-pyasn1-modules python3-service-identity
2020-10-04T06:42:57.1507930Z     azure-arm:   python3-twisted python3-twisted-bin python3-zope.interface uidmap xdelta3
2020-10-04T06:42:57.1508627Z     azure-arm: Use 'sudo apt autoremove' to remove them.
2020-10-04T06:42:57.1854791Z     azure-arm: The following NEW packages will be installed:
2020-10-04T06:42:57.1855630Z     azure-arm:   clang-format-9
2020-10-04T06:42:57.2109868Z     azure-arm: 0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
2020-10-04T06:42:57.2110740Z     azure-arm: Need to get 38.2 kB of archives.
2020-10-04T06:42:57.2113281Z     azure-arm: After this operation, 178 kB of additional disk space will be used.
2020-10-04T06:42:57.2139616Z     azure-arm: Get:1 http://azure.archive.ubuntu.com/ubuntu bionic-updates/universe amd64 clang-format-9 amd64 1:9-2~ubuntu18.04.2 [38.2 kB]
2020-10-04T06:42:57.5652868Z     azure-arm: Fetched 38.2 kB in 0s (2536 kB/s)
2020-10-04T06:42:57.6491378Z     azure-arm: Selecting previously unselected package clang-format-9.
2020-10-04T06:42:57.7158156Z     azure-arm: (Reading database ... 151188 files and directories currently installed.)
2020-10-04T06:42:57.7212845Z     azure-arm: Preparing to unpack .../clang-format-9_1%3a9-2~ubuntu18.04.2_amd64.deb ...
2020-10-04T06:42:57.7399344Z     azure-arm: Unpacking clang-format-9 (1:9-2~ubuntu18.04.2) ...
2020-10-04T06:42:57.9821172Z     azure-arm: Setting up clang-format-9 (1:9-2~ubuntu18.04.2) ...
2020-10-04T06:42:58.0385700Z     azure-arm: Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
2020-10-04T06:42:59.7825712Z     azure-arm: Testing to make sure that script performed as expected, and basic scenarios work
2020-10-04T06:42:59.7832142Z     azure-arm: /usr/bin/clang-9
2020-10-04T06:42:59.7832880Z     azure-arm: /usr/bin/clang++-9
2020-10-04T06:42:59.7833482Z     azure-arm: Documenting clang-9...
2020-10-04T06:42:59.8283326Z     azure-arm: - Clang 9 (9.0.0)
2020-10-04T06:42:59.8322896Z     azure-arm: update-alternatives: using /usr/bin/clang++-9 to provide /usr/bin/clang++ (clang++) in auto mode
2020-10-04T06:42:59.8661235Z     azure-arm: update-alternatives: using /usr/bin/clang-9 to provide /usr/bin/clang (clang) in auto mode
2020-10-04T06:42:59.8803837Z     azure-arm: update-alternatives: using /usr/bin/clang-format-9 to provide /usr/bin/clang-format (clang-format) in auto mode

Hope that helps.

Darleev commented 3 years ago

Hello @ausfestivus, Thank you for the detailed response, very appreciate it. Could you please provide a region and build machine configuration? We are looking forward to your reply.

ausfestivus commented 3 years ago

Region: australiaeast Machine: Standard_DS2_v2

Darleev commented 3 years ago

@ausfestivus @erphunranjbar we have disabled all possible concurrent updates on Ubuntu images and added apt retry logic in ubuntu images. Now, it is probably can fail with the same error on the cleanup.sh script, but we are working to fix this as well. Could you please try to build from the main branch and let us know a result? We are looking forward to your reply.

erphunranjbar commented 3 years ago

@Darleev sorry for not responding! I just got a chance to rerun our build and I still got an apt-lock issue:

==> azure-arm: Provisioning with shell script: /agent/_work/1/s/images/linux/scripts/installers/azure-cli.sh
==> azure-arm: export DEBIAN_FRONTEND=noninteractive
==> azure-arm: apt-get update
    azure-arm: Hit:1 http://ppa.launchpad.net/ansible/ansible/ubuntu bionic InRelease
    azure-arm: Hit:2 http://ppa.launchpad.net/apt-fast/stable/ubuntu bionic InRelease
    azure-arm: Hit:3 http://security.ubuntu.com/ubuntu bionic-security InRelease
    azure-arm: Hit:4 https://packages.microsoft.com/ubuntu/18.04/prod bionic InRelease
    azure-arm: Hit:5 http://azure.archive.ubuntu.com/ubuntu bionic InRelease
    azure-arm: Hit:6 http://azure.archive.ubuntu.com/ubuntu bionic-updates InRelease
    azure-arm: Hit:7 http://azure.archive.ubuntu.com/ubuntu bionic-backports InRelease
    azure-arm: Reading package lists...
==> azure-arm: apt-get install -y apt-transport-https lsb-release gnupg curl
==> azure-arm: E: Could not get lock /var/lib/dpkg/lock-frontend - open (11: Resource temporarily unavailable)
==> azure-arm: E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?

I am guessing it is from the https://aka.ms/InstallAzureCLIDeb script that gets ran

miketimofeev commented 3 years ago

@erphunranjbar is it from the latest main? We've just merged another helpful PR https://github.com/actions/virtual-environments/pull/1883, could you please pull the latest main one more time and give it another try?

ausfestivus commented 3 years ago

@Darleev I switched over to the main branch for a run on the weekend. The pipeline now gets stuck at the following point and never progresses. Suspect this is a different issue in main though.

Im running a build with the ubuntu18/20201026.1 tag now instead. Unsure if the fixes for this issue are in it tbh. Didnt have much time to dive in too deeply.

2020-11-08T22:17:57.8175840Z     azure-arm: Get:1 http://ppa.launchpad.net/apt-fast/stable/ubuntu bionic/main amd64 apt-fast all 1.9.10-1~ubuntu18.04.1 [35.4 kB]
2020-11-08T22:17:58.3511615Z ==> azure-arm: debconf: unable to initialize frontend: Dialog
2020-11-08T22:17:58.3519997Z ==> azure-arm: debconf: (Dialog frontend will not work on a dumb terminal, an emacs shell buffer, or without a controlling terminal.)
2020-11-08T22:17:58.3520868Z ==> azure-arm: debconf: falling back to frontend: Readline
2020-11-08T22:17:58.3589938Z ==> azure-arm: debconf: unable to initialize frontend: Readline
2020-11-08T22:17:58.3591809Z ==> azure-arm: debconf: (This frontend requires a controlling tty.)
2020-11-08T22:17:58.3593237Z ==> azure-arm: debconf: falling back to frontend: Teletype
2020-11-08T22:17:58.3627844Z ==> azure-arm: dpkg-preconfigure: unable to re-open stdin:
2020-11-08T22:17:58.3947460Z     azure-arm: Fetched 35.4 kB in 1s (40.6 kB/s)
2020-11-08T22:17:59.3481233Z     azure-arm: Selecting previously unselected package apt-fast.
2020-11-08T22:17:59.3809430Z     azure-arm: (Reading database ... 76729 files and directories currently installed.)
2020-11-08T22:17:59.3828800Z     azure-arm: Preparing to unpack .../apt-fast_1.9.10-1~ubuntu18.04.1_all.deb ...
2020-11-08T22:17:59.4209283Z     azure-arm: Unpacking apt-fast (1.9.10-1~ubuntu18.04.1) ...
2020-11-08T22:17:59.8548429Z     azure-arm: Setting up apt-fast (1.9.10-1~ubuntu18.04.1) ...
2020-11-08T22:18:00.0596183Z     azure-arm: debconf: unable to initialize frontend: Dialog
2020-11-08T22:18:00.0600533Z     azure-arm: debconf: (Dialog frontend will not work on a dumb terminal, an emacs shell buffer, or without a controlling terminal.)
2020-11-08T22:18:00.0604676Z     azure-arm: debconf: falling back to frontend: Readline
2020-11-08T22:18:00.1860792Z     azure-arm: Configuring apt-fast
2020-11-08T22:18:00.1872191Z     azure-arm: --------------------
2020-11-08T22:18:00.1874835Z     azure-arm:
2020-11-08T22:18:00.1877521Z     azure-arm:
2020-11-08T22:18:00.1880020Z     azure-arm:   1. apt-get  2. apt  3. aptitude
2020-11-08T22:18:00.1881086Z     azure-arm:
ausfestivus commented 3 years ago

Latest output from failed run on ubuntu18/20201026.1

2020-11-08T23:45:33.6075780Z     azure-arm: Running cmd: /usr/bin/python virtualenv.py --no-download --python /usr/bin/python /usr/local/aws
2020-11-08T23:45:33.6087827Z     azure-arm: Running cmd: /usr/local/aws/bin/pip install --no-binary :all: --no-cache-dir --no-index --find-links file://. setuptools_scm-3.3.3.tar.gz
2020-11-08T23:45:33.6089255Z     azure-arm: Running cmd: /usr/local/aws/bin/pip install --no-binary :all: --no-cache-dir --no-index --find-links file://. wheel-0.33.6.tar.gz
2020-11-08T23:45:33.6091723Z     azure-arm: Running cmd: /usr/local/aws/bin/pip install --no-binary :all: --no-build-isolation --no-cache-dir --no-index  --find-links file:///home/packer/awscli-bundle/packages awscli-1.18.173.tar.gz
2020-11-08T23:45:33.6096192Z     azure-arm: You can now run: /usr/local/bin/aws --version
2020-11-08T23:45:33.6101175Z     azure-arm:
2020-11-08T23:45:33.6108628Z     azure-arm: Note: AWS CLI version 2, the latest major version of the AWS CLI, is now stable and recommended for general use. For more information, see the AWS CLI version 2 installation instructions at: https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html
2020-11-08T23:45:33.6151351Z     azure-arm: Validate the installation
2020-11-08T23:45:33.6156186Z     azure-arm: /usr/local/bin/aws
2020-11-08T23:45:33.6251851Z ==> azure-arm:   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
2020-11-08T23:45:33.6252672Z ==> azure-arm:                                  Dload  Upload   Total   Spent    Left  Speed
2020-11-08T23:45:36.3619021Z ==> azure-arm: 100 4678k  100 4678k    0     0  1709k      0  0:00:02  0:00:02 --:--:-- 1708k
2020-11-08T23:45:36.3713133Z ==> azure-arm: dpkg: error: dpkg frontend is locked by another process
2020-11-08T23:45:36.3744831Z ==> azure-arm: Provisioning step had errors: Running the cleanup provisioner, if present...
btrx-rkazi commented 3 years ago

Even i m getting this issue. Please find the logs below.

amazon-ebs: output will be in this color.

==> amazon-ebs: Prevalidating any provided VPC information ==> amazon-ebs: Prevalidating AMI Name: eks-ubuntu1804-none-1605185371 amazon-ebs: Found Image ID: [removed ami id as i m using custom one] ==> amazon-ebs: Creating temporary keypair: packer_5fad2f5b-ccba-029d-2bdc-aee6908970ba ==> amazon-ebs: Creating temporary security group for this instance: packer_5fad2f5d-a821-afa3-f293-11e9d7f4721f ==> amazon-ebs: Authorizing access to port 22 from [0.0.0.0/0] in the temporary security groups... ==> amazon-ebs: Launching a source AWS instance... ==> amazon-ebs: Adding tags to source instance amazon-ebs: Adding tag: "Name": "packer-builder-eks-ubuntu1804-none-ami" amazon-ebs: Instance ID: i-0ff8ab8f2cb21ac30 ==> amazon-ebs: Waiting for instance (i-0ff8ab8f2cb21ac30) to become ready... ==> amazon-ebs: Using ssh communicator to connect: 18.232.63.45 ==> amazon-ebs: Waiting for SSH to become available... ==> amazon-ebs: Connected to SSH! ==> amazon-ebs: Provisioning with shell script: /tmp/packer-shell433829695 ==> amazon-ebs: Provisioning with shell script: /tmp/packer-shell619662087 amazon-ebs: NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT amazon-ebs: loop0 7:0 0 97.8M 1 loop /snap/core/10185 amazon-ebs: loop1 7:1 0 28.1M 1 loop /snap/amazon-ssm-agent/2012 amazon-ebs: loop2 7:2 0 96.6M 1 loop /snap/core/9804 amazon-ebs: nvme1n1 259:0 0 50G 0 disk amazon-ebs: nvme0n1 259:1 0 50G 0 disk amazon-ebs: └─nvme0n1p1 259:2 0 50G 0 part / ==> amazon-ebs: Provisioning with shell script: /tmp/packer-shell252683582 amazon-ebs: E: Could not get lock /var/lib/dpkg/lock-frontend - open (11: Resource temporarily unavailable) amazon-ebs: E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it? ==> amazon-ebs: Provisioning step had errors: Running the cleanup provisioner, if present... ==> amazon-ebs: Terminating the source AWS instance... ==> amazon-ebs: Cleaning up any extra volumes... ==> amazon-ebs: No volumes to clean up, skipping ==> amazon-ebs: Deleting temporary security group... ==> amazon-ebs: Deleting temporary keypair...

dbeattie71 commented 3 years ago

I'm getting this too with the ubuntu/20201115.1 release.

Darleev commented 3 years ago

Hello @erphunranjbar, We've added retry logic and some additional fixes for the issue. Could you please try to build ubuntu images one more time? We are looking forward to your reply.

Darleev commented 3 years ago

Hello, We have added all possible fixes for the issue from our side and made sure that the issue is not reproducing anymore for our ubuntu build. Please note, the issue can appear again due to specific environment configuration.

In case of any questions, do not hesitate to contact us.