search

charmed-kubernetes / actions-operator

Apache License 2.0
7 stars 17 forks source link

Setup fails on Ubuntu-22.04 #39

Closed jdkandersson closed 2 years ago

jdkandersson commented 2 years ago

Using this action:

  name: Integration tests
    runs-on: ubuntu-22.04
    steps:
      - name: Checkout
        uses: actions/checkout@v3
      - name: Setup operator environment
        uses: charmed-kubernetes/actions-operator@main
        with:
          provider: lxd

It gets stuck on cmd bootstrap.go:485 Running machine configuration script.... I have let it run for 30 minutes so far. On 22.04, the step finishes within a few minutes.

/usr/bin/sudo apt-get update -yqq
[43](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:43)
  /usr/bin/sudo apt-get install -yqq python3-pip
[44](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:44)
  /usr/bin/sudo --preserve-env=http_proxy,https_proxy,no_proxy pip3 install tox
[45](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:45)
  Collecting tox
[46](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:46)
    Downloading tox-3.25.1-py2.py3-none-any.whl (85 kB)
[47](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:47)
       ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 86.0/86.0 KB 2.8 MB/s eta 0:00:00
[48](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:48)
  Collecting filelock>=3.0.0
[49](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:49)
    Downloading filelock-3.8.0-py3-none-any.whl (10 kB)
[50](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:50)
  Collecting py>=1.4.17
[51](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:51)
    Downloading py-1.11.0-py2.py3-none-any.whl (98 kB)
[52](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:52)
       ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 98.7/98.7 KB 8.3 MB/s eta 0:00:00
[53](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:53)
  Requirement already satisfied: six>=1.14.0 in /usr/lib/python3/dist-packages (from tox) (1.16.0)
[54](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:54)
  Collecting toml>=0.9.4
[55](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:55)
    Downloading toml-0.10.2-py2.py3-none-any.whl (16 kB)
[56](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:56)
  Collecting virtualenv!=20.0.0,!=20.0.1,!=20.0.2,!=20.0.3,!=20.0.4,!=20.0.5,!=20.0.6,!=20.0.7,>=16.0.0
[57](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:57)
    Downloading virtualenv-20.16.3-py2.py3-none-any.whl (8.8 MB)
[58](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:58)
       ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.8/8.8 MB 29.3 MB/s eta 0:00:00
[59](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:59)
  Requirement already satisfied: packaging>=14 in /usr/local/lib/python3.10/dist-packages (from tox) (21.3)
[60](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:60)
  Collecting pluggy>=0.12.0
[61](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:61)
    Downloading pluggy-1.0.0-py2.py3-none-any.whl (13 kB)
[62](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:62)
  Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/lib/python3/dist-packages (from packaging>=14->tox) (2.4.7)
[63](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:63)
  Collecting platformdirs<3,>=2.4
[64](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:64)
    Downloading platformdirs-2.5.2-py3-none-any.whl (14 kB)
[65](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:65)
  Collecting distlib<1,>=0.3.5
[66](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:66)
    Downloading distlib-0.3.5-py2.py3-none-any.whl (466 kB)
[67](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:67)
       ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 467.0/467.0 KB 44.1 MB/s eta 0:00:00
[68](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:68)
  Installing collected packages: distlib, toml, py, pluggy, platformdirs, filelock, virtualenv, tox
[69](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:69)
  Successfully installed distlib-0.3.5 filelock-3.8.0 platformdirs-2.5.2 pluggy-1.0.0 py-1.11.0 toml-0.10.2 tox-3.25.1 virtualenv-20.16.3
[70](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:70)
  WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
[71](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:72)
Install Juju
[72](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:73)
  /usr/bin/sudo snap install juju --classic --channel=latest/stable
[73](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:74)
  juju 2.9.33 from Canonical** installed
[74](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:76)
Install tools
[75](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:77)
  /usr/bin/sudo snap install jq
[76](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:78)
  jq 1.5+dfsg-1 from Michael Vogt (mvo*) installed
[77](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:79)
  /usr/bin/sudo snap install charm --classic --channel=latest/stable
[78](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:80)
  charm 2.8.2 from Canonical** installed
[79](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:81)
  /usr/bin/sudo snap install charmcraft --classic --channel=latest/stable
[80](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:82)
  charmcraft 2.0.0 from Canonical** installed
[81](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:83)
  /usr/bin/sudo snap install juju-bundle --classic --channel=latest/stable
[82](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:84)
  juju-bundle 0.4.0 from Kenneth Koski (kennethkoski) installed
[83](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:85)
  /usr/bin/sudo snap install juju-crashdump --classic --channel=latest/stable
[84](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:86)
  juju-crashdump 1.0.2+git120.a838717 from Jason Hobbs installed
[85](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:88)
Bootstrap controller
[86](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:89)
  /usr/bin/sg lxd -c juju bootstrap --debug --verbose lxd github-pr-04a11 --model-default test-mode=true --model-default automatically-retry-hooks=false --model-default logging-config="<root>=DEBUG"  --bootstrap-constraints="cores=2 mem=4G"
[237](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:240)
   - Retrieving image: rootfs: 98% (35.19MB/s)
[238](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:241)
   - Retrieving image: rootfs: 99% (35.25MB/s)
[239](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:242)
   - Retrieving image: rootfs: 100% (35.31MB/s)
[240](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:243)

[241](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:244)
  08:52:40 DEBUG juju.service discovery.go:67 discovered init system "systemd" from series "focal"
[242](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:245)
  08:52:40 DEBUG juju.provider.lxd environ_broker.go:223 LXD user data; 3867 bytes
[243](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:246)
   - Creating container                        
[244](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:247)
  08:52:40 INFO  juju.container.lxd container.go:256 starting new container "juju-223c17-0" (image "ubuntu-20.04-server-cloudimg-amd64-lxd.tar.xz")
[245](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:248)
  08:52:40 DEBUG juju.container.lxd container.go:257 new container has profiles [default juju-controller]
[246](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:249)
  08:52:52 DEBUG juju.container.lxd container.go:286 created container "juju-223c17-0", waiting for start...
[247](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:250)
   - Container started 
[248](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:251)

[249](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:252)
  08:52:52 INFO  juju.provider.lxd environ_broker.go:48 started instance "juju-223c17-0"
[250](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:253)
  08:52:52 INFO  cmd bootstrap.go:305  - juju-223c17-0 (arch=amd64 mem=4G cores=2)
[251](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:254)
  08:52:52 INFO  juju.environs.bootstrap bootstrap.go:989 newest version: 2.9.33
[252](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:255)
  08:52:52 INFO  juju.environs.bootstrap bootstrap.go:1004 picked bootstrap agent binary version: 2.9.33
[253](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:256)
  08:52:52 INFO  cmd bootstrap.go:625 Installing Juju agent on bootstrap instance
[254](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:257)
  08:52:54 DEBUG juju.environs.simplestreams simplestreams.go:417 searching for signed metadata in datasource "gui simplestreams"
[255](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:258)
  08:52:54 DEBUG juju.environs.simplestreams simplestreams.go:452 looking for data index using path streams/v1/index2.sjson
[256](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:259)
  08:52:54 DEBUG juju.environs.simplestreams simplestreams.go:464 looking for data index using URL https://streams.canonical.com/juju/gui/streams/v1/index2.sjson
[257](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:260)
  08:52:54 DEBUG juju.environs.simplestreams simplestreams.go:467 streams/v1/index2.sjson not accessed, actual error: [{github.com/juju/juju/environs/simplestreams.(*urlDataSource).Fetch:192: "https://streams.canonical.com/juju/gui/streams/v1/index2.sjson" not found}]
[258](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:261)
  08:52:54 DEBUG juju.environs.simplestreams simplestreams.go:468 streams/v1/index2.sjson not accessed, trying legacy index path: streams/v1/index.sjson
[259](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:262)
  08:52:54 DEBUG juju.environs.simplestreams simplestreams.go:487 read metadata index at "https://streams.canonical.com/juju/gui/streams/v1/index.sjson"
[260](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:263)
  08:52:54 DEBUG juju.environs.simplestreams simplestreams.go:1019 finding products at path "streams/v1/com.canonical.streams-released-dashboard.sjson"
[261](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:264)
  08:52:54 INFO  cmd bootstrap.go:782 Fetching Juju Dashboard 0.8.1
[262](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:265)
  08:52:54 DEBUG juju.cloudconfig.instancecfg instancecfg.go:913 Setting numa ctl preference to false
[263](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:266)
  Waiting for address
[264](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:267)
  Attempting to connect to 10.53.219.164:22
[265](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:268)
  08:53:05 DEBUG juju.provider.common bootstrap.go:647 connection attempt for 10.53.219.164 failed: /var/lib/juju/nonce.txt does not exist
[266](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:269)
  08:53:11 DEBUG juju.provider.common bootstrap.go:647 connection attempt for 10.53.219.164 failed: /var/lib/juju/nonce.txt does not exist
[267](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:270)
  08:53:16 DEBUG juju.provider.common bootstrap.go:647 connection attempt for 10.53.219.164 failed: /var/lib/juju/nonce.txt does not exist
[268](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:271)
  08:53:22 DEBUG juju.provider.common bootstrap.go:647 connection attempt for 10.53.219.164 failed: /var/lib/juju/nonce.txt does not exist
[269](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:272)
  08:53:27 DEBUG juju.provider.common bootstrap.go:647 connection attempt for 10.53.219.164 failed: /var/lib/juju/nonce.txt does not exist
[270](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:273)
  08:53:32 DEBUG juju.provider.common bootstrap.go:647 connection attempt for 10.53.219.164 failed: /var/lib/juju/nonce.txt does not exist
[271](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:274)
  08:53:38 DEBUG juju.provider.common bootstrap.go:647 connection attempt for 10.53.219.164 failed: /var/lib/juju/nonce.txt does not exist
[272](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:275)
  08:53:43 DEBUG juju.provider.common bootstrap.go:647 connection attempt for 10.53.219.164 failed: /var/lib/juju/nonce.txt does not exist
[273](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:276)
  08:53:49 INFO  cmd bootstrap.go:415 Connected to 10.53.219.164
[274](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:277)
  08:53:49 INFO  juju.cloudconfig userdatacfg_unix.go:575 Fetching agent: curl -sSfw 'agent binaries from %{url_effective} downloaded: HTTP %{http_code}; time %{time_total}s; size %{size_download} bytes; speed %{speed_download} bytes/s ' --retry 10 -o $bin/tools.tar.gz <[https://streams.canonical.com/juju/tools/agent/2.9.33/juju-2.9.33-linux-amd64.tgz]>
[275](https://github.com/canonical/jenkins-agent-operator/runs/7969368204?check_suite_focus=true#step:3:278)
  08:53:49 INFO  cmd bootstrap.go:485 Running machine configuration script...
jnsgruk commented 2 years ago

@addyess I think I just ran into this, and I think it's because on 22.04 there is a clash between the iptables rules in Docker and LXD - which you've likely come across before.

I added a workaround in my CI pipeline here: https://github.com/jnsgruk/parca-k8s-operator/commit/891fa34848bf4a3db7675460f33e28895cc8787b

I think adding those iptables commands into the setup would likely solve this issue too?

jnsgruk commented 2 years ago

FTR, this is where I changed the base for the runner: https://github.com/jnsgruk/parca-k8s-operator/commit/132a1bd606b54a4f004e1acbab755d9f942ed1cc

addyess commented 2 years ago

@jnsgruk does this affect only the lxd provider? Or should it be adjusted for more than just lxd?

jnsgruk commented 2 years ago

So I was actually on MicroK8s -- mine was bailing when doing a charmcraft pack from my integration tests because the container it created couldn't access the internet. My suspicion is that's what is happening here

jnsgruk commented 2 years ago

(So tl;dr I think it should be a change for all providers right after all the tools are installed)

addyess commented 2 years ago

alright, let me see what i can do

addyess commented 2 years ago

@jdkandersson and @jnsgruk lemme know if this is resolved.

jnsgruk commented 2 years ago

Certainly seems to have fixed it in my case :)

jdkandersson commented 2 years ago

Yes it looks like it is working, thank you for the fix!