Closed carlcsaposs-canonical closed 1 year ago
carlcsaposs-canonical
commented
1 year ago Was able to replicate the issue when moving these commands from cloud-init to running with sudo as ubuntu user:
snap install juju --classic --channel=2.9/stable
snap install jhack
snap connect jhack:dot-local-share-juju snapdOn the same environment, installing via pip works so it appears to be snap related
PietroPasotti
commented
1 year ago Hi Carl, thanks for the report. I suspect it has something to do with some juju-envvars being unset in the python subprocess.
I normally don't develop in VMs so I'm not that familiar with the process.
If you run juju status from the console, does it show what's expected?
Could you pastebin me the output of listenv?
I see that the Popen I use already passes down the environment to the juju status process, but the question is if the snapped jhack has it in the first place.
carlcsaposs-canonical
commented
1 year ago juju status shows what's expected:
Model Controller Cloud/Region Version SLA Timestamp
foo micro microk8s/localhost 2.9.29 unsupported 13:46:27Z
Model "admin/foo" is empty.What is listenv (i.e. how do I run that)?
PietroPasotti
commented
1 year ago
juju statusshows what's expected:Model Controller Cloud/Region Version SLA Timestamp foo micro microk8s/localhost 2.9.29 unsupported 13:46:27Z Model "admin/foo" is empty.What is
listenv(i.e. how do I run that)?Here's the output of
printenvSHELL=/bin/bash PWD=/home/ubuntu LOGNAME=ubuntu XDG_SESSION_TYPE=tty MOTD_SHOWN=pam HOME=/home/ubuntu LANG=C.UTF-8 LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.webp=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: SSH_CONNECTION=[redacted] [redacted] 10.150.0.20 22 LESSCLOSE=/usr/bin/lesspipe %s %s XDG_SESSION_CLASS=user TERM=xterm-256color LESSOPEN=| /usr/bin/lesspipe %s USER=ubuntu SHLVL=1 XDG_SESSION_ID=7 XDG_RUNTIME_DIR=/run/user/1000 SSH_CLIENT=[redacted] 52072 22 XDG_DATA_DIRS=/usr/local/share:/usr/share:/var/lib/snapd/desktop PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus SSH_TTY=/dev/pts/0 _=/usr/bin/printenv
yeah sorry, printenv is what I meant. And I think I need the env as available to the snap -- something like snap run --shell jhack and then printenv
carlcsaposs-canonical
commented
1 year ago Here's the output of printenv after running snap run --shell jhack
ubuntu@dev-env-jhack-snap:~$ snap run --shell jhack
To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.
ubuntu@dev-env-jhack-snap:/home/ubuntu$ printenv
SHELL=/bin/bash
SNAP_REVISION=120
SNAP_REAL_HOME=/home/ubuntu
SNAP_USER_COMMON=/home/ubuntu/snap/jhack/common
SNAP_INSTANCE_KEY=
PWD=/home/ubuntu
LOGNAME=ubuntu
XDG_SESSION_TYPE=tty
TEMPDIR=/tmp
SNAP_CONTEXT=Rc6K8rbx5anfQPZ3R9PhsnOjKT9djFoxN1Ue8eGJeajMBy_VPDEm
MOTD_SHOWN=pam
HOME=/home/ubuntu/snap/jhack/120
LANG=C.UTF-8
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.webp=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:
SNAP_ARCH=amd64
SNAP_INSTANCE_NAME=jhack
SNAP_USER_DATA=/home/ubuntu/snap/jhack/120
TMPDIR=/tmp
SSH_CONNECTION=[redacted] [redacted] 10.150.0.20 22
SNAP_REEXEC=
LESSCLOSE=/usr/bin/lesspipe %s %s
XDG_SESSION_CLASS=user
TERM=xterm-256color
LESSOPEN=| /usr/bin/lesspipe %s
USER=ubuntu
SNAP=/snap/jhack/120
SNAP_COMMON=/var/snap/jhack/common
SNAP_VERSION=0.3.2
SHLVL=2
SNAP_LIBRARY_PATH=/var/lib/snapd/lib/gl:/var/lib/snapd/lib/gl32:/var/lib/snapd/void
SNAP_COOKIE=Rc6K8rbx5anfQPZ3R9PhsnOjKT9djFoxN1Ue8eGJeajMBy_VPDEm
XDG_SESSION_ID=3
SNAP_DATA=/var/snap/jhack/120
LD_LIBRARY_PATH=/var/lib/snapd/lib/gl:/var/lib/snapd/lib/gl32:/var/lib/snapd/void:/snap/jhack/120/lib:/snap/jhack/120/usr/lib
XDG_RUNTIME_DIR=/run/user/1000/snap.jhack
SSH_CLIENT=[redacted] [redacted] 22
SNAP_NAME=jhack
XDG_DATA_DIRS=/usr/local/share:/usr/share:/var/lib/snapd/desktop
PATH=/snap/jhack/120/usr/sbin:/snap/jhack/120/usr/bin:/snap/jhack/120/sbin:/snap/jhack/120/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
SSH_TTY=/dev/pts/0
_=/usr/bin/printenvdoes anything show up with LOGLEVEL=INFO jhack tail?
And if you try and run juju status from the snap (after snap run --shell jhack). Does it give some output?
carlcsaposs-canonical
commented
1 year ago LOGLEVEL=INFO jhack tail
::= Verbose mode (INFO). =::
INFO:jhack:jhack running in snapped mode. Configuring...
INFO:jhack:juju command is b'/snap/jhack/120/bin/juju\n'
INFO:jhack:Previous env JUJU_DATA = None.
INFO:jhack:Set JUJU_DATA to /home/ubuntu/.local/share/juju.
Traceback (most recent call last):
File "/snap/jhack/120/bin/jhack", line 8, in <module>
sys.exit(main())
File "/snap/jhack/120/lib/python3.8/site-packages/jhack/main.py", line 134, in main
app()
File "/snap/jhack/120/lib/python3.8/site-packages/typer/main.py", line 214, in __call__
return get_command(self)(*args, **kwargs)
File "/snap/jhack/120/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "/snap/jhack/120/lib/python3.8/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/snap/jhack/120/lib/python3.8/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/snap/jhack/120/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/snap/jhack/120/lib/python3.8/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/snap/jhack/120/lib/python3.8/site-packages/typer/main.py", line 500, in wrapper
return callback(**use_params) # type: ignore
File "/snap/jhack/120/lib/python3.8/site-packages/jhack/utils/tail_charms.py", line 917, in tail_events
return _tail_events(
File "/snap/jhack/120/lib/python3.8/site-packages/jhack/utils/tail_charms.py", line 972, in _tail_events
targets = parse_targets(targets) if not files else (targets or [])
File "/snap/jhack/120/lib/python3.8/site-packages/jhack/utils/tail_charms.py", line 104, in parse_targets
return get_all_units()
File "/snap/jhack/120/lib/python3.8/site-packages/jhack/utils/tail_charms.py", line 97, in get_all_units
chain(*(app.get("units", ()) for app in status["applications"].values()))
KeyError: 'applications'juju status after snap run --shell jhack
snap run --shell jhack
To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.
ubuntu@dev-env-jhack-snap:/home/ubuntu$ juju status
Model Controller Cloud/Region Version SLA Timestamp
foo micro microk8s/localhost 2.9.29 unsupported 13:18:15Z
Model "admin/foo" is empty.does it only happen when there are no applications (the status you posted shows an empty model), or also after you've deployed something? Maybe something changed in the juju status output. Can you share the output of
ubuntu@dev-env-jhack-snap:/home/ubuntu$ juju status --format=json?
carlcsaposs-canonical
commented
1 year ago It also happens after I deploy something
Output of juju status --format=json outside of the snap
{
"model": {
"name": "foo",
"type": "caas",
"controller": "micro",
"cloud": "microk8s",
"region": "localhost",
"version": "2.9.29",
"model-status": {
"current": "available",
"since": "03 Jan 2023 13:37:30Z"
},
"sla": "unsupported"
},
"machines": {},
"applications": {
"mysql-k8s": {
"charm": "mysql-k8s",
"series": "kubernetes",
"os": "kubernetes",
"charm-origin": "charmhub",
"charm-name": "mysql-k8s",
"charm-rev": 28,
"charm-channel": "edge",
"scale": 1,
"provider-id": "db3fcd29-0dee-4bee-a0af-3c5d2a79dc37",
"address": "10.152.183.72",
"exposed": false,
"application-status": {
"current": "waiting",
"message": "installing agent",
"since": "05 Jan 2023 15:06:33Z"
},
"relations": {
"database-peers": [
"mysql-k8s"
],
"restart": [
"mysql-k8s"
]
},
"units": {
"mysql-k8s/0": {
"workload-status": {
"current": "error",
"message": "hook failed: \"install\"",
"since": "05 Jan 2023 15:09:24Z"
},
"juju-status": {
"current": "idle",
"since": "05 Jan 2023 15:09:24Z",
"version": "2.9.29"
},
"leader": true,
"address": "10.1.193.22",
"provider-id": "mysql-k8s-0"
}
},
"endpoint-bindings": {
"": "alpha",
"certificates": "alpha",
"database": "alpha",
"database-peers": "alpha",
"mysql": "alpha",
"osm-mysql": "alpha",
"restart": "alpha"
}
}
},
"storage": {
"storage": {
"database/0": {
"kind": "filesystem",
"life": "alive",
"status": {
"current": "attached",
"message": "Successfully provisioned volume pvc-8fbae690-2722-4f71-9534-38182f6dd14c",
"since": "05 Jan 2023 15:06:42Z"
},
"persistent": false,
"attachments": {
"units": {
"mysql-k8s/0": {
"life": "alive"
}
}
}
}
},
"filesystems": {
"0": {
"provider-id": "8fbae690-2722-4f71-9534-38182f6dd14c",
"volume": "0",
"storage": "database/0",
"Attachments": {
"containers": {
"mysql-k8s/0": {
"mount-point": "/var/lib/juju/storage/database/0",
"read-only": false,
"life": "alive"
}
},
"units": {
"mysql-k8s/0": {
"life": "alive"
}
}
},
"pool": "kubernetes",
"size": 1024,
"life": "alive",
"status": {
"current": "attached",
"message": "Successfully provisioned volume pvc-8fbae690-2722-4f71-9534-38182f6dd14c",
"since": "05 Jan 2023 15:06:42Z"
}
}
},
"volumes": {
"0": {
"provider-id": "pvc-8fbae690-2722-4f71-9534-38182f6dd14c",
"storage": "database/0",
"attachments": {
"containers": {
"mysql-k8s/0": {
"read-only": false,
"life": "alive"
}
},
"units": {
"mysql-k8s/0": {
"life": "alive"
}
}
},
"pool": "kubernetes",
"size": 1024,
"persistent": false,
"life": "alive",
"status": {
"current": "attached",
"since": "05 Jan 2023 15:06:42Z"
}
}
}
},
"controller": {
"timestamp": "15:10:15Z"
}
}and inside of the snap shell
juju status --format=json
{}
ERROR juju client with major version 3 used with a controller having major version 2 not supported
update your juju client to match the version running on the controllerJuju version inside of the snap shell: 3.0.0-genericlinux-amd64
Juju version outside of the snap: 2.9.37-ubuntu-amd64
juju status without --format=json inside the snap shell works as expected
The juju agent version is 2.9.29
PietroPasotti
commented
1 year ago uh that's surprising. juju status works, but with --format=json gives {}?
carlcsaposs-canonical
commented
1 year ago uh that's surprising.
juju statusworks, but with--format=jsongives{}?
Correct
PietroPasotti
commented
1 year ago opened a broader issue addressing the root cause
Output from
jhack tail:I used
pdb.set_trace()here to check the values of: (by following this method)juju_status(json=True)was{}jhack.helpers.juju_version()was'3.0.0'jhack.helpers.juju_models()was''In my local environment:
juju versionwas2.9.37-ubuntu-amd64Copy-pasting code fromtail_charms.pyinto a python3 interpreter gave me:rawwas'2.9.37-ubuntu-amd64'raw.split("-")[0]was'2.9.37'I did run this command:
sudo snap connect jhack:dot-local-share-juju snapdI also tried refreshing the snap from
latest/stable: 0.3.2 2022-11-02 (120)tolatest/beta: 0.3.2 2022-11-22 (124)andlatest/edge: 0.3.2 2022-11-24 (126)with the same issue.Running on Google Compute Engine Ubuntu 22.04
GCE environment:
```json { "creationTimestamp": "2022-11-28T10:47:13.287-08:00", "description": "", "id": "3183457864939781342", "kind": "compute#instanceTemplate", "name": "dev-env", "properties": { "confidentialInstanceConfig": { "enableConfidentialCompute": false }, "description": "", "scheduling": { "onHostMaintenance": "MIGRATE", "provisioningModel": "STANDARD", "automaticRestart": true, "instanceTerminationAction": "STOP", "maxRunDuration": { "seconds": "36000", "nanos": 0 }, "preemptible": false }, "tags": {}, "disks": [ { "type": "PERSISTENT", "deviceName": "instance-template-1", "autoDelete": true, "index": 0, "boot": true, "kind": "compute#attachedDisk", "mode": "READ_WRITE", "initializeParams": { "sourceImage": "projects/ubuntu-os-cloud/global/images/ubuntu-2204-jammy-v20221123", "diskType": "pd-ssd", "diskSizeGb": "50" } } ], "networkInterfaces": [ { "stackType": "IPV4_ONLY", "name": "nic0", "network": "projects/dev-env-369519/global/networks/default", "accessConfigs": [ { "name": "External NAT", "type": "ONE_TO_ONE_NAT", "kind": "compute#accessConfig", "networkTier": "PREMIUM" } ], "kind": "compute#networkInterface" } ], "reservationAffinity": { "consumeReservationType": "ANY_RESERVATION" }, "canIpForward": false, "keyRevocationActionType": "NONE", "machineType": "n2-standard-4", "metadata": { "fingerprint": "rhj2YiywKp4=", "kind": "compute#metadata", "items": [ { "value": "#cloud-config\npackage_upgrade: true\npackages:\n - gnome-keyring\n - tox\nsnap:\n commands:\n - snap refresh\n - snap install juju --classic --channel=2.9/stable\n - snap install charmcraft --classic\n - snap install lxd\n - snap install microk8s --classic\n - snap alias microk8s.kubectl kubectl\n - snap install jhack\n - snap connect jhack:dot-local-share-juju snapd\nruncmd:\n - adduser ubuntu lxd\n - newgrp lxd\n - lxd init --auto\n - lxc network set lxdbr0 ipv6.address none\n - adduser ubuntu microk8s\n - newgrp microk8s\n - microk8s status --wait-ready\n - microk8s enable dns\n - kubectl rollout status -n kube-system -w --timeout=5m deployments/coredns\n - microk8s enable hostpath-storage\n - kubectl rollout status -n kube-system -w --timeout=5m deployments/hostpath-provisioner\n", "key": "user-data" }, { "value": "ubuntu:ssh-ed25519 [redacted] ubuntu", "key": "ssh-keys" } ] }, "shieldedVmConfig": { "enableSecureBoot": false, "enableVtpm": true, "enableIntegrityMonitoring": true }, "shieldedInstanceConfig": { "enableSecureBoot": false, "enableVtpm": true, "enableIntegrityMonitoring": true }, "displayDevice": { "enableDisplay": false } }, "selfLink": "projects/dev-env-369519/global/instanceTemplates/dev-env" } ```Cloud-init:
```yaml #cloud-config package_upgrade: true packages: - gnome-keyring - tox snap: commands: - snap refresh - snap install juju --classic --channel=2.9/stable - snap install charmcraft --classic - snap install lxd - snap install microk8s --classic - snap alias microk8s.kubectl kubectl - snap install jhack - snap connect jhack:dot-local-share-juju snapd runcmd: - adduser ubuntu lxd - newgrp lxd - lxd init --auto - lxc network set lxdbr0 ipv6.address none - adduser ubuntu microk8s - newgrp microk8s - microk8s status --wait-ready - microk8s enable dns - kubectl rollout status -n kube-system -w --timeout=5m deployments/coredns - microk8s enable hostpath-storage - kubectl rollout status -n kube-system -w --timeout=5m deployments/hostpath-provisioner ```Commands run as `ubuntu` user after cloud-init:
``` juju bootstrap microk8s micro --agent-version=2.9.29 juju bootstrap localhost lxd --agent-version=2.9.29 juju model-defaults logging-config='