search

canonical / pebble

Pebble is a lightweight Linux service manager with layered configuration and an HTTP API.
https://canonical-pebble.readthedocs-hosted.com/
GNU General Public License v3.0
149 stars 54 forks source link

"working-dir" in services part has no effect #258

Closed gatici closed 1 year ago

gatici commented 1 year ago

Hello, We are using working-dir in the services to set the current directory for service.

"services": { "nbi": { "override": "replace", "summary": "nbi service", "command": "python3 -m osm_nbi.nbi", "startup": "enabled", "user": "appuser", "group": "appuser", "working-dir": "/app/osm_nbi", "environment": {...} }

Here is the charm file. https://osm.etsi.org/gerrit/#/c/osm/devops/+/13704/1/installers/charm/osm-nbi/src/charm.py

We are using following rockcraft yaml to build the image.

https://osm.etsi.org/gerrit/#/c/osm/NBI/+/13702/1/rockcraft.yaml

However, when the service started it uses / directory as path.

ubuntu@osm-gulsum:~/devops/installers/charm/osm-nbi$ kubectl logs   nbi-0 -n osm  -c nbi 
2023-07-20T13:26:09.925Z [pebble] HTTP API server listening on ":38813".
2023-07-20T13:26:09.925Z [pebble] Started daemon.
2023-07-20T13:27:31.370Z [pebble] POST /v1/layers 1.501561ms 200
2023-07-20T13:27:31.380Z [pebble] POST /v1/services 7.783332ms 202
2023-07-20T13:27:31.387Z [pebble] Service "nbi" starting: sh -c "pwd; ls /var/lib/pebble/default/layers; cat /var/lib/pebble/default/layers/*; ls .; ls /app; ls /etc/osm"
2023-07-20T13:27:31.390Z [nbi] /
2023-07-20T13:27:31.396Z [nbi] cat: '/var/lib/pebble/default/layers/*': No such file or directory
2023-07-20T13:27:31.398Z [nbi] app
2023-07-20T13:27:31.398Z [nbi] bin
2023-07-20T13:27:31.398Z [nbi] boot
2023-07-20T13:27:31.398Z [nbi] charm
2023-07-20T13:27:31.398Z [nbi] dev
2023-07-20T13:27:31.398Z [nbi] etc
2023-07-20T13:27:31.398Z [nbi] home
2023-07-20T13:27:31.399Z [nbi] include
2023-07-20T13:27:31.399Z [nbi] lib
2023-07-20T13:27:31.399Z [nbi] lib32
2023-07-20T13:27:31.399Z [nbi] lib64
2023-07-20T13:27:31.399Z [nbi] libx32
2023-07-20T13:27:31.399Z [nbi] media
2023-07-20T13:27:31.399Z [nbi] mnt
2023-07-20T13:27:31.399Z [nbi] nbi.cfg
2023-07-20T13:27:31.399Z [nbi] opt
2023-07-20T13:27:31.399Z [nbi] proc
2023-07-20T13:27:31.399Z [nbi] root
2023-07-20T13:27:31.399Z [nbi] run
2023-07-20T13:27:31.399Z [nbi] sbin
2023-07-20T13:27:31.399Z [nbi] share
2023-07-20T13:27:31.399Z [nbi] srv
2023-07-20T13:27:31.399Z [nbi] sys
2023-07-20T13:27:31.399Z [nbi] tmp
2023-07-20T13:27:31.399Z [nbi] usr
2023-07-20T13:27:31.399Z [nbi] var
2023-07-20T13:27:31.401Z [nbi] log
2023-07-20T13:27:31.401Z [nbi] osm_nbi
2023-07-20T13:27:31.401Z [nbi] storage
2023-07-20T13:27:31.404Z [nbi] ls: cannot access '/etc/osm': No such file or directory
2023-07-20T13:27:31.424Z [pebble] GET /v1/changes/1/wait?timeout=4.000s 42.170815ms 200
2023-07-20T13:27:37.684Z [pebble] POST /v1/layers 1.162148ms 200
2023-07-20T13:27:37.697Z [pebble] POST /v1/services 10.683253ms 202

We saw from the output of pebble plan that "working-dir" is ignored.

ubuntu@osm-gulsum:~/devops/installers/charm/osm-nbi$ kubectl logs   nbi-0 -n osm  -c nbi 
2023-07-20T14:12:57.833Z [pebble] HTTP API server listening on ":38813".
2023-07-20T14:12:57.833Z [pebble] Started daemon.
2023-07-20T14:16:10.152Z [pebble] POST /v1/layers 2.190104ms 200
2023-07-20T14:16:10.162Z [pebble] POST /v1/services 9.042617ms 202
2023-07-20T14:16:10.170Z [pebble] Service "nbi" starting: sh -c "set -x; ls /var/lib/pebble/default/; pebble version; pebble plan"
2023-07-20T14:16:10.174Z [nbi] + ls /var/lib/pebble/default/
2023-07-20T14:16:10.180Z [nbi] layers
2023-07-20T14:16:10.180Z [nbi] + pebble version
2023-07-20T14:16:10.196Z [nbi] client  v1.2.0+git12.g00bcd1f
2023-07-20T14:16:10.196Z [nbi] server  unknown
2023-07-20T14:16:10.197Z [nbi] + pebble plan
2023-07-20T14:16:10.205Z [pebble] GET /v1/plan?format=yaml 953.195µs 200
2023-07-20T14:16:10.207Z [nbi] services:
2023-07-20T14:16:10.207Z [nbi]     nbi:
2023-07-20T14:16:10.207Z [nbi]         summary: nbi service
2023-07-20T14:16:10.207Z [nbi]         startup: enabled
2023-07-20T14:16:10.207Z [nbi]         override: replace
2023-07-20T14:16:10.207Z [nbi]         command: sh -c "set -x; ls /var/lib/pebble/default/; pebble version; pebble plan"
2023-07-20T14:16:10.207Z [nbi]         environment:
2023-07-20T14:16:10.207Z [nbi]             OSMNBI_AUTHENTICATION_AUTH_PORT: "5000"
2023-07-20T14:16:10.207Z [nbi]             OSMNBI_AUTHENTICATION_AUTH_URL: http://keystone:5000/v3
2023-07-20T14:16:10.207Z [nbi]             OSMNBI_AUTHENTICATION_BACKEND: keystone
2023-07-20T14:16:10.207Z [nbi]             OSMNBI_AUTHENTICATION_PROJECT_DOMAIN_NAME: default
2023-07-20T14:16:10.207Z [nbi]             OSMNBI_AUTHENTICATION_SERVICE_PASSWORD: 0ff622df13d3491f032bd15e2717e71c
2023-07-20T14:16:10.207Z [nbi]             OSMNBI_AUTHENTICATION_SERVICE_PROJECT: service
2023-07-20T14:16:10.207Z [nbi]             OSMNBI_AUTHENTICATION_SERVICE_USERNAME: nbi
2023-07-20T14:16:10.207Z [nbi]             OSMNBI_AUTHENTICATION_USER_DOMAIN_NAME: default
2023-07-20T14:16:10.207Z [nbi]             OSMNBI_DATABASE_COMMONKEY: osm
2023-07-20T14:16:10.207Z [nbi]             OSMNBI_DATABASE_DRIVER: mongo
2023-07-20T14:16:10.207Z [nbi]             OSMNBI_DATABASE_URI: mongodb://relation-62:YN3julKZOfNvbZcDVRrUXiHFR1E552R7@mongodb-0.mongodb-endpoints/osm?replicaSet=mongodb&authSource=admin
2023-07-20T14:16:10.207Z [nbi]             OSMNBI_LOG_LEVEL: INFO
2023-07-20T14:16:10.207Z [nbi]             OSMNBI_MESSAGE_DRIVER: kafka
2023-07-20T14:16:10.207Z [nbi]             OSMNBI_MESSAGE_HOST: kafka
2023-07-20T14:16:10.207Z [nbi]             OSMNBI_MESSAGE_PORT: "9092"
2023-07-20T14:16:10.207Z [nbi]             OSMNBI_PROMETHEUS_HOST: prometheus
2023-07-20T14:16:10.207Z [nbi]             OSMNBI_PROMETHEUS_PORT: "9090"
2023-07-20T14:16:10.207Z [nbi]             OSMNBI_SERVER_ENABLE_TEST: "false"
2023-07-20T14:16:10.207Z [nbi]             OSMNBI_SERVER_SSL_CERTIFICATE: ""
2023-07-20T14:16:10.207Z [nbi]             OSMNBI_SERVER_SSL_MODULE: ""
2023-07-20T14:16:10.207Z [nbi]             OSMNBI_SERVER_SSL_PASS_PHRASE: ""
2023-07-20T14:16:10.207Z [nbi]             OSMNBI_SERVER_SSL_PRIVATE_KEY: ""
2023-07-20T14:16:10.207Z [nbi]             OSMNBI_STATIC_DIR: /app/osm_nbi/html_public
2023-07-20T14:16:10.207Z [nbi]             OSMNBI_STORAGE_COLLECTION: files
2023-07-20T14:16:10.207Z [nbi]             OSMNBI_STORAGE_DRIVER: mongo
2023-07-20T14:16:10.207Z [nbi]             OSMNBI_STORAGE_PATH: /app/storage
2023-07-20T14:16:10.207Z [nbi]             OSMNBI_STORAGE_URI: mongodb://relation-62:YN3julKZOfNvbZcDVRrUXiHFR1E552R7@mongodb-0.mongodb-endpoints/osm?replicaSet=mongodb&authSource=admin
2023-07-20T14:16:10.207Z [nbi]             OSMNBI_TEMPORAL_HOST: osm-temporal
2023-07-20T14:16:10.207Z [nbi]             OSMNBI_TEMPORAL_PORT: "7233"
2023-07-20T14:16:10.207Z [nbi]         user: appuser
2023-07-20T14:16:10.207Z [nbi]         group: appuser
gatici commented 1 year ago

We found that this is ignored because of the ops version which is < 2.2.
We need to update ops version.

gatici commented 1 year ago

The problem still exists with ops 2.4.1 (latest available version).

benhoyt commented 1 year ago

It looks like you're using Pebble in the context of Juju and ops, correct? If so, we only just updated the version of Pebble in Juju to the latest, but those versions of Juju haven't been released yet. Service working-dir will be available in Juju 3.1.6 and 3.2.2 (neither of which have been released yet).

Unfortunately older versions of Pebble don't return an error on unknown fields in the JSON API, so you don't get an error on older versions.

In the meantime (while waiting for the new versions of Juju), it's probably best to use a workaround, for example, make the command /bin/sh -c 'cd /app/osm_nbi && python3 -m osm_nbi.nbi'. That will be compatible with older versions of Juju/Pebble.

benhoyt commented 1 year ago

Confirmed @gatici is using an older 2.9.x version of Juju which doesn't have this feature.