search

canonical / paas-app-charmer

2 stars 2 forks source link

We try to run schema update before database is ready #10

Closed mthaddon closed 3 weeks ago

mthaddon commented 6 months ago

Bug Description

Currently we try to run the schema update before the database is ready. We do it on any call of restart, so it fails a bunch of times before succeeding. Also, the message is misleading, we report to the end user (via unit status:

database migration command {command} failed, will retry in next update-status

But this isn't true, it's not only on the next update status hook that this happens.

To Reproduce

repo-policy-compliance is a flask application built with paas-app-charmer

juju deploy repo-policy-compliance --channel=edge --config charm_token=xy --config github_token=foobar
juju deploy postgresql-k8s --trust
juju integrate postgresql-k8s repo-policy-compliance
juju deploy nginx-ingress-integrator --trust --config service-hostname="test.example.com" --config path-routes="/" --channel stable
juju integrate nginx-ingress-integrator repo-policy-compliance

Environment

Microk8s 1.28.7, Juju 3.1.8

Relevant log output

$ egrep ' ran |POSTGRESQL_DB|execute database migration command' debug.log 
unit-repo-policy-compliance-0: 17:50:37 INFO juju.worker.uniter.operation ran "install" hook (via hook dispatching script: dispatch)
unit-repo-policy-compliance-0: 17:50:38 INFO juju.worker.uniter.operation ran "ingress-relation-created" hook (via hook dispatching script: dispatch)
unit-repo-policy-compliance-0: 17:50:38 INFO juju.worker.uniter.operation ran "postgresql-relation-created" hook (via hook dispatching script: dispatch)
unit-repo-policy-compliance-0: 17:50:39 INFO juju.worker.uniter.operation ran "secret-storage-relation-created" hook (via hook dispatching script: dispatch)
unit-repo-policy-compliance-0: 17:50:39 INFO juju.worker.uniter.operation ran "leader-elected" hook (via hook dispatching script: dispatch)
unit-repo-policy-compliance-0: 17:50:40 INFO unit.repo-policy-compliance/0.juju-log execute database migration command: ['bash', '-eo', 'pipefail', 'migrate.sh']
    config.set_main_option("sqlalchemy.url", os.environ["POSTGRESQL_DB_CONNECT_STRING"])
KeyError: 'POSTGRESQL_DB_CONNECT_STRING'
unit-repo-policy-compliance-0: 17:50:40 INFO juju.worker.uniter.operation ran "config-changed" hook (via hook dispatching script: dispatch)
unit-repo-policy-compliance-0: 17:50:41 INFO juju.worker.uniter.operation ran "start" hook (via hook dispatching script: dispatch)
unit-repo-policy-compliance-0: 17:50:42 INFO unit.repo-policy-compliance/0.juju-log execute database migration command: ['bash', '-eo', 'pipefail', 'migrate.sh']
    config.set_main_option("sqlalchemy.url", os.environ["POSTGRESQL_DB_CONNECT_STRING"])
KeyError: 'POSTGRESQL_DB_CONNECT_STRING'
unit-repo-policy-compliance-0: 17:50:42 INFO juju.worker.uniter.operation ran "flask-app-pebble-ready" hook (via hook dispatching script: dispatch)
unit-repo-policy-compliance-0: 17:50:43 INFO unit.repo-policy-compliance/0.juju-log secret-storage:0: execute database migration command: ['bash', '-eo', 'pipefail', 'migrate.sh']
    config.set_main_option("sqlalchemy.url", os.environ["POSTGRESQL_DB_CONNECT_STRING"])
KeyError: 'POSTGRESQL_DB_CONNECT_STRING'
unit-repo-policy-compliance-0: 17:50:43 INFO juju.worker.uniter.operation ran "secret-storage-relation-changed" hook (via hook dispatching script: dispatch)
unit-nginx-ingress-integrator-0: 17:50:56 INFO juju.worker.uniter.operation ran "install" hook (via hook dispatching script: dispatch)
unit-postgresql-k8s-0: 17:50:56 INFO juju.worker.uniter.operation ran "install" hook (via hook dispatching script: dispatch)
unit-repo-policy-compliance-0: 17:50:56 INFO juju.worker.uniter.operation ran "ingress-relation-joined" hook (via hook dispatching script: dispatch)
unit-nginx-ingress-integrator-0: 17:50:56 INFO juju.worker.uniter.operation ran "nginx-peers-relation-created" hook (via hook dispatching script: dispatch)
unit-repo-policy-compliance-0: 17:50:57 INFO juju.worker.uniter.operation ran "ingress-relation-changed" hook (via hook dispatching script: dispatch)
unit-postgresql-k8s-0: 17:50:57 INFO juju.worker.uniter.operation ran "upgrade-relation-created" hook (via hook dispatching script: dispatch)
unit-repo-policy-compliance-0: 17:50:57 INFO juju.worker.uniter.operation ran "postgresql-relation-joined" hook (via hook dispatching script: dispatch)
unit-repo-policy-compliance-0: 17:50:58 INFO juju.worker.uniter.operation ran "postgresql-relation-changed" hook (via hook dispatching script: dispatch)
unit-postgresql-k8s-0: 17:50:58 INFO juju.worker.uniter.operation ran "database-peers-relation-created" hook (via hook dispatching script: dispatch)
unit-postgresql-k8s-0: 17:50:58 INFO juju.worker.uniter.operation ran "restart-relation-created" hook (via hook dispatching script: dispatch)
unit-postgresql-k8s-0: 17:50:59 INFO juju.worker.uniter.operation ran "database-relation-created" hook (via hook dispatching script: dispatch)
unit-postgresql-k8s-0: 17:51:00 INFO juju.worker.uniter.operation ran "leader-elected" hook (via hook dispatching script: dispatch)
unit-self-signed-certificates-0: 17:51:03 INFO juju.worker.uniter.operation ran "install" hook (via hook dispatching script: dispatch)
unit-postgresql-k8s-0: 17:51:03 INFO juju.worker.uniter.operation ran "postgresql-pebble-ready" hook (via hook dispatching script: dispatch)
unit-self-signed-certificates-0: 17:51:03 INFO juju.worker.uniter.operation ran "certificates-relation-created" hook (via hook dispatching script: dispatch)
unit-self-signed-certificates-0: 17:51:04 INFO juju.worker.uniter.operation ran "leader-elected" hook (via hook dispatching script: dispatch)
unit-postgresql-k8s-0: 17:51:04 INFO juju.worker.uniter.operation ran "pgdata-storage-attached" hook (via hook dispatching script: dispatch)
unit-self-signed-certificates-0: 17:51:04 INFO juju.worker.uniter.operation ran "config-changed" hook (via hook dispatching script: dispatch)
unit-self-signed-certificates-0: 17:51:05 INFO juju.worker.uniter.operation ran "start" hook (via hook dispatching script: dispatch)
unit-self-signed-certificates-0: 17:51:05 INFO juju.worker.uniter.operation ran "certificates-relation-joined" hook (via hook dispatching script: dispatch)
unit-self-signed-certificates-0: 17:51:06 INFO juju.worker.uniter.operation ran "certificates-relation-changed" hook (via hook dispatching script: dispatch)
unit-postgresql-k8s-0: 17:51:29 INFO juju.worker.uniter.operation ran "config-changed" hook (via hook dispatching script: dispatch)
unit-postgresql-k8s-0: 17:51:53 INFO juju.worker.uniter.operation ran "start" hook (via hook dispatching script: dispatch)
unit-postgresql-k8s-0: 17:52:25 INFO juju.worker.uniter.operation ran "upgrade-relation-changed" hook (via hook dispatching script: dispatch)
unit-postgresql-k8s-0: 17:52:26 INFO juju.worker.uniter.operation ran "database-relation-changed" hook (via hook dispatching script: dispatch)
unit-postgresql-k8s-0: 17:52:27 INFO juju.worker.uniter.operation ran "database-relation-joined" hook (via hook dispatching script: dispatch)
unit-repo-policy-compliance-0: 17:52:27 INFO unit.repo-policy-compliance/0.juju-log postgresql:4: execute database migration command: ['bash', '-eo', 'pipefail', 'migrate.sh']
unit-repo-policy-compliance-0: 17:52:29 INFO juju.worker.uniter.operation ran "postgresql-relation-changed" hook (via hook dispatching script: dispatch)
unit-postgresql-k8s-0: 17:52:38 INFO juju.worker.uniter.operation ran "database-peers-relation-changed" hook (via hook dispatching script: dispatch)
unit-postgresql-k8s-0: 17:52:39 INFO juju.worker.uniter.operation ran "database-relation-changed" hook (via hook dispatching script: dispatch)
unit-self-signed-certificates-0: 17:55:16 INFO juju.worker.uniter.operation ran "update-status" hook (via hook dispatching script: dispatch)
unit-postgresql-k8s-0: 17:55:23 INFO juju.worker.uniter.operation ran "update-status" hook (via hook dispatching script: dispatch)
unit-repo-policy-compliance-0: 17:55:50 INFO juju.worker.uniter.operation ran "update-status" hook (via hook dispatching script: dispatch)

Additional context

No response

javierdelapuente commented 3 months ago

Thanks for the issue @mthaddon.

In the latest version of paas-app-charmer, the charm can block on non optional integrations.

That is, if the postgresql relation in charmcraft.yaml is not optional (optional: false or not set), the charm will not try to run the migration where there in no integration with postegresq.

javierdelapuente commented 3 weeks ago

Closing this issue because: