search

etalab / admin_api_entreprise

Site vitrine / backoffice de API Entreprise
https://entreprise.api.gouv.fr
MIT License
9 stars 5 forks source link

Échec de déploiement lorsque la base de données est vide #847

Closed am-beta closed 1 year ago

am-beta commented 1 year ago

Sur une base de données complètement vierge (aucune table et bien entendu aucune donnée), le déploiement de la branche develop échoue lors de l'étape de la migration.

Si avant le déploiement j'importe les données de la base depuis l'autre machine où la même application tourne, alors le déploiement se passe bien.

Même avec une base vierge, il ne devrait pas y avoir d'erreur au déploiement.

Voici les erreurs affichées :

-----> Migrating database
       == 20191001133751 MoveContactsRelationFromUsersToJwtAPIEntreprise: migrating ==
       -- rename_table(:contacts, :old_contacts)
          -> 0.0036s
       -- create_table(:contacts, {:id=>:uuid})
          -> 0.1365s
       rake aborted!
       StandardError: An error has occurred, this and all later migrations canceled:

       uninitialized constant MoveContactsRelationFromUsersToJwtAPIEntreprise::JwtAPIEntreprise
       /var/www/admin_apientreprise_sandbox/tmp/build-167283133329520/db/migrate/20191001133751_move_contacts_relation_from_users_to_jwt_api_entreprise.rb:28:in `link_contacts_to_jwt'
       /var/www/admin_apientreprise_sandbox/tmp/build-167283133329520/db/migrate/20191001133751_move_contacts_relation_from_users_to_jwt_api_entreprise.rb:17:in `change'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/migration.rb:870:in `exec_migration'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/migration.rb:854:in `block (2 levels) in migrate'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/migration.rb:853:in `block in migrate'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/connection_adapters/abstract/connection_pool.rb:215:in `with_connection'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/migration.rb:852:in `migrate'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/migration.rb:1046:in `migrate'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/migration.rb:1360:in `block in execute_migration_in_transaction'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/connection_adapters/abstract/transaction.rb:319:in `block in within_new_transaction'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.4/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.4/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.4/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.4/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/connection_adapters/abstract/transaction.rb:317:in `within_new_transaction'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/connection_adapters/abstract/database_statements.rb:316:in `transaction'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/transactions.rb:209:in `transaction'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/migration.rb:1411:in `ddl_transaction'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/migration.rb:1359:in `execute_migration_in_transaction'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/migration.rb:1333:in `each'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/migration.rb:1333:in `migrate_without_lock'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/migration.rb:1282:in `block in migrate'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/migration.rb:1432:in `block in with_advisory_lock'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/connection_adapters/abstract/connection_pool.rb:215:in `with_connection'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/migration.rb:1447:in `with_advisory_lock_connection'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/migration.rb:1428:in `with_advisory_lock'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/migration.rb:1282:in `migrate'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/migration.rb:1117:in `up'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/migration.rb:1092:in `migrate'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/tasks/database_tasks.rb:262:in `migrate'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/railties/databases.rake:92:in `block (2 levels) in <main>'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/sentry-ruby-5.7.0/lib/sentry/rake.rb:26:in `execute'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/rake-13.0.6/exe/rake:27:in `<top (required)>'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/bundler-2.3.10/lib/bundler/cli/exec.rb:58:in `load'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/bundler-2.3.10/lib/bundler/cli/exec.rb:58:in `kernel_load'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/bundler-2.3.10/lib/bundler/cli/exec.rb:23:in `run'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/bundler-2.3.10/lib/bundler/cli.rb:483:in `exec'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/bundler-2.3.10/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/bundler-2.3.10/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/bundler-2.3.10/lib/bundler/vendor/thor/lib/thor.rb:392:in `dispatch'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/bundler-2.3.10/lib/bundler/cli.rb:31:in `dispatch'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/bundler-2.3.10/lib/bundler/vendor/thor/lib/thor/base.rb:485:in `start'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/bundler-2.3.10/lib/bundler/cli.rb:25:in `start'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/bundler-2.3.10/exe/bundle:48:in `block in <top (required)>'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/bundler-2.3.10/lib/bundler/friendly_errors.rb:103:in `with_friendly_errors'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/bundler-2.3.10/exe/bundle:36:in `<top (required)>'
       /usr/local/rbenv/versions/3.2.0/bin/bundle:25:in `load'
       /usr/local/rbenv/versions/3.2.0/bin/bundle:25:in `<main>'

       Caused by:
       NameError: uninitialized constant MoveContactsRelationFromUsersToJwtAPIEntreprise::JwtAPIEntreprise
       /var/www/admin_apientreprise_sandbox/tmp/build-167283133329520/db/migrate/20191001133751_move_contacts_relation_from_users_to_jwt_api_entreprise.rb:28:in `link_contacts_to_jwt'
       /var/www/admin_apientreprise_sandbox/tmp/build-167283133329520/db/migrate/20191001133751_move_contacts_relation_from_users_to_jwt_api_entreprise.rb:17:in `change'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/migration.rb:870:in `exec_migration'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/migration.rb:854:in `block (2 levels) in migrate'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/migration.rb:853:in `block in migrate'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/connection_adapters/abstract/connection_pool.rb:215:in `with_connection'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/migration.rb:852:in `migrate'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/migration.rb:1046:in `migrate'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/migration.rb:1360:in `block in execute_migration_in_transaction'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/connection_adapters/abstract/transaction.rb:319:in `block in within_new_transaction'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.4/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.4/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.4/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.4/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/connection_adapters/abstract/transaction.rb:317:in `within_new_transaction'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/connection_adapters/abstract/database_statements.rb:316:in `transaction'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/transactions.rb:209:in `transaction'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/migration.rb:1411:in `ddl_transaction'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/migration.rb:1359:in `execute_migration_in_transaction'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/migration.rb:1333:in `each'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/migration.rb:1333:in `migrate_without_lock'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/migration.rb:1282:in `block in migrate'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/migration.rb:1432:in `block in with_advisory_lock'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/connection_adapters/abstract/connection_pool.rb:215:in `with_connection'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/migration.rb:1447:in `with_advisory_lock_connection'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/migration.rb:1428:in `with_advisory_lock'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/migration.rb:1282:in `migrate'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/migration.rb:1117:in `up'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/migration.rb:1092:in `migrate'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/tasks/database_tasks.rb:262:in `migrate'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.4/lib/active_record/railties/databases.rake:92:in `block (2 levels) in <main>'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/sentry-ruby-5.7.0/lib/sentry/rake.rb:26:in `execute'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/rake-13.0.6/exe/rake:27:in `<top (required)>'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/bundler-2.3.10/lib/bundler/cli/exec.rb:58:in `load'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/bundler-2.3.10/lib/bundler/cli/exec.rb:58:in `kernel_load'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/bundler-2.3.10/lib/bundler/cli/exec.rb:23:in `run'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/bundler-2.3.10/lib/bundler/cli.rb:483:in `exec'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/bundler-2.3.10/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/bundler-2.3.10/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/bundler-2.3.10/lib/bundler/vendor/thor/lib/thor.rb:392:in `dispatch'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/bundler-2.3.10/lib/bundler/cli.rb:31:in `dispatch'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/bundler-2.3.10/lib/bundler/vendor/thor/lib/thor/base.rb:485:in `start'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/bundler-2.3.10/lib/bundler/cli.rb:25:in `start'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/bundler-2.3.10/exe/bundle:48:in `block in <top (required)>'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/bundler-2.3.10/lib/bundler/friendly_errors.rb:103:in `with_friendly_errors'
       /var/www/admin_apientreprise_sandbox/shared/vendor/bundle/ruby/3.2.0/gems/bundler-2.3.10/exe/bundle:36:in `<top (required)>'
       /usr/local/rbenv/versions/3.2.0/bin/bundle:25:in `load'
       /usr/local/rbenv/versions/3.2.0/bin/bundle:25:in `<main>'
       Tasks: TOP => db:migrate
       (See full trace by running task with --trace)
 !     ERROR: Deploy failed.
skelz0r commented 1 year ago

Pour moi ce n'est pas vraiment un bug. Y'a aucune raison d'effectuer des migrations et/ou un schema load sur une app vierge, vu que ça sera écrasé no matter what.

Le fix ici est "trivial" : suffit de supprimer tous les fichiers de migration 🤷

am-beta commented 1 year ago

Sur une machine qui vient d'être installée, par exemple, il y aura un premier déploiement applicatif, et c'est ce déploiement qui est censé créer la structure de la base (Ansible ne crée pas les tables). S'il échoue, la seule façon de recréer la structure de la base est d'aller chercher sur l'autre machine ou depuis une sauvegarde. C'est faisable mais ça me semble bancal.

De mémoire, jusqu'à récemment j'ai toujours pu déployer l'application sur une installation vierge sans problème.

skelz0r commented 1 year ago

Je vois très bien le problème que tu évoques, cependant pour nous il n'existe pas : à aucun moment on repart d'une db sans avoir à charger une backup.

Le "fix" que je propose va fonctionner pour un fresh deployment, et ne changera rien pour les futures migrations/setup de machine de dev/dev ..

Les alternatives (qui me semblent useless d'un point de vue fonctionnel):

Pour le dernier point on pourrait à la limite surcharger le mina setup avec un rails db:schema:load.. mais quand bien même ça ne sert à rien car l'app ne serait pas fonctionnel ( vu qu'on a eu la "bonne" idée de mettre de la données en dur dans les tables 😅 ) (et d'ailleurs.. on utilise encore mina setup ?)

am-beta commented 1 year ago

Qu'est-ce qui soudainement rend cela un peu compliqué, alors qu'il me semble que depuis 2 ans ce problème ne s'est jamais posé, sachant que j'ai fait quand même un paquet de déploiements sur des installations vierges ?

Et oui j'utilise toujours mina setup, pareil c'est utile sur une installation fraiche. Ceci dit, ça doit être supprimable assez facilement en mettant les créations de répertoires dans Ansible, qui en a déjà un bon paquet mais il en manque sans doute quelques uns.

skelz0r commented 1 year ago

Qu'est-ce qui soudainement rend cela un peu compliqué, alors qu'il me semble que depuis 2 ans ce problème ne s'est jamais posé, sachant que j'ai fait quand même un paquet de déploiements sur des installations vierges ?

Ce n'est pas compliqué, faut juste supprimer les fichiers inutiles (i.e. les fichiers de migration) car ça n'apporte rien fonctionnellement.

skelz0r commented 1 year ago

(et ça ne marche plus maintenant car JwtAPIEntreprise a été renommé en Token, et ça fait maintenant 7 mois 258c9790)

skelz0r commented 1 year ago

(et renommer des constantes ça se fait souvent sur une code base, si on doit maintenir des migrations à chaque fois c'est vraiment perdre son temps car une migration c'est du code mort dès que celle-ci a été migré)

skelz0r commented 1 year ago

J'ai fait une PR dans ce sens: https://github.com/etalab/admin_api_entreprise/pull/848

A tester en sandbox, si c'est OK ça peut partir en "prod"

am-beta commented 1 year ago

Je viens de tester, le déploiement se passe bien sur une base vierge. Merci !

skelz0r commented 1 year ago

J'ai aussi testé (par acquis de conscience) sur une app up-to-date ça fonctionne.

Je close donc.