Docker: reuse the manageiq-appliance and/or manageiq-appliance-build repos

[Fryguy]

Avoid repeating the scripts in the manageiq repo by reusing a clone of the https://github.com/manageiq/manageiq-appliance and/or https://github.com/manageiq/manageiq-appliance-build repos

Once this gets fixed, I can change the automatic build on Docker Hub to automatically look at these other repos as a trigger on the manageiq build.

cc @bazulay @fbladilo

[bdunne]

Also, the current container is running in development mode, it should be running in production. Reusing the files in manageiq-appliance and manageiq-appliance-build should resolve that.

[fbladilo]

I'm actually working on both of these issues currently , here is some feedback :

As soon as I moved the appliance rootdir to /var/www/miq/vmdb, started cloning and utilizing the manageiq-appliance scripts in appliance mode, I noticed we were missing a good number of packages in our base centos7 docker image (so far this is the list) : initscripts, chrony, psmisc, lvm2, httpd, mod_ssl, mod_auth_kerb, mod_authnz_pam mod_intercept_form_submit, mod_lookup_identity

Also I added the pglogical repos, is working great now, thanks for the pointers :)

The good news is that I was able to stay close to kickstart procedure within the Dockerfile and re-utilize most of tools, I'm trying to keep differences as little as possible.

[fbladilo]

@Fryguy @bdunne One question, in the developer build we use docker_setup to create, migrate and seed the DB. Basically :
rake db:create rake db:migrate db:seed

How is this handled in appliance mode? I noticed appliance-initialize.sh sets DB access/pwd but I haven't found a migration/seed step yet.

[bdunne]

@fbladilo It is done through our appliance console cli. https://github.com/ManageIQ/manageiq-appliance-build/blob/df875e9bf58fbcc6a1cd0c313000be03de4832cd/kickstarts/partials/post/db_init.ks.erb#L7

[fbladilo]

@bdunne I implemented that step identical, for some reason the DB didn't appear to be ready when starting evmserverd, I will try again and report details.

[fbladilo]

@bdunne @Fryguy @bazulay

Here are the details of the issue I see , even when successfully run appliance-initialize.sh : [root@fbcf2eb8c61e /]# /bin/appliance-initialize.sh create encryption key configuring internal database Initialize postgresql starting Initialize postgresql complete Checking for connections to the database...

Create region starting Create region complete [root@fbcf2eb8c61e /]# /usr/bin/evmserver.sh start DEPRECATION WARNING: serve_static_files is deprecated and will be removed in Rails 5.1. Please use config.public_file_server.enabled = false instead. (called from block in <top (required)> at /var/www/miq/vmdb/config/environments/production.rb:13) Starting EVM... rake aborted! ActiveRecord::StatementInvalid: PG::UndefinedTable: ERROR: relation "miq_servers" does not exist LINE 7: WHERE a.attrelid = '"miq_servers"'::regclass ^ : SELECT a.attname, format_type(a.atttypid, a.atttypmod), pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod, (SELECT c.collname FROM pg_collation c, pg_type t WHERE c.oid = a.attcollation AND t.oid = a.atttypid AND a.attcollation <> t.typcollation) FROM pg_attribute a LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum WHERE a.attrelid = '"miq_servers"'::regclass AND a.attnum > 0 AND NOT a.attisdropped ORDER BY a.attnum /var/www/miq/vmdb/lib/extensions/ar_virtual.rb:100:in load_schema!' /var/www/miq/vmdb/app/models/miq_server.rb:562:inblock in class:MiqServer' /var/www/miq/vmdb/gems/pending/util/extensions/miq-module.rb:37:in call' /var/www/miq/vmdb/gems/pending/util/extensions/miq-module.rb:37:inblock (2 levels) in cache_with_timeout' /var/www/miq/vmdb/gems/pending/util/extensions/miq-module.rb:25:in block in cache_with_timeout' /var/www/miq/vmdb/app/models/miq_server/worker_management.rb:14:inkill_all_workers' /var/www/miq/vmdb/lib/tasks/evm_application.rb:21:in start' /var/www/miq/vmdb/lib/tasks/evm.rake:7:inblock (2 levels) in <top (required)>' PG::UndefinedTable: ERROR: relation "miq_servers" does not exist LINE 7: WHERE a.attrelid = '"miq_servers"'::regclass

[root@fbcf2eb8c61e /]# psql --list | grep vmdb vmdb_production | root | UTF8 | en_US.UTF-8 | en_US.UTF-8 |

It seems like the database is created but not migrated/seeded? Usually migrating takes around 2 minutes by itself, appliance-initialize.sh seems to come back in around 30s.

PS : If I manually rake db:migrate db:seed after appliance-initialize.sh, then evmserver comes online without issues.

Thanks in advance,

[fbladilo]

Guys, any comments on this one? Did I miss something? I can work around it but I want to avoid duplication as much as possible. @Fryguy @bdunne

[bdunne]

@fbladilo Do you have a PR? I'm not sure what is going wrong, it sounds like something is going wrong on creating or raking the db. If I can look at the diff or poke around in a container, I may be able to help troubleshoot.

[blomquisg]

@Fryguy who should own this issue?

[fbladilo]

Need review, #8064 switches the docker image from development to appliance mode and resolves #7798 and #7797 as well. @Fryguy @bazulay @bdunne