Unable to bake plugin into new build image

[cguentherTUChemnitz]

I tried to bake the sprint plugin into the image during docker build. My intention to do this is to keep the plugin and the phabricator version in sync. So that only a rebuild updates the phabricator and the plugin version. I scripted a very small example, based on the advanced configuration wiki section. This example can be found here: https://github.com/cguentherTUChemnitz/docker-phabricator-sprint/tree/plugin_baking

The problem is, that during docker build . the /bake /myScript.sh statement does not succeed. The tail of the generated output is:

+ pushd /srv/phabricator/phabricator
/srv/phabricator/phabricator /
+ '[' '' '!=' '' ']'
+ '[' '' '!=' '' ']'
+ '[' '' '!=' '' ']'
+ '[' '' '!=' '' ']'
+ '[' phabricator '!=' '' ']'
+ sudo -u git ./bin/config set storage.default-namespace phabricator
[2016-05-13 09:26:23] EXCEPTION: (FilesystemException) Requested path '/srv/phabricator/phabricator/conf/local/local.json' is not writable. at [<phutil>/src/filesystem/Filesystem.php:1081]
arcanist(head=master, ref.master=c58f1b9a2507), phabricator(head=master, ref.master=984dff0ae335), phutil(head=master, ref.master=b25e0477b280), sprint(head=master, ref.master=df6e9dee03e4)
  #0 Filesystem::assertWritable(string) called at [<phutil>/src/filesystem/Filesystem.php:69]
  #1 Filesystem::assertWritableFile(string) called at [<phutil>/src/filesystem/Filesystem.php:89]
  #2 Filesystem::writeFile(string, string) called at [<phabricator>/src/infrastructure/env/PhabricatorConfigLocalSource.php:41]
  #3 PhabricatorConfigLocalSource::saveConfig() called at [<phabricator>/src/infrastructure/env/PhabricatorConfigLocalSource.php:12]
  #4 PhabricatorConfigLocalSource::setKeys(array) called at [<phabricator>/src/applications/config/management/PhabricatorConfigManagementSetWorkflow.php:165]
  #5 PhabricatorConfigManagementSetWorkflow::execute(PhutilArgumentParser) called at [<phutil>/src/parser/argument/PhutilArgumentParser.php:410]
  #6 PhutilArgumentParser::parseWorkflowsFull(array) called at [<phutil>/src/parser/argument/PhutilArgumentParser.php:303]
  #7 PhutilArgumentParser::parseWorkflows(array) called at [<phabricator>/scripts/setup/manage_config.php:21]
The command '/bin/sh -c /myScript.sh' returned a non-zero code: 255

[hach-que]

Hmm, does it bake successfully with an empty script? (i.e. Make /myScript.sh do nothing)

[cguentherTUChemnitz]

When the script only contains the bake statement and nothing else, the error goes away.

+ sudo -u git ./bin/config set storage.default-namespace phabricator
/srv/phabricator/phabricator /
Set 'storage.default-namespace' in local configuration.
+ '[' '' '!=' '' ']'
+ '[' '' '!=' '' ']'
+ sudo -u git ./bin/config set phd.user git
Set 'phd.user' in local configuration.
+ sudo -u git ./bin/config set pygments.enabled true
Set 'pygments.enabled' in local configuration.
+ '[' '' '!=' '' ']'
+ '[' mysql == mysql ']'
+ sudo -u git ./bin/config set storage.mysql-engine.max-size 67108864
Set 'storage.mysql-engine.max-size' in local configuration.
+ '[' mysql == disk ']'
+ '[' mysql == s3 ']'
+ '[' '' '!=' '' ']'
+ '[' '' '!=' '' ']'
+ '[' none == none ']'
+ APHLICT_PROTOCOL=http
+ APHLICT_PORT=80
+ sudo -u git ./bin/config set notification.servers '[{"type": "client","host": "","port": 80,"protocol": "http","path": "/ws/"},{"type": "admin","host": "127.0.0.1","port": 22281,"protocol": "http"}]'
Set 'notification.servers' in local configuration.
+ unset APHLICT_PROTOCOL
+ unset APHLICT_PORT
+ popd
+ '[' '' == true ']'
+ '[' '' '!=' '' ']'
+ mysqlcheck --host= --port= --user= --password= --databases phabricator_search
/
mysqlcheck: Got error: 2002: Can't connect to local MySQL server through socket '/var/run/mysql/mysql.sock' (2 "No such file or directory") when trying to connect
+ true
+ /srv/phabricator/phabricator/bin/storage upgrade --force
MySQL Credentials Not Configured

Unable to connect to MySQL using the configured credentials. You must
configure standard credentials before you can upgrade storage. Run these
commands to set up credentials:

  phabricator/ $ ./bin/config set mysql.host __host__
  phabricator/ $ ./bin/config set mysql.user __username__
  phabricator/ $ ./bin/config set mysql.pass __password__

These standard credentials are separate from any administrative credentials
provided to this command with __--user__ or __--password__, and must be
configured correctly before you can proceed.

Raw MySQL Error: Attempt to connect to root@localhost failed with error
#2002: No such file or directory.

Afterwards mysqlcheck complains about the non accessible mysql server. Do i have to configure the sql server settings during baking?

[hach-que]

I just realised you uploaded the script. You have to clone the repo and set up settings with su git because once your bake script sets the config as root, it's no longer writable to git (probably because it's also the first thing set up).

In baking mode, you need to set up all the configuration environment variables you'd otherwise be passing to the image, because it will bake in all that configuration. So yeah, you need to provide MySQL connection information because all of the stuff that would normally happen on startup of a non-baked image (with the exclusion of operations that actually require external resources like the database schema upgrade) will happen during the bake.

[cguentherTUChemnitz]

Thanks so far. The su git does solve some problems. Now i am not able to connect to my mariadb instance. It seems that my mariadb instance is not reachable during the baking step. I wonder why, because other running containers are able to connect. Might this be caused of an missing link container statement?

I do export the following environment variables in the script, which i try to bake:

export MYSQL_LINKED_CONTAINER="mariadb101-phabricator"
export MYSQL_HOST="mariadb101-phabricator"
export MYSQL_USER="myMYSQLUser"
export MYSQL_PASS="myMYSQLPass"

[cguentherTUChemnitz]

I assume, that the baking step is not able to talk directly to exposed ports of other containers. So do i have to map the mysql port to the host, to be able to connect during baking?

[hach-que]

The thing is that you have this as a public GitHub repository - once this image is baked, those credentials are in the image and can't be changed by providing environment variables to the Docker container.

Are you intending to create a derived image that is usable by multiple people? In that case, you don't want to run the baking step, you probably just want to add an init script.

The baking step shouldn't be requiring any connection to the MySQL database (even though it is the step in which you configure the credentials). During baking, it shouldn't be performing an upgrade or anything like that. Can you provide the log that shows where it's attempting to connect to MySQL?

[cguentherTUChemnitz]

You are totally right. I don't want to push my baked image to github. What i want to achieve is to put an additional plugin into the phabricator image. The point, why i tried the baking is, that the plugin should be build against current git master of phabricator. So i need a mechanism to sync the plugin version with the phabricator version. Additionally i have the requirement to recreate a phabricator container without updating the phabricator. So i need some kind of phabricator and plugin snapshotting, baked into an image. When i understand correctly your proposed ways to extend your image i assume the way to go should be:

1. extend the phabricator image by an additional init script, which downloads and loads the plugin (this can be public on github)
2. snapshot the phab and plugin version by baking the extended image (with credentials) as non-public step

Is that correct? Is the baking able to consider init script previously added by extended images?

Can you provide the log that shows where it's attempting to connect to MySQL?

It is already part of the snipped, which i pasted above. Is this snipped sufficient?

[hach-que]

That's correct, you'll add a new init script like 25-sprint, which will clone and modify the load-libraries setting of Phabricator so that your extension is loaded at runtime. Then you should just build your image without calling the bake script.

[cguentherTUChemnitz]

You are right. Running the script during image creation is what i really want. Thanks for clarification.