passed environment variables seem to be ignored

randyrue commented 6 years ago

Bear with me if I'm missing something obvious, I'm pretty new to docker and very new to rundeck.

No matter what I do the command to launch the rundeck container seems to be ignoring -e flags passing things like EXTERNAL_SERVER_URL, RUNDECK_WITH_SSL and even RUNDECK_ADMIN_PASSWORD. I've tried both of your examples and many variations. In every case, for example,I've tried setting EXTERNAL_SERVER_URL to the both fqdn of my docker host and to its IP, the startup output tells me "Server URL set to https://localhost:4443" If I've launched docker with "-p 4443:4443" I can then load the rundeck login page on a browser on my docker host, but not from a browser on a remote machine, even though nmap on the remote machine shows a response on 4443 on the docker host. If I try "-p 4440:4440" as in your non-SSL example, from what I can tell the container is still running SSL/4443 internally and with the wrong port forwarding I can't reach it at all.

I might be having two issues, a) failing to get env variables passed to and heeded by the container, and b) exposing the container properly so it can be reached outside the docker host.

Any guidance would be greatly appreciated.

-r

jjethwa commented 6 years ago

Hi @randyrue

Are you trying to front Rundeck with something like HAProxy, nginx, or similar? If not, let's try a basic example that should hopefully work:

docker run -p 4440:4440 -e EXTERNAL_SERVER_URL=http://$(hostname):4440 --name rundeck -t jordan/rundeck:latest

That should allow you to connect to Rundeck using the hostname of the host server and port 4440 (HTTP). Let me know if that works for you 😄

randyrue commented 6 years ago

Thank you for your response.

I've tried a handful of variations on the below with no luck but tried again, pasting your syntax exactly.

The docker host didn't have a variable exported for $hostname so I set one. Also tried substituting hostname for your env variable call.

In every case, when I launch the container, the output describes the generation of an SSL cert and the import of a list of root cert authority pem files, and the end of the startup says it's launched an instance at https://localhost:4443. If I put -p 4443:4443 on the docker run line, I can then reach rundeck via SSL from a browser on the docker host only.

If I pass "-e SERVER_URL=http://`hostname`:4440" the stdout lists the correct URL, but still all the SSL info, suggesting the stdout might be still looking to the deprecated variable even though it's actually defaulting to the SSL instance?

If I use your syntax and then bash into the container, /etc/rundeck/rundeck-config.properties does show an entry:

grails.serverURL=http://marten:4440 (the name of the docker host)

docker container inspect shows both the desired and apparently running URLs:

rgrue@marten:~# docker container inspect rundeck [ { "Id": "b04887b0beb1502cf00763c46f2922578a8dd43e393cd17b077d89ade499e96a", "Created": "2018-01-22T17:33:00.885581398Z", "Path": "/opt/run", "Args": [], "State": { "Status": "running", "Running": true, "Paused": false, "Restarting": false, "OOMKilled": false, "Dead": false, "Pid": 18207, "ExitCode": 0, "Error": "", "StartedAt": "2018-01-22T17:33:04.651414382Z", "FinishedAt": "0001-01-01T00:00:00Z" }, "Image": "sha256:03e794691c7816877f77a3c530006b6603ce7ed1498bcce7b9b9be3ae5f1ea0b", "ResolvConfPath": "/local_static/dockerd/containers/b04887b0beb1502cf00763c46f2922578a8dd43e393cd17b077d89ade499e96a/resolv.conf", "HostnamePath": "/local_static/dockerd/containers/b04887b0beb1502cf00763c46f2922578a8dd43e393cd17b077d89ade499e96a/hostname", "HostsPath": "/local_static/dockerd/containers/b04887b0beb1502cf00763c46f2922578a8dd43e393cd17b077d89ade499e96a/hosts", "LogPath": "/local_static/dockerd/containers/b04887b0beb1502cf00763c46f2922578a8dd43e393cd17b077d89ade499e96a/b04887b0beb1502cf00763c46f2922578a8dd43e393cd17b077d89ade499e96a-json.log", "Name": "/rundeck", "RestartCount": 0, "Driver": "devicemapper", "MountLabel": "", "ProcessLabel": "", "AppArmorProfile": "docker-default", "ExecIDs": [ "f6dc7ab5d411475d8ed2a3961711e4325c656e2d7f34da3213d738b99064667d" ], "HostConfig": { "Binds": null, "ContainerIDFile": "", "LogConfig": { "Type": "json-file", "Config": {} }, "NetworkMode": "default", "PortBindings": { "4440/tcp": [ { "HostIp": "", "HostPort": "4440" } ] }, "RestartPolicy": { "Name": "no", "MaximumRetryCount": 0 }, "AutoRemove": false, "VolumeDriver": "", "VolumesFrom": null, "CapAdd": null, "CapDrop": null, "Dns": [], "DnsOptions": [], "DnsSearch": [], "ExtraHosts": null, "GroupAdd": null, "IpcMode": "", "Cgroup": "", "Links": null, "OomScoreAdj": 0, "PidMode": "", "Privileged": false, "PublishAllPorts": false, "ReadonlyRootfs": false, "SecurityOpt": null, "UTSMode": "", "UsernsMode": "", "ShmSize": 67108864, "Runtime": "runc", "ConsoleSize": [ 0, 0 ], "Isolation": "", "CpuShares": 0, "Memory": 0, "NanoCpus": 0, "CgroupParent": "", "BlkioWeight": 0, "BlkioWeightDevice": null, "BlkioDeviceReadBps": null, "BlkioDeviceWriteBps": null, "BlkioDeviceReadIOps": null, "BlkioDeviceWriteIOps": null, "CpuPeriod": 0, "CpuQuota": 0, "CpuRealtimePeriod": 0, "CpuRealtimeRuntime": 0, "CpusetCpus": "", "CpusetMems": "", "Devices": [], "DeviceCgroupRules": null, "DiskQuota": 0, "KernelMemory": 0, "MemoryReservation": 0, "MemorySwap": 0, "MemorySwappiness": -1, "OomKillDisable": false, "PidsLimit": 0, "Ulimits": null, "CpuCount": 0, "CpuPercent": 0, "IOMaximumIOps": 0, "IOMaximumBandwidth": 0 }, "GraphDriver": { "Data": { "DeviceId": "55", "DeviceName": "docker-8:3-1610612800-93ec516f3cf394d708427dd0f3c917979dfab678b36454aadaf3859774914e18", "DeviceSize": "10737418240" }, "Name": "devicemapper" }, "Mounts": [ { "Type": "volume", "Name": "a3fc1e10ce5b421a2e38ba06e82f8a204cff85c1439f8c523b411be650552278", "Source": "/local_static/dockerd/volumes/a3fc1e10ce5b421a2e38ba06e82f8a204cff85c1439f8c523b411be650552278/_data", "Destination": "/var/lib/mysql", "Driver": "local", "Mode": "", "RW": true, "Propagation": "" }, { "Type": "volume", "Name": "9043498c93d648400b4b68c8e70f8c72df508d080c64960ee85dc8976595784d", "Source": "/local_static/dockerd/volumes/9043498c93d648400b4b68c8e70f8c72df508d080c64960ee85dc8976595784d/_data", "Destination": "/var/lib/rundeck", "Driver": "local", "Mode": "", "RW": true, "Propagation": "" }, { "Type": "volume", "Name": "ca4a347e8542f949f987d472cf23e21a87ede95fe66e4f0493f471360cc15b10", "Source": "/local_static/dockerd/volumes/ca4a347e8542f949f987d472cf23e21a87ede95fe66e4f0493f471360cc15b10/_data", "Destination": "/var/lib/rundeck/logs", "Driver": "local", "Mode": "", "RW": true, "Propagation": "" }, { "Type": "volume", "Name": "d81c3e69c691a776cc0a79e50a8977614d3cb8fe34c9506205ed2b73993d1b38", "Source": "/local_static/dockerd/volumes/d81c3e69c691a776cc0a79e50a8977614d3cb8fe34c9506205ed2b73993d1b38/_data", "Destination": "/var/lib/rundeck/var/storage", "Driver": "local", "Mode": "", "RW": true, "Propagation": "" }, { "Type": "volume", "Name": "bb5c0c784cba825021af8a2c0a806b3b2f5b13c92f35783f9aef6467c5c87e25", "Source": "/local_static/dockerd/volumes/bb5c0c784cba825021af8a2c0a806b3b2f5b13c92f35783f9aef6467c5c87e25/_data", "Destination": "/var/log/rundeck", "Driver": "local", "Mode": "", "RW": true, "Propagation": "" }, { "Type": "volume", "Name": "94fc2c38e524f664f1c2aa200123d46e6b597bf84d5d664c0a4c09a1f584d05e", "Source": "/local_static/dockerd/volumes/94fc2c38e524f664f1c2aa200123d46e6b597bf84d5d664c0a4c09a1f584d05e/_data", "Destination": "/var/rundeck", "Driver": "local", "Mode": "", "RW": true, "Propagation": "" }, { "Type": "volume", "Name": "38ec59132cf22a4cd8242a6826212117fa9849f025b80f75060bbadd503dcc38", "Source": "/local_static/dockerd/volumes/38ec59132cf22a4cd8242a6826212117fa9849f025b80f75060bbadd503dcc38/_data", "Destination": "/etc/rundeck", "Driver": "local", "Mode": "", "RW": true, "Propagation": "" }, { "Type": "volume", "Name": "e167af56eff50146372b4330bd6463120864665e726bd451e384d942db498722", "Source": "/local_static/dockerd/volumes/e167af56eff50146372b4330bd6463120864665e726bd451e384d942db498722/_data", "Destination": "/opt/rundeck-plugins", "Driver": "local", "Mode": "", "RW": true, "Propagation": "" } ], "Config": { "Hostname": "b04887b0beb1", "Domainname": "", "User": "", "AttachStdin": false, "AttachStdout": true, "AttachStderr": true, "ExposedPorts": { "4440/tcp": {}, "4443/tcp": {} }, "Tty": true, "OpenStdin": false, "StdinOnce": false, "Env": [ "EXTERNAL_SERVER_URL=http://marten:4440", "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", "SERVER_URL=https://localhost:4443", "RUNDECK_STORAGE_PROVIDER=file", "RUNDECK_PROJECT_STORAGE_TYPE=file", "NO_LOCAL_MYSQL=false", "LOGIN_MODULE=RDpropertyfilelogin", "JAAS_CONF_FILE=jaas-loginmodule.conf", "KEYSTORE_PASS=adminadmin", "TRUSTSTORE_PASS=adminadmin" ], "Cmd": null, "ArgsEscaped": true, "Image": "jordan/rundeck:latest", "Volumes": { "/etc/rundeck": {}, "/opt/rundeck-plugins": {}, "/var/lib/mysql": {}, "/var/lib/rundeck": {}, "/var/lib/rundeck/logs": {}, "/var/lib/rundeck/var/storage": {}, "/var/log/rundeck": {}, "/var/rundeck": {} }, "WorkingDir": "", "Entrypoint": [ "/opt/run" ], "OnBuild": null, "Labels": {} }, "NetworkSettings": { "Bridge": "", "SandboxID": "08797d3fccf084a7dc3bbac5b4c4be9a27a2a38efc02c4a990c4d0d1cb830cde", "HairpinMode": false, "LinkLocalIPv6Address": "", "LinkLocalIPv6PrefixLen": 0, "Ports": { "4440/tcp": [ { "HostIp": "0.0.0.0", "HostPort": "4440" } ], "4443/tcp": null }, "SandboxKey": "/var/run/docker/netns/08797d3fccf0", "SecondaryIPAddresses": null, "SecondaryIPv6Addresses": null, "EndpointID": "0a75ae18af2e9055fc6b53ba9d17c764acb30051192b2c4ba78eb6ff44885204", "Gateway": "172.17.0.1", "GlobalIPv6Address": "", "GlobalIPv6PrefixLen": 0, "IPAddress": "172.17.0.2", "IPPrefixLen": 16, "IPv6Gateway": "", "MacAddress": "02:42:ac:11:00:02", "Networks": { "bridge": { "IPAMConfig": null, "Links": null, "Aliases": null, "NetworkID": "a97bf906f27ff56328bef28dfa677ee1673831b19a39e6cc0631104c3c1040e0", "EndpointID": "0a75ae18af2e9055fc6b53ba9d17c764acb30051192b2c4ba78eb6ff44885204", "Gateway": "172.17.0.1", "IPAddress": "172.17.0.2", "IPPrefixLen": 16, "IPv6Gateway": "", "GlobalIPv6Address": "", "GlobalIPv6PrefixLen": 0, "MacAddress": "02:42:ac:11:00:02" } } } } ] rgrue@marten:~#

On 1/19/2018 4:11 PM, Jordan Jethwa wrote:

Hi @randyrue https://github.com/randyrue

Are you trying to front Rundeck with something like HAProxy, nginx, or similar? If not, let's try a basic example that should hopefully work:

|docker run -p 4440:4440 -e EXTERNAL_SERVER_URL=http://$(hostname):4440 --name rundeck -t jordan/rundeck:latest |

That should allow you to connect to Rundeck using the hostname of the host server and port 4440 (HTTP). Let me know if that works for you 😄

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/jjethwa/rundeck/issues/103#issuecomment-359124634, or mute the thread https://github.com/notifications/unsubscribe-auth/AHXuETA-_omQMykaoM-87cVnVuD3nlaDks5tMS83gaJpZM4RlMKV.

jjethwa commented 6 years ago

Hi @randyrue

The certificate output can be safely ignored. It's part of the initial startup if a keystore is not available (usually through the use of a volume). The output at the end that says "Server URL set to..." needs to be updated, it was not updated properly when SERVER_URL was deprecated. The latest image should fix that.

Can you provide the output of /var/log/rundeck/service.log and /var/log/rundeck/rundeck.log ?

randyrue commented 6 years ago

You can close this issue, after a few weeks of getting used to docker in general and your rundeck image in particular, I have a better grasp of which ENV variables your image passes along to rundeck and which I still need to manage in the rundeck-config.properties file directly.

As a way to manage the settings and content I want to be persistent, I'm mounting a local directory inside the container as /etc/rundeck, is this an acceptable way to do that? I also have a file full of other ENV variables that I'm passing with the docker run command.

jjethwa commented 6 years ago

Hi @randyrue

That's great news! Using a volume for /etc/rundeck is the best way to manage persisting the rundeck configs. You'll probably want /var/rundeck and /var/lib/mysql for the project configs and executions as well.

randyrue commented 6 years ago

Thank you for your help with this, I'm clear that these are more newbie config issues than any problem with the code.

I'm connecting to an external pgsql db for project information. What other information do I need to make sure is kept outside the container?

Bonus question, is there a way to somehow use existing SSH keys that are users' home drives? Those drives are already mounted to the host machine and inside the container, I'm hoping for a way to configure RD so that when user connects, they get their own keys by default.

jjethwa commented 6 years ago

Hi @randyrue NP at all! So you'll definitely want /etc/rundeck, /var/rundeck, /var/log/rundeck (for rundeck logs and execution logs), and /var/lib/rundeck/var/storage if you use the filestore for key storage instead of the database. Filestore is the default.

For the SSH key setup you're looking at, Rundeck's key storage is the best bet. Check out the section at the bottom: http://rundeck.org/docs/administration/key-storage.html

jjethwa / rundeck

passed environment variables seem to be ignored #103