Open nottrobin opened 4 years ago
Thank you for opening the issue @nottrobin ! Sorry for the headaches! Let me try to repro the condition to investigate this a bit more...
For what its worth, I am getting a similar issue trying to minify morpheusdata/morpheus-cli
(ruby). I even tried using --include-path=/etc/ssl
to no avail. It seems fine, but is unable to validate SSL certificates. However, it reports valid SSL certs as self signed.
Error Communicating with the Appliance. SSL_connect returned=1 errno=0 state=error: certificate verify failed (self signed certificate in certificate chain)
I was really glad to see someone else with an SSL error, cause this one might be hard to reproduce. :)
NOTE: I did "docker exec" into the container while it was sitting on the "Press
Please let me know if there's any way I could help dig into this further.
I am wondering if you are logging into your container while it is in "build" mode and running it thorough its different options or whatever. I am not sure exactly how it works, but I assume it is watching to see what files are opened by the running app. I am starting to believe that it doesn't work to "docker exec" into it :-/
@nottrobin Sorry, didn't get a chance to respond sooner (was traveling at the time). Some application stacks might require additional coverage when the http probe in docker-slim
runs. By default, the http probe sends a GET /
request to the target http interface. In your case that was not enough. The /blog
endpoint also needs to be covered, so your webapp gets a chance to load the cert file. To make the blog endpoint work I added an extra http probe command like this: docker-slim build --expose 80 --http-probe-cmd /blog ubuntu-com
. When docker-slim
is minifying the image you should now see something like this on you screen:
docker-slim[build]: info=http.probe.call status=200 method=GET target=http://127.0.0.1:32774/blog attempt=1 time=2020-01-21T15:58:29Z
docker-slim[build]: info=http.probe.call status=200 method=GET target=http://127.0.0.1:32774/ attempt=1 time=2020-01-21T15:58:29Z
If you have a lot of extra http probe commands or if you prefer using a config file you can also use the --http-probe-cmd-file
parameter. There are a few notes about http probing and how to specify additional http probe commands here (including an http probe command file example): https://github.com/docker-slim/docker-slim#http-probe-commands
I'll add an example to the examples
repo for your webapp container if you don't mind :)
@TJM , @nottrobin docker-slim
has several continue-after
modes. By default, when http probing is enabled (it's enabled when there's, at least, one exposed port) docker-slim
will run its http probes and then it'll finish its execution once the http probes finish, but you can also configure it to wait allowing you to run your own tests for your container (that would invoke various endpoints in your webapp): docker-slim build --continue-after enter your-image
. With enter
docker-slim
will wait for you to press enter to finish processing. With timeout
it'll wait the number of seconds you specify to wait. With signal
it'll wait for you or your automation tooling to send the USR1 signal to docker-slim
to continue.
One of the future enhancements will allow docker-slim
to discover the http routes in your application, so it can auto-generate additional http probe commands on its own. It does require stack-specific static analysis. Definitely doable, but need help to make it happen sooner :-)
@nottrobin added an example for your app image here: https://github.com/docker-slim/examples/tree/master/3rdparty/ubuntu-com
Note that I also needed to add a couple of extra directories with static web artifacts (/srv/templates
and /srv/static
) for the web app to be fully operational. Another alternative there is to have a more complete set of HTTP probes that would discover all of those static resources on their own.
@TJM I'm trying to repro your condition... and there's a WIP example for it here: https://github.com/docker-slim/examples/tree/master/3rdparty/morpheus-cli I'm not an expert with morpheus and its cli though. What would be a good set of morpheus cli commands to execute? We can put those cli commands in a shell script, mount running the docker-slim
build
command and set it as the entrypoint, so it gets to run when docker-slim
is inspecting the temporary container it creates.
What are the command line parameters you used trying to minify your morpheus-cli container image? This is what I used for my single command example: docker-slim build --http-probe=false --show-clogs=true --cmd="remote add demo https://demo.morpheusdata.com -N" morpheusdata/morpheus-cli
, which should be similar to doing this with docker: docker run -it --rm morpheusdata/morpheus-cli remote add demo https://demo.morpheusdata.com -N
P.S.
docker exec
doesn't help because it's executed as separate processes that are not in the process tree that's being observed.
@kcq If docker exec
won't work, then, for sure, my testing was defunct. As it is a command line tool that interacts with the API of an appliance, there are many different commands that could be run. I was trying to just do a remote add
and remote list -a
. I was actually bind mounting my local config file so that I didn't have to do the "add" or authentication part. I have asked the morpheus devs for help on a list of commands that we should be running instead of guessing. :)
At a minimum, I think we would need to add all the CA certs. If there is some way to detect when python/ruby/go/etc is opening CA certs, and load all of them, it would probably be a more complete solution.
Tommy
@TJM docker-slim
does keep the certs that get used. For example, with the current WIP example (that tries to execute remote add demo https://demo.morpheusdata.com -N
) it saves /usr/share/ca-certificates/mozilla/DigiCert_Global_Root_CA.crt
and the links that point to it in /etc/ssl/certs
.
@kcq Right, that works as expected. but this is a CLI tool that could be used to connect to any other morpheus appliance. That only will work if your appliance's cert was signed by that specific CA. However, if it is using the global system certs, perhaps it is as simple as ensuring everything in /etc/ssl is included?
~tommy
@TJM It's not always enough to include /etc/ssl
. In this case the actual certs are in another directory ( /usr/share/ca-certificates
) and /etc/ssl
includes only links. Try doing something like this: docker-slim build --http-probe=false --show-clogs=true --include-path /etc/ssl --include-path /usr/share/ca-certificates --cmd="remote add demo https://demo.morpheusdata.com -N" morpheusdata/morpheus-cli
Ah yes... its stuff like that where it feels like you still need intimate knowledge of the underlying container OS. The -N
is a syntax error, but without it, you get to the next barrier, which is that it cant load morpheus/cli/login.rb. I tried loading a known good config file and giving it the command "remote check -a" but then I get another error. I feel like you should at least be able to run the same command that you ran during the slimming, but alas. I am hoping the morpheus-cli team can provide a list of commands that we can use instead of just arbitrarily something like "remote add xxx url"
I know nothing about
docker-slim
, I don't have a clue how it works, but it looks exciting so I'm giving it a try on our Dockerfile for ubuntu.com.I build the original image as follows:
Then I run
docker-slim
, which appears to succed, and does indeed more than halve the size of the image:But now if I run the site from the new image:
Then I browse to http://127.0.0.1:8222/blog, the "blog" feed fails to load, and I see these errors in the image output:
The container is using requests to query a wordpress API at https://admin.insights.ubuntu.com/wp-json/wp/v2/posts, and it works in the original
ubuntu-com:latest
image. Something about the slimming process appears to be removing something the Requests library needs to verify HTTPS certificates, or something.