CI: Docker image for testing throws quite a few errors, no consistent base image

helje5 commented 6 years ago

Expected behavior

The docker build should run w/o errors. That doesn't increase trust in a test environment.
Also for a test environment, there should be an image in the first place. Otherwise you can guarantee reproducibility. (and it makes the setup slow as a minor inconvenience, but that somewhat matters for contributors who actually just want to contribute a single line fix). The current image pulls the latest versions of packages, which may very well get out of sync.

Actual behavior

Setting up the local image from the Dockerfile produces quite a few errors, looks as if mostly related to not having an interactive terminal. For example:

 ---> 53ee6818e23b
Step 4/39 : RUN dpkg-reconfigure locales
 ---> Running in a4728f63b96c
debconf: unable to initialize frontend: Dialog
debconf: (TERM is not set, so the dialog frontend is not usable.)
debconf: falling back to frontend: Readline
debconf: unable to initialize frontend: Readline
debconf: (This frontend requires a controlling tty.)
debconf: falling back to frontend: Teletype
Generating locales...
  en_US.ISO-8859-1... up-to-date

 Removing any system startup links for /etc/init.d/rsync ...
update-rc.d: warning: default stop runlevel arguments (0 1 6) do not match rsync Default-Stop values (none)
 Adding system startup for /etc/init.d/rsync ...
   /etc/rc0.d/K20rsync -> ../init.d/rsync
...
invoke-rc.d: policy-rc.d denied execution of restart.

There are quite a lot of those errors, maybe around 20-30 (showing up red in the macOS terminal).

Steps to reproduce

in a clean docker environment, call docker-compose -f docker/docker-compose.yaml up test

If possible, minimal yet complete reproducer code (or URL to code)

docker-compose -f docker/docker-compose.yaml up test

SwiftNIO version/commit hash

This is on the helje5:bugs/fix-issue-110 fork branch, hash b2920b6100b8ad76dfb65c8a6b4cd2088dfc20ae, but that shouldn't matter.

Swift & OS version (output of `swift --version && uname -a`)

Apple Swift version 4.0.3 (swiftlang-900.0.74.1 clang-900.0.39.2) Target: x86_64-apple-macosx10.9 Darwin ZeeMBP 17.4.0 Darwin Kernel Version 17.4.0: Sun Dec 17 09:19:54 PST 2017; root:xnu-4570.41.2~1/RELEASE_X86_64 x86_64

Docker: Version 17.12.0-ce-mac55 (23011), Channel: stable, d62ef8d1b0

ianpartridge commented 6 years ago

I think adding ARG DEBIAN_FRONTEND=noninteractive to the Dockerfile might be enough to fix the debconf messages.

helje5 commented 6 years ago

There are a few others, like the policy-rc.d permission thing above.

ianpartridge commented 6 years ago

Yes I saw them - just trying to help out with the debconf one :)

toffaletti commented 6 years ago

As of c8d198020717b8bbb4c3be4e47c06aab4085c7fc the image built by the Dockerfile doesn't seem to setup the environment correctly. I have to manually source /root/.profile.

~/C/swift-nio (master|…) $ docker run --security-opt seccomp:unconfined -v(pwd):/code -it --rm swiftnio
root@9108f799cf7b:/# env
HOSTNAME=9108f799cf7b
TERM=xterm
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lz=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.axa=00;36:*.oga=00;36:*.spx=00;36:*.xspf=00;36:
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PWD=/
SHLVL=1
HOME=/root
LESSOPEN=| /usr/bin/lesspipe %s
LESSCLOSE=/usr/bin/lesspipe %s %s
_=/usr/bin/env
root@9108f799cf7b:/# swift
bash: swift: command not found
root@9108f799cf7b:/# source /root/.profile 
root@9108f799cf7b:/# swift --version
Swift version 4.0.2 (swift-4.0.2-RELEASE)
Target: x86_64-unknown-linux-gnu

toffaletti commented 6 years ago

libcurl.so seems to be built / installed incorrectly as well:

root@9108f799cf7b:/code# swift test /root/.swift/usr/bin/swift-test: /usr/local/lib/libcurl.so.4: no version information available (required by /root/.swift/usr/bin/../lib/swift/linux/libFoundation.so)

tomerd commented 6 years ago

@toffaletti try docker run --security-opt seccomp:unconfined -v $(PWD):/code -w /code -it --rm swiftnio /bin/bash -l

helje5 commented 6 years ago

It may be crazy to ask in the context of how much other efforts NIO duplicates, but is there a specific reason why this doesn't just use the official Swift docker image as a base?

ianpartridge commented 6 years ago

AFAIK that image is not "official", but is created and maintained by members of the Swift OSS community. I agree that this is a wheel that shouldn't need to be reinvented though - although we at IBM are also guilty of this... https://github.com/IBM-Swift/swift-ubuntu-docker

helje5 commented 6 years ago

Yes, it is community maintained (kinda like Linux Swift Foundation), but AFAIK it is the official, Swift.org approved Swift Docker image.

Look, it even says "OFFICIAL REPOSITORY" 😁: https://hub.docker.com/_/swift/

It would be great if IBM would contribute to this, but this is out of scope for the issue here ;-) For the issue here it would be cool not do redo all the work and rather contribute to the mainline if there is a reason for it.

tomerd commented 6 years ago

@helje5 this is an interesting point and requires a separate discussion, do you mind carving out a separate issue to track it, so we can use this one to track your and @toffaletti original hygiene feedback, since it goes beyond which base image we use

helje5 commented 6 years ago

OK, here you go: Use official Swift Docker image #143

weissi commented 5 years ago

closing this for #143. That'll hopefully be a thing soon and we'll use the official one.

apple / swift-nio