changing port to 80 results in error

[cherub-i]

Setup

Docker image is connected to a macvlan, so it has its own IP. From log:

[INFO] Docker image version: 1.1.10
[INFO] Alpine Linux version: 3.6.2

Steps to reproduce problem

1. http://xxx.xxx.xxx.xxx:8083 brings me to the config screen
2. change port from 8083 to 80 and confirm

Result

• UI is not reachable
• docker logs calibre-web shows:

  ...
  [INFO] Launching Calibre-Web ...
  Error starting server: Permission denied: ('0.0.0.0', 80)

• docker container keeps restarting

Expected behaviour

• calibre web runs on port 80

If there is anything else I can provide to help fix this behaviour or to test the result - please let me know. Cheers! -bastian

[cherub-i]

Just realized that I was not running the latest version. I updated but the problem stays. Log, after having changed the port to 80 in the config dialogue:

[INFO] Docker image version: 1.2.3
[INFO] Alpine Linux version: 3.8.0
...
[INFO] Launching Calibre-Web ...
[2019-01-26 16:31:33,003] INFO in web: Starting Calibre Web...
[2019-01-26 16:31:33,188] INFO in server: Starting Gevent server
[2019-01-26 16:31:33,189] INFO in server: Unable to listen on '', trying on IPv4 only...
[2019-01-26 16:31:33,189] INFO in server: Error starting server: Permission denied: ('0.0.0.0', 80)
Error starting server: Permission denied: ('0.0.0.0', 80)

[Technosoft2000]

Hi @cherub-i

http://xxx.xxx.xxx.xxx:8083 brings me to the config screen change port from 8083 to 80 and confirm

That is the wrong usage - you shouldn't change the internal port of the application inside the container. If you want that your container and therefore the application is available at http://xxx.xxx.xxx.xxx:80 you have to set the correct port mapping when you create your container.

So for example instead of creating the container like that:

docker create --name=calibre-web --restart=always \
-v /volume1/books/calibre:/books \
-v /etc/localtime:/etc/localtime:ro \
-e PGID=65539 -e PUID=1029 \
-p 8083:8083 \
technosoft2000/calibre-web

you've to set your expected port via the port mapping option -p <outside-port>:<inside-port> and therefore like this -p 80:8083:

docker create --name=calibre-web --restart=always \
-v /volume1/books/calibre:/books \
-v /etc/localtime:/etc/localtime:ro \
-e PGID=65539 -e PUID=1029 \
-p 80:8083 \
technosoft2000/calibre-web

and if you don't use the port mapping option then by default the standard port 8083 is exposed as the reachable one.

See also at the Docker documentation https://runnable.com/docker/binding-docker-ports

Hope it helps :)

[cherub-i]

Hello, thanks for the quick answer. I am aware, that I can map ports for a docker container. As far as I understand, this is not possible when using a macvlan network - which is what I do.

A macvlan is a bit like giving a dedicated network interface with its own mac address (and IP) to one container and the container has full control over that interface.

Just to be sure, I added a port mapping to my docker run command, but the mapping has no effect, the application runs on the port defined inside the docker container - same as it does without the -p 80:8083

docker run \
 --name=calibre-web \
 --network=mymacvlan \
 --ip=192.168.2.192 \
 -p 80:8083 \
 --restart=always \
 -v /etc/localtime:/etc/localtime:ro \
 -v /etc/timezone:/etc/timezone:ro \
 -v /PATH:/books \
 -v /PATH:/calibre-web/config \
 -e USE_CONFIG_DIR=true \
 -e PUID=997 \
 -e PGID=997 \
 -d \
 technosoft2000/calibre-web

Edit: just read this blogpost which also states, that port mappings have no effect when using macvlan networks.

Reconfiguring the Calibre Web UI port to port numbers >1024 seems possible (I just tried 8084 and that works fine), so I believe Calibre is restriced to use the lower ports. For e.g. Grafana, there is a remark that you have to give capabilities to the runtime, to allow just that: http://docs.grafana.org/installation/configuration/#http-port

[Technosoft2000]

Hi @cherub-i ,

thanks for the info - never tried to use a MACVLAN and I wasn't aware that port mappings have no effect when using macvlan networks.

so I believe Calibre is restriced to use the lower ports.

I'll do some tests and check the source code when I've more time again.

[cherub-i]

I found a way to get the UI to work with port 80. I am not a Linux nor a Docker expert, though - so there may be easier ways or problems coming with my approach of which I am not aware.

Based on the assumption, that the Calibre web UI is missing the proper permissions to open up a server on port 80 and the knowledge, that the web UI is run via Python, I did the following:

$ docker exec -it calibre-web bash
bash-4.4# apk add --update libcap
bash-4.4# setcap 'cap_net_bind_service=+ep' /usr/bin/python2.7

Then I changed the port to 80 in the web UI, aaaaaand: web UI comes up on port 80! I did not check all functions, but I assume everything will work on port 80, now that the server could bind to that port. I must say, that based on the restricted amount of experience that I have, I am pretty happy to have found all this out and see it is working as I understood it should be :-)

From what I read, it is not really good (in terms of security) to give the 'cap_net_bind_service' capability to an interpreter (is it an interpreter?) like python (or sh, etc.), as now every script run by that interpreter can open up ports below 1024. An alternative I read about (but did not try) would be to use authbind and give general allowance for all programs to one or some ports below 1024.

Do think it would be a good change, to adapt the general docker image, to allow the web UI to run on these lower level ports? Is this something, you might do in the future?

From more reading today, I found out I could also create my own dockerfile based on your image and add the steps I did interactively in the container as part of the build process for my image. Do you think that would be a worthwhile approach? It should get me going without the need for you to change anything.

I'll do some tests and check the source code when I've more time again.

Don't feel rushed by me to do anything. I just had the time today to try out a few more things.

[Technosoft2000]

I've tested today the setcapapproach at my latest docker image which is based now on Alpine 3.9 and it fails Failed to set capabilities on file '/usr/bin/python2.7' (Not supported)

So I've to test authbind as next possibillity.

[Technosoft2000]

@cherub-i

could you please test authbind with the latest image - here is a tutorial how to use it: https://mutelight.org/authbind

Thanks for your help

[cherub-i]

@Technosoft2000

It seems like, authbind is not available on Alpine Linux:

bash-4.4# apk add authbind
ERROR: unsatisfiable constraints:
  authbind (missing):
    required by: world[authbind]

Also, not part of this package listing.

My approach using setcap still works for me. I use this dockerfile:

FROM technosoft2000/calibre-web

RUN apk --no-cache --no-progress add libcap \
&& setcap 'cap_net_bind_service=+ep' /usr/bin/python2.7

Which results in this output during build:

$ docker build -f ../calibre-web/dockerfile -t bastian/calibre-web3 .
Sending build context to Docker daemon  1.583kB
Step 1/2 : FROM technosoft2000/calibre-web
latest: Pulling from technosoft2000/calibre-web
6c40cc604d8e: Pull complete
91be124ab4e3: Pull complete
5a5cacc0b68a: Pull complete
389e2447745f: Pull complete
bd4cc9a3d64e: Pull complete
2d09f60bc251: Pull complete
fc5293980f94: Pull complete
551c3ffe79ee: Pull complete
a0806e35c24c: Pull complete
a61b0156d999: Pull complete
810c29a0b26c: Pull complete
826bf292b78c: Pull complete
e0933863de55: Pull complete
Digest: sha256:bd9e72769050fd8b8f7a32f1274b2c637e6796b376428d0996cbac6abb9634c9
Status: Downloaded newer image for technosoft2000/calibre-web:latest
 ---> 2de3f7a98925
Step 2/2 : RUN apk --no-cache --no-progress add libcap && setcap 'cap_net_bind_service=+ep' /usr/bin/python2.7
 ---> Running in fc1d7207de09
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/community/x86_64/APKINDEX.tar.gz
(1/1) Installing libcap (2.26-r0)
Executing busybox-1.29.3-r10.trigger
Executing glibc-bin-2.28-r0.trigger
OK: 295 MiB in 128 packages
Removing intermediate container fc1d7207de09
 ---> ee3ed8651eda
Successfully built ee3ed8651eda
Successfully tagged bastian/calibre-web3:latest

and the final image can be set to work on port 80 :-)

When did you try to set the capabilities? As part of the dockerfile? When you docker exec -it calibre-container bash can you install libcap and set the capabilities from there?

[Technosoft2000]

When you docker exec -it calibre-container bash can you install libcap and set the capabilities from there?

I tried this of course in my container and get the error too - here the test from now:

bash-4.4# apk --no-cache --no-progress add libcap             
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/community/x86_64/APKINDEX.tar.gz
OK: 295 MiB in 128 packages

bash-4.4# setcap 'cap_net_bind_service=+ep' /usr/bin/python2.7
Failed to set capabilities on file `/usr/bin/python2.7' (Not supported)
usage: setcap [-q] [-v] [-n <rootid>] (-r|-|<caps>) <filename> [ ... (-r|-|<capsN>) <filenameN> ]

 Note <filename> must be a regular (non-symlink) file.
bash-4.4# 

My environment is this one:

bash-4.4# env 
LC_ALL=C
LD_LIBRARY_PATH=/usr/lib:/opt/calibre/lib
LANG=en_US.UTF-8
APP_HOME=/calibre-web
HOSTNAME=67a71fe870d3
APP_REPO=https://github.com/janeczku/calibre-web.git
PKG_IMAGES_DEV=curl file fontconfig-dev freetype-dev ghostscript-dev lcms2-dev     libjpeg-turbo-dev libpng-dev libtool libwebp-dev perl-dev tiff-dev xz zlib-dev
PUID=1029
CALIBRE_PATH=/books
USE_CONFIG_DIR=true
VERSION=1.3.0
PGID=100
PWD=/calibre-web/app
HOME=/root
GOSU_VERSION=1.10
SET_CONTAINER_TIMEZONE=true
PKG_PYTHON=ca-certificates py-pip python py-libxml2 py-libxslt py-lxml libev
PKG_IMAGES=fontconfig freetype ghostscript lcms2 libjpeg-turbo libltdl libpng   libwebp libxml2 tiff zlib
PKG_DEV=make gcc g++ python-dev openssl-dev libffi-dev libxml2-dev libxslt-dev
PUSER=calibre
APP_NAME=Calibre-Web
TERM=xterm
CALIBRE_INSTALLER_SOURCE_CODE_URL=https://raw.githubusercontent.com/kovidgoyal/calibre/master/setup/linux-installer.py
PGROUP=calibre
SHLVL=1
LANGUAGE=en_US.UTF-8
AMAZON_KG_TAR=kindlegen_linux_2.6_i386_v2_9.tar.gz
PKG_BASE=bash tzdata git coreutils shadow tree
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/calibre
CONTAINER_TIMEZONE=Europe/Vienna
MAGICK_HOME=/usr
AMAZON_KG_URL=http://kindlegen.s3.amazonaws.com/kindlegen_linux_2.6_i386_v2_9.tar.gz
APP_BRANCH=master
_=/usr/bin/env
bash-4.4# 

So I really wonder now why it works on your machine :-o

[cherub-i]

Hi @Technosoft2000, this seems totally weird to me. Disclaimer: I am by far no Linux or docker expert, so anything below may be completely wrong 😄

I would like to find out, why setcap behaves differently for us both, in spite of using the exact same image. I can only assume, that the problem lies above the image layer - for me there is: docker / linux OS / the VM in which all that runs.

Googling, I found this article (especially this comment) which states a similar problem, where the cause for setcap not functioning lies with the storage driver used by docker (whatever the storage driver is or does - I have not yet taken the time to read into that).

So maybe it's interesting to compare the docker info with special regards to the storage driver for our systems. Mine is using the overlay2 storage driver. Is your setup using aufs by any chance?

$ docker info
Containers: 11
 Running: 10
 Paused: 0
 Stopped: 1
Images: 19
Server Version: 18.09.0
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: c4446665cb9c30056f4998ed953e6d4ff22c7c39
runc version: 4fc53a81fb7c994640722ac585fa9ca548971871
init version: fec3683
Security Options:
 apparmor
 seccomp
  Profile: default
Kernel Version: 4.4.0-138-generic
Operating System: Ubuntu 16.04.5 LTS
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 5.827GiB
Name: xxx
ID: 5VZZ:EGKJ:PWHB:HMQ4:Z6HD:CRB3:ZY6A:R3JX:7ZQ5:FVIX:WOUJ:RRSI
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false
Product License: Community Engine

WARNING: No swap limit support

[Technosoft2000]

Hi @cherub-i,

thanks for the additional info, btw. I don't see me as Linux or docker expert too ;)

I've compared your output with mine and I see that there differences, e.g. version, storage driver and so on. I think this comes through the underlaying host system which differs between us too - I use my Synology NAS DS918+

So here is my docker info:

Containers: 14
 Running: 8
 Paused: 0
 Stopped: 6
Images: 44
Server Version: 17.05.0-ce
Storage Driver: aufs
 Root Dir: /volume1/@docker/aufs
 Backing Filesystem: extfs
 Dirs: 173
 Dirperm1 Supported: true
Logging Driver: db
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: fd189da3e13a3ef3d6d9eb73c5cd4697b4536cdd (expected: 9048e5e50717ea4497b757314bad98ea3763c145)
runc version: a2d6e07aab95ff37fb63cf5dec3c40d29940194f (expected: 9c2d8d184e5da67c95d601382adf14862e4f2228)
init version: 7a83305 (expected: 949e6fa)
Security Options:
 apparmor
Kernel Version: 4.4.59+
Operating System: <unknown>
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 3.688GiB
Name: ---------
ID: KU6K:XIIF:3RBR:N2XO:7QJN:CO27:R2N2:WGNW:AZFZ:63X5:YNLA:ZNC6
Docker Root Dir: /volume1/@docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

WARNING: No kernel memory limit support
WARNING: No cpu cfs quota support
WARNING: No cpu cfs period support

I'll do some additional tests in future to see how I can integrate this feature into my image in a 'safe' way :)