Port mapping must be to the same port number

[chbrandt]

If Dachs is setup to run on port 8080 and the mapping on Docker is done to another port number not 8080 (8888, for example), navigation within the site will be compromised.

The server works just fine, but because dachs uses the port number specified on its config file (gavo.rc) to build its internal links, whenever the user navigates through the site the "wrong" port (right from Dachs' point-of-view, wrong from outside the container) the "wrong" port will be used.

This will raise a "not found" error; the port number is manually change in the address bar and the user can proceed...

[msdemlei]

Note that most of the links within DaCHS are relative (i.e., don't include [web]serverURL). Perhaps one could make a couple more relative, too, with some effort; but DaCHS needs to have an idea of how it can be accessed from the outside, as it puts out access URLs on various places (the info pages, registry records, etc).

So, I believe the only way to fix this is to control [web]serverPort from the docker script. [web]serverURL somehow needs to be user configurable either way, as there's no way any software can infer what a (world-visible) access URL of a running instance may turn out to be (reverse proxies, ...).

[chbrandt]

Hi @msdemlei, I see. Fair enough. I will see how to deal with this when the container is instanciated.

In the meantime, I'm playing with the ports (80 <-> 8080) and I have a couple of questions to clarify:

1. Consider the file /etc/gavo.rc. If I don't define bindAddress and serverPort, the server shuts down instantaneously:

root@docker:/# cat /etc/gavo.rc 
[web]
sitename: Dachs-on-Docker
root@docker:/#
root@docker:/# service dachs start
[ ok ] Starting VO server: dachs.
root@docker:/#

and the browser when http://localhost:8080/ says the "site can't be reached".

It works if I start the server in debug though, gavo serve debug (instead of service dachs start).

So, I understand that I have to define bindAddress and serverPort in a production (non-debug) environment, is that so? Sounds reasonable to me, but want to check out.

1. Consider now the following lines using the master/latest docker image. The port now is 80.

user@host$ docker run -d --name arihip chbrandt/dachs:data
user@host$ docker run --rm -it -p 80:80 --volumes-from arihip chbrandt/dachs:latest
root@docker:$ service postgresql start
root@docker:$ service dachs start
root@docker:$ gavo imp -M 1000 arihip/q
root@docker:$ gavo pub arihip/q
root@docker:$ service dachs reload

Now going to your (@host) browser and typing http://localhost you should see the "ARIHIP" entry on the frontpage. The link to ARihip's query (i.e, form) page is OK, but the link to Arihip's info page is pointing to http://localhost:8080/arihip/q/cone/info. Do you have any idea of what's going on? From where does the 8080 comes from?

Thanks.

[msdemlei]

On Sun, Oct 02, 2016 at 02:48:09PM -0700, Carlos Brandt wrote:

1. Consider the file /etc/gavo.rc. If I don't define bindAddress and serverPort, the server shuts down instantaneously:

root@docker:/# cat /etc/gavo.rc 
[web]
sitename: Dachs-on-Docker
root@docker:/#
root@docker:/# service dachs start
[ ok ] Starting VO server: dachs.
root@docker:/#

The init script of course detaches the service, so it's to be expected that the command returns immediately. If you couldn't reach the service: Are you sure you tried on the right box? The service literally binds to 127.0.0.1 and not any public IP of the box (but that's regardless of whether it detaches or not).

and the browser when http://localhost:8080/ says the "site can't be reached".

It works if I start the server in debug though, gavo serve debug (instead of service dachs start).

So, I understand that I have to define bindAddress and serverPort in a production (non-debug) environment, is that so? Sounds reasonable to me, but want to check out.

No, gavo serve debug and gavo serve start do pretty much the same thing. If it's really true that after the commands above the service doesn't run, I'd be interested in either the dockerfile that exhibits that behaviour or just the contents for /var/gavo/logs/server.stderr.

1. Consider now the following lines using the master/latest docker image. The port now is 80.

user@host$ docker run -d --name arihip chbrandt/dachs:data
user@host$ docker run --rm -it -p 80:80 --volumes-from arihip chbrandt/dachs:latest
root@docker:$ service postgresql start
root@docker:$ service dachs start
root@docker:$ gavo imp -M 1000 arihip/q
root@docker:$ gavo pub arihip/q
root@docker:$ service dachs reload

Now going to your (@host) browser and typing http://localhost you should see the "ARIHIP" entry on the frontpage. The link to ARihip's query (i.e, form) page is OK, but the link to Arihip's info page is pointing to http://localhost:8080/arihip/q/cone/info. Do you have any idea of what's going on? From where does the 8080 comes from?

Unfortunately, in today's internet, bindAddress and serverPort don't let DaCHS work out its "visible" URL reliably. Hence, whenever DaCHS (thinks it) must build an absolute URL for something, it uses an extra configuration setting, serverURL; cf. http://dachs-doc.readthedocs.io/en/latest/opguide.html#section-web

If it makes your life easier, I could add some magic that would infer serverURL's default from bindAddress and serverPort. I'm not wild about that, though, as it'd code uglification with, I suspect, limited use in almost all deployment scenarios, where bindAddress and serverPort have little relation to serverURL anyway.

   -- Markus

[chbrandt]

First of all, @msdemlei , I don't know how worth is it to understand/fix the issue below -- the ambiguous behaviour when running "production"/"debug". I am curious about it -- and I can imagine you also -- but since such behaviour is easily solved by correctly defining all the parameters (for instance, serverPort) in gavorc[web], please fill free to just ignore/close the issue (or I'll do) with a "wont fix" label. (I don't want to be the pain in the a--, and am actually asking myself why I got in here).

Anyway, follows the answer regarding the dockerfile/logfile:

No, gavo serve debug and gavo serve start do pretty much the same thing. If it's really true that after the commands above the service doesn't run, I'd be interested in either the dockerfile that exhibits that behaviour or just the contents for /var/gavo/logs/server.stderr.

The dockerfile is the one from the master branch. (Which generates the chbrandt/dachs:latest image)

Unfortunately, /var/gavo/logs/server.stderr is empty. Take a look to the screenshot below to see the exact steps I took till I find the empty logfile.

And then (right after the above up/down of the srevice), I ran dachs in debug mode and it worked. Also, I put the screenshot in the hope it can clarify any wrong doing.

it uses an extra configuration setting, serverURL; cf. http://dachs-doc.readthedocs.io/en/latest/opguide.html#section-web

Ooh.. Somehow I completly forgot it (sure I use it at vo.bsd.icranet.org). Sorry. My bad :-1:

[msdemlei]

Hi,

On Wed, Oct 05, 2016 at 04:45:29AM -0700, Carlos Brandt wrote:

First of all, @msdemlei , I don't know how worth is it to understand/fix the issue below -- the ambiguous behaviour when running "production"/"debug". I am curious about it -- and I can

If that's different, it's alarming to me, so I'd certainly like to understand what's going on.

Anyway, follows the answer regarding the dockerfile/logfile:

No, gavo serve debug and gavo serve start do pretty much the same thing. If it's really true that after the commands above the service doesn't run, I'd be interested in either the dockerfile that exhibits that behaviour or just the contents for /var/gavo/logs/server.stderr.

The dockerfile is the one from the master branch. (Which generates the chbrandt/dachs:latest image)

Unfortunately, /var/gavo/logs/server.stderr is empty. Take a look to the screenshot below to see the exact steps I took till I find the empty logfile.

Hm -- I'm afraid I cannot reproduce that behaviour. I ran

docker run -it -p 8080:80

(need the port mapping since port 80 in the host machine is already used, but that shouldn't matter, really), then within the container

service dachs start

Then, outside the container,

w3m http://localhost:8080

yielded the expected response (DaCHS complaining it cannot connect to a database).

Hm...

   -- Markus

[chbrandt]

Hm -- I'm afraid I cannot reproduce that behaviour. I ran

docker run -it -p 8080:80

(need the port mapping since port 80 in the host machine is already used, but that shouldn't matter, really), then within the container

service dachs start

OK. "The devil is" when you clean out the serverPort setting from /etc/gavo.rc.

Try removing it before service dachs start.

From within the container:

rm /etc/gavo.rc
service dachs start

If you succeed, then I'll dig into my Docker setup...

Also, I would ask you to docker run with both port mappings (8080 and 80) in place:

docker run -it -p 8080:8080 -p 8888:80 <your container>

because defaults (port 8080, right?) will apply when you remove the serverPort setting.

The exact commands I do:

@host      $ docker pull chbrandt/dachs:latest
@host      $ docker run --rm -it -p 8080:8080 -p 8888:80 chbrandt/dachs:latest
@container $ rm /etc/gavo.rc
@container $ service dachs start

And nothing comes out when I try localhost:8080 (neither localhost:8888).

Now, if you do the same, but in debug mode:

@host      $ docker pull chbrandt/dachs:latest
@host      $ docker run --rm -it -p 8080:8080 -p 8888:80 chbrandt/dachs:latest
@container $ rm /etc/gavo.rc
@container $ service postgresql start     # here we have to start Postgres, otherwise no dachs for us
@container $ gavo serve debug

And now when I do localhost:8080 I succeed!

If you don't see it happening, I will search for some cache-or-whatever on my docker/browser...

[msdemlei]

On Thu, Oct 06, 2016 at 05:11:32AM -0700, Carlos Brandt wrote:

Hm -- I'm afraid I cannot reproduce that behaviour. I ran

docker run -it -p 8080:80

(need the port mapping since port 80 in the host machine is already used, but that shouldn't matter, really), then within the container

service dachs start

OK. "The devil is" when you clean out the serverPort setting from /etc/gavo.rc.

Try removing it before service dachs start.

Ah! That would explain part of it, because then DaCHS binds do 127.0.0.1:8080; the magic statement was the bindAddress: in the gavo.rc, which let DaCHS bind to all interfaces, which in turn makes it visible to the Docker server.

From within the container:

rm /etc/gavo.rc
service dachs start

If you succeed, then I'll dig into my Docker setup...

To see that all is as it's supposed to be, in the container run

$ apt-get install curl $ curl http://localhost:8080

And nothing comes out when I try localhost:8080 (neither localhost:8888).

Within the container it would have worked (with 8080).

Now, if you do the same, but in debug mode:

@host      $ docker pull chbrandt/dachs:latest
@host      $ docker run --rm -it -p 8080:8080 -p 8888:80 chbrandt/dachs:latest
@container $ rm /etc/gavo.rc
@container $ service postgresql start     # here we have to start Postgres, otherwise no dachs for us
@container $ gavo serve debug

And now when I do localhost:8080 I succeed!

Whoa! That's a bug, indeed; gavo serve debug hadn't honoured the bindAddress setting. It does now (DaCHS rev. 5336). Which means you will, in that situation, no longer see the service from outside either. The minimal gavo.rc to serve to the outside of the container is

[general]
bindAddress:

Thanks for catching this.

     -- Markus