Queries load when run manually but not via Docker Compose

[ghost]

Hi, I'm trying to get prometheus-sql working in my environment, and am running into a problem that only happens when running via Docker Compose. If I run manually, using the following commands:

docker run --net custom-network -d --name sqlagent dbhi/sql-agent
docker run --net custom-network -d --name prometheus-sql -p 8080:8080 -v /home//setup/mysql_exporter/queries.yml:/queries.yml --link sqlagent:sqlagent dbhi/prometheus-sql

I get the following, and everything works:

prometheus-sql starting up...
Load queries from file [queries.yml]
* Listening on 0.0.0.0:8080...
[num_products] 2018/10/11
Fetch took 5.532749ms
Creating num_products{}
Registering metric num_products{}

However, if I run it via Docker Compose, I get the following (note no creation of queries):

prometheus-sql starting up...
Load queries from file [queries.yml]
* Listening on 0.0.0.0:8080...

Here are some files that may help:

queries.yml:

- num_products:
    driver: mysql
    connection:
        host: REMOVED
        port: REMOVED
        user: REMOVED
        password: REMOVED
        database: REMOVED
    sql: >
        SELECT COUNT(1)
        FROM REMOVED
    interval: 5s

docker-compose.yml:

sqlagent:
    image: dbhi/sql-agent
    networks:
      - back-tier

  prometheussql:
    image: dbhi/prometheus-sql
    links:
      - sqlagent:sqlagent
    command:
      -service http://sqlagent:5000
      -queries /queries.yml
    volumes:
      - /home/REMOVED/setup/mysql_exporter/queries.yml:/queries.yml
    ports:
      - 8080:8080
    networks:
      - back-tier
    restart: always
    deploy:
      mode: global

Thanks in advance for any help you can offer!

[bruth]

Not sure if this would be the cause, but your volumes path has an extra forward slash in it? Also you specify -queries queries.yml, but should be -queries /queries.yml (unless you know the working directory).

[ghost]

Whoops, that was GitHub removing a redacted piece of text I put in, I have fixed it in the post now. I have also adjusted to -queries /queries.yml as per your suggestion, but I'm afraid it hasn't helped.

[ghost]

Well this is interesting... I inadvertently discovered that Docker Compose does start the rules correctly if you give the container a name... even if it's the same name it would have otherwise been given!

So, with this config:

  prometheussql:
#    container_name: setup_prometheussql_1
    image: dbhi/prometheus-sql
    links:
      - sqlagent:sqlagent
    command:
      -service http://sqlagent:5000
      -config /config.yml
      -queries /queries.yml
    volumes:
      - /home/REMOVED/setup/mysql_exporter/config.yml:/config.yml
      - /home/REMOVED/setup/mysql_exporter/queries2.yml:/queries.yml
    ports:
      - 8080:8080
    networks:
      - back-tier
    restart: always
    deploy:
      mode: global

I get no dice:

docker logs setup_prometheussql_1 2018/10/12 11:35:38 prometheus-sql starting up... 2018/10/12 11:35:38 Load config from file [/config.yml] 2018/10/12 11:35:38 Load queries from file [/queries.yml] 2018/10/12 11:35:38 * Listening on 0.0.0.0:8080...

And yet with this config (the only difference is the un-commented container name):

  prometheussql:
    container_name: setup_prometheussql_1
    image: dbhi/prometheus-sql
    links:
      - sqlagent:sqlagent
    command:
      -service http://sqlagent:5000
      -config /config.yml
      -queries /queries.yml
    volumes:
      - /home/REMOVED/setup/mysql_exporter/config.yml:/config.yml
      - /home/REMOVED/setup/mysql_exporter/queries2.yml:/queries.yml
    ports:
      - 8080:8080
    networks:
      - back-tier
    restart: always
    deploy:
      mode: global

It works:

docker logs setup_prometheussql_1 2018/10/12 11:35:11 prometheus-sql starting up... 2018/10/12 11:35:11 Load config from file [/config.yml] 2018/10/12 11:35:11 Load queries from file [/queries.yml] 2018/10/12 11:35:11 * Listening on 0.0.0.0:8080... Creating num_products{} Registering metric num_products{}

[bruth]

Interesting. I wonder if it was misinterpreted due to some kind of weird YAML parsing. In any case.. I am glad it is working for you!

[ghost]

That said, I'm still failing to establish a connection between the sqlagent and prometheus-sql containers:

docker logs prometheus-sql 2018/10/12 11:43:43 prometheus-sql starting up... 2018/10/12 11:43:43 Load config from file [/config.yml] 2018/10/12 11:43:43 Load queries from file [/queries.yml] 2018/10/12 11:43:43 * Listening on 0.0.0.0:8080... Creating num_products{} Registering metric num_products{} [num_products] 2018/10/12 11:43:48 Post http://sqlagent:5000: net/http: request canceled (Client.Timeout exceeded while awaiting headers) [num_products] 2018/10/12 11:43:48 Backing off for 1s

Here is the latest docker-compose.yml, as I'm conscious it has changed a little during the course of this issue:

  sqlagent:
    container_name: sqlagent
    image: dbhi/sql-agent
    ports:
      - 5000:5000
    networks:
      - back-tier

  prometheussql:
    container_name: prometheus-sql
    image: dbhi/prometheus-sql
    links:
      - sqlagent:sqlagent
    command:
      -service http://sqlagent:5000
      -config /config.yml
      -queries /queries.yml
    volumes:
      - /home/REMOVED/setup/mysql_exporter/config.yml:/config.yml
      - /home/REMOVED/setup/mysql_exporter/queries.yml:/queries.yml
    ports:
      - 8080:8080
    networks:
      - back-tier
    restart: always
    deploy:
      mode: global

[bruth]

The command: entry may be being parsed as an array rather than a multi-line. Try adding the > for multi-line.

    command: >
      -service http://sqlagent:5000
      -config /config.yml
      -queries /queries.yml

[ghost]

No change with that, still stating "Post http://sqlagent:5000: net/http: request canceled (Client.Timeout exceeded while awaiting headers)"

[bruth]

I am not really up on the latest set of docker-compose options and their nuances, so I am not sure what is causing this. I do use a much simpler compose file elsewhere and it works fine (like the one in the repo). But this is not directly a prometheus-sql issue. I will reopen for others to hopefully chime into.

[haxorof]

For others that have similar issues look at working example here: https://github.com/chop-dbhi/prometheus-sql/blob/master/examples/working_example/docker-compose.yml

Closing this issues since user is deleted