Timeout in restart setup step

jwgn commented 1 month ago

Hi,

I deployed LabCA using docker-compose.yml directly from the git repo in Portainer.

Please install the root certificate in the Trusted Root Certification Authorities store of your client machine now.
[Windows (.der) format](https://labca.home.arpa/certs/root-ca.der) | [Linux (.pem) format](https://labca.home.arpa/certs/root-ca.pem)

Then, [restart LabCA](javascript:void();)

After clicking on Restart, it runs into a timeout.

Docker logs from labca-bconsul-1:

2024-10-23T14:05:47.847Z [ERROR] agent.dns: all resolvers failed for question from client: question="{_grpc_config.localhost.home.arpa. 16 1}" client=10.55.55.55:35112 client_network=udp
2024-10-23T14:05:47.847Z [ERROR] agent.dns: all resolvers failed for question from client: question="{_grpc_config.localhost.home.arpa. 16 1}" client=10.55.55.55:44344 client_network=udp
2024-10-23T14:05:48.983Z [ERROR] agent.dns: all resolvers failed for question from client: question="{_grpc_config.localhost.home.arpa. 16 1}" client=10.55.55.55:37196 client_network=udp
2024-10-23T14:05:48.984Z [ERROR] agent.dns: all resolvers failed for question from client: question="{_grpc_config.localhost.home.arpa. 16 1}" client=10.55.55.55:55184 client_network=udp
2024-10-23T14:05:50.021Z [ERROR] agent.dns: all resolvers failed for question from client: question="{_grpc_config.localhost.home.arpa. 16 1}" client=10.55.55.55:38766 client_network=udp
2024-10-23T14:05:50.022Z [ERROR] agent.dns: all resolvers failed for question from client: question="{_grpc_config.localhost.home.arpa. 16 1}" client=10.55.55.55:48883 client_network=udp

I configured my local DNS 192.168.100.1 during GUI setup. Is it possible that the container is not able to access it as it's in it's own network?

Which more logs are needed and how to obtain them?

Logs from WebUI:

ACME Backend Log
ERROR! On line 114 in commander script

LabCA Log
ERROR! On line 133 in commander script

jwgn commented 3 weeks ago

Seems I am able to ping 192.168.100.1 from inside the container. So what might be the issue here?

hakwerk commented 3 weeks ago

I'm sorry but I have no idea why this is happening. It's using the standard hashicorp/consul image. Maybe there is some clue in the logs of the boulder container?

You could try stopping and removing the bconsul container and it's volume, and then recreating it.

jwgn commented 2 weeks ago

Ok, I started everything from scratch, this time I used the install script on a fresh container instead of my existing docker installation. First restart went through, now I am at

Almost there! Now we will request a certificate for this website and restart one more time...

boulder-1   | Wed Nov  6 15:51:09 UTC 2024 - still trying to connect to bpkilint:80
bconsul-1   | 2024-11-06T15:51:14.479Z [ERROR] agent.dns: all resolvers failed for question from client: question="{bpkilint.home.arpa. 28 1}" client=10.55.55.55:58149 client_network=udp
bconsul-1   | 2024-11-06T15:51:14.479Z [ERROR] agent.dns: all resolvers failed for question from client: question="{bpkilint.home.arpa. 1 1}" client=10.55.55.55:34723 client_network=udp
bconsul-1   | 2024-11-06T15:51:14.479Z [ERROR] agent.dns: all resolvers failed for question from client: question="{bpkilint.home.arpa. 28 1}" client=10.55.55.55:47779 client_network=udp
bconsul-1   | 2024-11-06T15:51:14.479Z [ERROR] agent.dns: all resolvers failed for question from client: question="{bpkilint.home.arpa. 1 1}" client=10.55.55.55:37612 client_network=udp
boulder-1   | Wed Nov  6 15:51:14 UTC 2024 - still trying to connect to bpkilint:80
bconsul-1   | 2024-11-06T15:51:19.482Z [ERROR] agent.dns: all resolvers failed for question from client: question="{bpkilint.home.arpa. 28 1}" client=10.55.55.55:43018 client_network=udp
bconsul-1   | 2024-11-06T15:51:19.482Z [ERROR] agent.dns: all resolvers failed for question from client: question="{bpkilint.home.arpa. 1 1}" client=10.55.55.55:34579 client_network=udp
bconsul-1   | 2024-11-06T15:51:19.483Z [ERROR] agent.dns: all resolvers failed for question from client: question="{bpkilint.home.arpa. 28 1}" client=10.55.55.55:48269 client_network=udp
bconsul-1   | 2024-11-06T15:51:19.483Z [ERROR] agent.dns: all resolvers failed for question from client: question="{bpkilint.home.arpa. 1 1}" client=10.55.55.55:33989 client_network=udp
boulder-1   | Wed Nov  6 15:51:19 UTC 2024 - still trying to connect to bpkilint:80
bconsul-1   | 2024-11-06T15:51:24.486Z [ERROR] agent.dns: all resolvers failed for question from client: question="{bpkilint.home.arpa. 1 1}" client=10.55.55.55:38353 client_network=udp
bconsul-1   | 2024-11-06T15:51:24.486Z [ERROR] agent.dns: all resolvers failed for question from client: question="{bpkilint.home.arpa. 28 1}" client=10.55.55.55:55049 client_network=udp
bconsul-1   | 2024-11-06T15:51:24.486Z [ERROR] agent.dns: all resolvers failed for question from client: question="{bpkilint.home.arpa. 1 1}" client=10.55.55.55:49636 client_network=udp
bconsul-1   | 2024-11-06T15:51:24.486Z [ERROR] agent.dns: all resolvers failed for question from client: question="{bpkilint.home.arpa. 28 1}" client=10.55.55.55:56173 client_network=udp
boulder-1   | Wed Nov  6 15:51:24 UTC 2024 - still trying to connect to bpkilint:80
bconsul-1   | 2024-11-06T15:51:29.489Z [ERROR] agent.dns: all resolvers failed for question from client: question="{bpkilint.home.arpa. 1 1}" client=10.55.55.55:60614 client_network=udp
bconsul-1   | 2024-11-06T15:51:29.490Z [ERROR] agent.dns: all resolvers failed for question from client: question="{bpkilint.home.arpa. 28 1}" client=10.55.55.55:55331 client_network=udp
bconsul-1   | 2024-11-06T15:51:29.490Z [ERROR] agent.dns: all resolvers failed for question from client: question="{bpkilint.home.arpa. 1 1}" client=10.55.55.55:59494 client_network=udp
bconsul-1   | 2024-11-06T15:51:29.490Z [ERROR] agent.dns: all resolvers failed for question from client: question="{bpkilint.home.arpa. 28 1}" client=10.55.55.55:46257 client_network=udp
boulder-1   | Wed Nov  6 15:51:29 UTC 2024 - still trying to connect to bpkilint:80

This is running for more that three hours. What now?

hakwerk commented 2 weeks ago

Does this command indeed resolve to the IP address of your LabCA machine?

docker compose exec bconsul nslookup labca.home.arpa

For some reason either the boulder container or the bconsul container is appending your domain ".home.arpa." to the query for bpkilint. It should just ask for "bpkilint" as that is the name within the docker-compose.yml file.

root@testpki:/home/labca/boulder# docker compose exec boulder wget bpkilint
--2024-11-11 19:33:57--  http://bpkilint/
Resolving bpkilint (bpkilint)... 10.77.77.9
Connecting to bpkilint (bpkilint)|10.77.77.9|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2024-11-11 19:33:57 ERROR 404: Not Found.

root@testpki:/home/labca/boulder# docker compose exec boulder wget bpkilint.my.domain
--2024-11-11 19:34:07--  http://bpkilint.my.domain/
Resolving bpkilint.my.domain (bpkilint.my.domain)... failed: Temporary failure in name resolution.
wget: unable to resolve host address 'bpkilint.my.domain'

jwgn commented 1 week ago

Yes, it resolves correctly to my LabCA container.

root@labca:/home/labca/boulder# docker compose exec bconsul nslookup labca.home.arpa
Server:         127.0.0.11
Address:        127.0.0.11:53

Non-authoritative answer:

Non-authoritative answer:
Name:   labca.home.arpa
Address: 192.168.100.35

hakwerk commented 1 week ago

Well, this is looking up the fqdn with the full domain, and doing so on the bconsul container.

Please check from the boulder container, and lookup just bpkilint without the domain

jwgn commented 1 week ago

Does not work:

root@labca:/home/labca/boulder# docker compose exec boulder wget bpkilint
--2024-11-17 19:14:41--  http://bpkilint/
Resolving bpkilint (bpkilint)... failed: Temporary failure in name resolution.
wget: unable to resolve host address 'bpkilint'

hakwerk / labca

Timeout in restart setup step #143