After using the ACME service command "update local UniFi keystore", UniFi controller is not starting anymore

[Qhilm]

As mentioned in this issue, I have deployed the cert of the opnsense router to UniFi using the "update local UniFi keystore" from @adn77.

The UniFi Controller never came back after this and it is impossible to keep it running, it stops right after being started since then.

I have moved the folder /usr/local/share/java/unifi/, deleted the plugin, reinstalled, rebooted opnsense, nothing helped so far.

Which logs can I look at to understand what's blocking the startup of the plugin?

[Qhilm]

So I removed all the files and folder I could find that contained "unifi" and the plugin still refuses to start.

I must be missing something obvious...

[mimugmail]

Can you check all the in unifi/logs folder? Actually I have no idea since I dont use other certificate

[Qhilm]

There isn't much. But I have removed the keystore, uninstalled the plugin, removed all files I could find with "unifi" in the name, I am really at a loss what could still prevent the plugin from starting...

root@sense:~ # cat /usr/local/share/java/unifi/logs/*
[2024-05-17 17:27:33,207] <main> WARN  system - reload system.properties failed: file not found
[2024-05-17 17:27:33,238] <main> WARN  system - reload system.properties.bk failed: file not found
[2024-05-17 17:27:33,243] <launcher> INFO  launcher - Initializing on FreeBSD(13.2-RELEASE-p11), jre17(17.0.10)
[2024-05-17 17:27:33,246] <launcher> INFO  system - *** Running for the first time, creating identity ***
[2024-05-17 17:27:33,251] <launcher> INFO  system - UUID: e40d79db-9f5b-43ec-843d-a1768be1c2fd
[2024-05-17 17:27:33,253] <launcher> WARN  system - reload system.properties failed: file not found
[2024-05-17 17:27:33,254] <launcher> WARN  system - reload system.properties.bk failed: file not found
[2024-05-17 17:27:33,333] <launcher> INFO  system - Reporter UUID: ae1cbac0-1541-4e51-a9d6-64ab5fb515b6
[2024-05-17 17:27:34,667] <launcher> INFO  startup - Initiating startup
[2024-05-17 17:30:28,651] <launcher> INFO  launcher - Initializing on FreeBSD(13.2-RELEASE-p11), jre17(17.0.10)
[2024-05-17 17:30:29,704] <launcher> INFO  startup - Initiating startup
[2024-05-17 17:30:29,704] <launcher> INFO  startup - Initiating startup

on subsequent restarts, it's even less, there is only three log lines added in total:

[2024-05-17 20:39:19,782] <launcher> INFO  launcher - Initializing on FreeBSD(13.2-RELEASE-p11), jre17(17.0.10)
[2024-05-17 20:39:21,026] <launcher> INFO  startup - Initiating startup
[2024-05-17 20:39:21,026] <launcher> INFO  startup - Initiating startup

[mimugmail]

And you are sure there is no other process running on unifi port like crowdsec?

[Qhilm]

omg, you saved me, at least now it's starting. I will redo the whole cert deployment and see what happens this time.

The reason I didn't even consider that is that I installed crowdsec long before unifi and never had that conflict...

For anyone wanting to check this:

root@sense:~ # sockstat -4 -l | grep :8080 | awk '{print $3}' | head -1
41005
root@sense:~ # ps -p 41005
  PID TT  STAT     TIME COMMAND
41005  -  S    17:45.86 /usr/local/bin/crowdsec -c /usr/local/etc/crowdsec/config.yaml

Change the LAPI port int he crowdsec settings and don't forget to restart.

Can we keep this open for now? I will redo the cert deploy and check what happens again.

Side note: a way to set logs to debug to see more would be useful here.

[Qhilm]

I can't reproduce this issue anymore. I deployed the cert and the UniFi service is working. It seems it was probably due to crowdsec also using port 8080, but I don't understand why I did not get this issue before: I have had the crowdsec plugin for much longer than UniFi Controller. Anyway – unreproducible, let's close it.