[FEAT]: Adding additional proxy settings

[pthoelken]

Feature Description

In my homelab there is a proxmox server running and I think there is missing some configurations for the proxy pass configuration like this: https://pve.proxmox.com/wiki/Web_Interface_Via_Nginx_Proxy

Is there a way to configure the mandatory steps in my Cosmos > URLs > Proxmox ?

[azukaar]

Nothing is missing, there is an issue with Proxmox specifically since 0.12, it's been reported already... I investigated but it's hard because I do not have Proxmox so I'm a bit blind. Something to do with the forward and host header most likely

[pthoelken]

Ah okay, I understand. I don't know if it helps for you, but Promox is very easily installed by an iso. Also in a virtual machine (with enabled virtualization technology).

[azukaar]

Yeah I just need to reinstall a server to debug this, I'll do it when I'll have the courage :D In the meantime if anyone using Proxmox has more info

[pthoelken]

So I have Proxmox, when your tell me which informations / command outputs do you need, letting me know. I will do my best to support you in this case.

[azukaar]

Basically what I need is the error Proxmox is outputing, we were able to identify the error was coming from Proxmox because the Cosmos logs show that the error code from Proxmox is 401, but then it does not seem to output any error in its logs (as far as we could tell) may be some logging level config or smtg I dont reallly know

[pthoelken]

So as I understand you correctly you need the logs from Proxmox Login Auth Log (from the Proxmox Server) right @azukaar - if it's correct, I'll check out this later at home.

[azukaar]

yep basically, thanks

[pthoelken]

I'll getting back to you here as soon as I have the logs from my server later.

[pthoelken]

I did not find any logs while login from /var/log* but in my console I saw these errors

[azukaar]

Yes those are the 401 that Proxmox are returning but I need to know why it is returning them to fix the issue :/

[pthoelken]

I've asked in the proxmox community, where are the logs are located. The weired thing is, that the cluster backend told me I'm logged in:

Nov 21 21:18:51 pve01 pvedaemon[1274]: <root@pam> successful auth for user 'root@pam' but the frontend give me the 401.

[pthoelken]

Sooo for now, I can start the pveproxy in debug mode and I get these informations, while the 401 appears. @azukaar

worker[88331]: PVE::APIServer::AnyEvent +189: client_do_disconnect: close connection AnyEvent::Handle=HASH(0x562510f32e58)
worker[88331]: PVE::APIServer::AnyEvent +189: client_do_disconnect: CLOSE FH10 CONN1
worker[88332]: PVE::APIServer::AnyEvent +189: client_do_disconnect: close connection AnyEvent::Handle=HASH(0x562510f26d80)
worker[88332]: PVE::APIServer::AnyEvent +189: client_do_disconnect: CLOSE FH10 CONN0
worker[88330]: PVE::APIServer::AnyEvent +189: client_do_disconnect: close connection AnyEvent::Handle=HASH(0x562510f67008)
worker[88330]: PVE::APIServer::AnyEvent +189: client_do_disconnect: CLOSE FH12 CONN2
worker[88330]: PVE::APIServer::AnyEvent +189: client_do_disconnect: close connection AnyEvent::Handle=HASH(0x5625112de280)
worker[88330]: PVE::APIServer::AnyEvent +189: client_do_disconnect: CLOSE FH13 CONN1
worker[88330]: PVE::APIServer::AnyEvent +189: client_do_disconnect: close connection AnyEvent::Handle=HASH(0x562510f33560)
worker[88330]: PVE::APIServer::AnyEvent +189: client_do_disconnect: CLOSE FH10 CONN0
worker[88331]: PVE::APIServer::AnyEvent +189: client_do_disconnect: close connection AnyEvent::Handle=HASH(0x562510f66ac0)
worker[88331]: PVE::APIServer::AnyEvent +189: client_do_disconnect: CLOSE FH12 CONN0
worker[88331]: PVE::APIServer::AnyEvent +1792: (eval): ACCEPT FH10 CONN1
worker[88331]: PVE::APIServer::AnyEvent +1792: (eval): Setting TLS to autostart
worker[88331]: PVE::APIServer::AnyEvent +1842: client_do_disconnect: close connection AnyEvent::Handle=HASH(0x562510f98bd0)
worker[88331]: PVE::APIServer::AnyEvent +1842: client_do_disconnect: CLOSE FH10 CONN0

[pthoelken]

Help this for your investigations @azukaar or should I do more here?

[I-Need-C8H10N4O2]

I have tested with Proxmox 7.4 and Proxmox 8.0. Both are working fine with no issues whatsoever. Perhaps the issue is with your config?

[pthoelken]

Can you share your configuration? So it doesn't work here.

[I-Need-C8H10N4O2]

@pthoelken I just added the URL of my Proxmox instance to the URLs.

• I chose "Proxy" mode and it points to https://10.10.10.1:8006 where 10.10.10.1 is the IP address of my Proxmox instance.
• I ticked the checkbox "Accept Insecure HTTPS Target (not recommended)" because my Proxmox is running a self-signed cert.
• Unticked "Authentication Required" and "Smart Shield Protection".
• In the Security tab, I chose both "Forward IP and Host Headers to target" and "Disable Header Hardening".

That is pretty much i.t You may not need the exact same options, but I have them this way because it fits my use case.

[dragons-library]

TL;DR - Clearing cookies in my browser resolved this issue for me.

I was hitting this same issue when I added my Proxmox proxy URL.

I initially entered http instead of https in my Target URL, which resulted in an infinite redirect loop (not sure if this somehow caused the next issue).

Fixed that and then hit the 401 issue as noted above.

Tried changing a bunch of proxy settings in Cosmos (disabling security features, etc) and didn't help. I tailed the logs on my pve node (tail -100f /var/log/pveproxy/access.log) and saw...

::ffff:192.168.0.20 - - [06/01/2024:16:29:44 -0800] "GET /api2/json/cluster/resources HTTP/1.1" 401 -
::ffff:192.168.0.20 - - [06/01/2024:16:29:44 -0800] "GET /api2/json/cluster/tasks HTTP/1.1" 401 -
::ffff:192.168.0.45 - root@pam [06/01/2024:16:29:47 -0800] "GET /api2/json/nodes/pve/status HTTP/1.1" 200 978
::ffff:192.168.0.45 - root@pam [06/01/2024:16:29:47 -0800] "GET /api2/json/cluster/tasks HTTP/1.1" 200 1058
::ffff:192.168.0.45 - root@pam [06/01/2024:16:29:47 -0800] "GET /api2/json/cluster/resources HTTP/1.1" 200 552
::ffff:192.168.0.45 - root@pam [06/01/2024:16:29:47 -0800] "GET /api2/json/cluster/log HTTP/1.1" 200 1015

Note that the requests coming from Cosmos (192.168.0.20) do not have a user, whereas the requests coming directly from my browser logged directly into proxmox (192.168.0.45) did have a user (root@pam).

I tried fiddling with more things and eventually resorted to reddit and found this post: https://www.reddit.com/r/Proxmox/comments/gt0nuj/putting_proxmox_behind_reverse_proxy_doesnt_work/

This gem from the comments was what I needed:

Went down a rabbit hole of debugging before trying the proxy in a private window.

I tried incognito and it authenticated just fine.

I went back to main browser session, cleared cookies, and it was smooth sailing.

I re-enabled all the security features in Cosmos and no problems since.

Here are my Cosmos proxy URL settings:

Setup

Security

[pthoelken]

Yep, it seems to be a simple caching problem. After using a new browser (to check out the cache problem), the problem is gone. So in this case, I don't know if it's really a feature request in here? What do you think @azukaar ?

[pthoelken]

After newer investigations in this case I think this depens from the Chrome Kit. In Edge and Chrome it didnt work. Either when I clear the cache.

When I try this on mac at my Safari browser all works well.

[colmcgiv]

I think I'm running into the same issue, it might be to do with the cookies Cosmos creates when a user logs in?

When Cosmos authenticates a user 2 cookies are created, client-infos and jwttoken. On my setup the Domain of each cookie is prefixed with . which makes the cookies available for all subdomains. It's this that Proxmox doesn't like because if client-infos and jwttoken are present when trying to access a current Proxmox session we see the behaviour described above.

This is Proxmox (Proxied through Cosmos) and it's auth cookie PVEAuthCookie:

---

If we login to Cosmos we can see it create client-infos and jwttoken:

---

Next, if we go back to our Proxmox tab we can see client-infos and jwttoken are present on the Proxmox subdomain:

---

And finally, if we refresh or try to navigate to a different page in Proxmox it seems Proxmox removes PVEAuthCookie and asks the user to login, at which point after logging in we get caught in a cycle of Proxmox creating PVEAuthCookie and then immediately removing it. This happens only while client-infos and jwttoken are present.

If client-infos and jwttoken are removed then Proxmox works as expected, but of course this also logs us out of Cosmos.

Removing the . prefix from the Domain for client-infos and jwttoken seems to keep both Proxmox and Cosmos happy, although I'm not sure what knock-on effects this would have. I assume it was added as part of a fix for a different issue?