Closed walterzilla closed 4 years ago
Hi. The MeshCentral Install Guide has instructions to install behind NGNIX and Traefik (sections 16 and 17). You should be able to use about the same instructions to install other reverse proxies like Apache and HAproxy.
Hope it helps, Ylian
Already tried that way, no luck...my bad for sure. Time to switch to Ngnix :-)
Thanks
Probably this is related to Apache support websocket proxying. I haven't looked at using Apache to perform websocket reverse proxy.
I am a bit swamped with bugs & feature request, so I am not going to get to this in a while. If anyone successfully installed MeshCentral behind the Apache reverse proxy and can post the configuration file to do it, that would be greatly appreciated.
Hi All,
I was working on this today and I'd like to share what I found (using Various online help pages).
These modules will need to be loaded (Using apache for Windows as example) " LoadModule rewrite_module modules/mod_rewrite.so LoadModule proxy_module modules/mod_proxy.so LoadModule proxy_wstunnel_module modules/mod_proxy_wstunnel.so "
The Virtual Host Configuration will need something along the lines of;
<VirtualHost *:443>
ServerName mesh.proxy.net
RewriteEngine On RewriteCond %{HTTP:UPGRADE} ^WebSocket$ RewriteCond %{HTTP:CONNECTION} Upgrade$ RewriteRule /(.*) ws://127.0.0.1:4430/ [P]
ProxyPass / http://127.0.0.1:4430/
ProxyRequests Off
I hope this helps anyone with this problem in the future.
Not a Complete Solution.
If I'm already authenticated in the browser, I can use everything as normal with TlsOffload configured. (Should this be a WAN or LAN address?) If I need to authenticate, then I can't log in but if I change to NoTls, then everything works again except adding clients have connections with ws:\ when they should have wss:\
Perhaps there is another rewrite rule needed?
SIDE NOTE, we are an Intel Gold Partner, about the only reason i've spent days trying to get this work work behind a Apache HTTP server....
I HAVE THIS WORKING and have found many reasons why the video will never work for most "Reverse Proxy" servers, mostly as the reason there is a reverse proxy in place is port 443 is in use and has many websites hosted on port 443 already. This means the webserver will be using server names to work out the website to serve.
You will notice a fair amount of 127.0.0.1 in the config in the below video, well that doesnt POINT to your website that has the SSL certificate, only the domain name does, but that creates other issues. So i went back to basics as i have port forward other sites that use https and those work, so i just copied the config, and what would you know, that also worked, once i re-did the config for MeshCentral a little.
I might add, ive not FULLY tested the system, but i can log in and i know websockets is working (as it remains logged in), the config i used in both MeshCentral and Apache is at the end of this message (to keep all this clean).
The video that gave me almost all the clues is: https://www.youtube.com/watch?v=ebDVAsistbk
NOTES: TLS Off loading will never work (with MeshCentrals current config method) while you are ReverseProxy from a webserver with other websites hosted, mostly as 127.0.0.1 doesnt point to the website with the SSL you need to read.
MeshCentral config "TlsOffload": "127.0.0.1"
This will never work, 127.0.0.1 doesnt have the SSL the ReverseProxy is using, it needs to be https://domain.com:443, the webserver will then work out the site your trying to get to and push the certificate forward, this was the main issue i had for MANY hours before i got it working.
Now TlsOffload is not used, then all other URLS need to be https://x.x.x.x:4430/ which will point to the MeshCentral server.
Apache also needs ProxyPreserverHost ON, as this will push the needed headers to MeshCentral.
My config is: Apache 2.4, running port 80 and 443 for MANY websites (some with certificates) MeshCentral on the same server (port 4430) We did NOT move the MPS port (thats still 4433) as we can directly access this through the firewall, its port 443 thats creating our issue as its in use on the firewall already to this Apache server.
CONFIG for MeshCentral
{
"settings": {
"cert": "sub.domain.ca",
"_minify": true,
"selfupdate": true,
"webrtc": true,
"port": 4430,
"AliasPort": 443,
"mpsport": 4433
},
"domains": {
"": {
"title": "COMPANY",
"title2": "Gives us money - 123 456 7890"
}
}
}
Your notice NO TlsOffload in the above section, this means MeshCentral will still answer on https://127.0.0.1:4430 on the local server (and fully work)
Apache config (our Apache is hosting MANY virtual sites including a few for WordPress, so the re-write module was already in place and used Clearly ive not listed ALL the Apache CONF file, only parts you need to look at.
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule proxy_connect_module modules/mod_proxy_connect.so
**: Needed to do ProxyPass**
LoadModule rewrite_module modules/mod_rewrite.so
**: Needed for re-write rule**
LoadModule ssl_module modules/mod_ssl.so
**: Needed for SSL**
LoadModule proxy_wstunnel_module modules/mod_proxy_wstunnel.so
**: Need for the WS Tunnel
: There maybe other modules needed, we have ALOT so i am quick browsing for them in the conf file.**
::: This virtual host is used to redirect port 80 to 443, this was instead of having MeshCentral doing it :::
<VirtualHost [your ip address]:80>
ServerName sub.domain.ca
HostnameLookups on
Redirect 302 / https://sub.domain.ca/
ErrorLog "c:\www\logs\error.log"
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
CustomLog c:\www\logs\access.log combined
</VirtualHost>
>>> Redirect 302 / https://sub.domain.ca/ <<< is the part that will be sent to the browser to redirect to the correct URL, this means it never leaves Apache
<VirtualHost [your ip address]:443>
ServerName sub.domain.ca
ErrorLog "c:\www\logs\error.log"
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
CustomLog c:\www\logs\access.log combined
SSLCertificateFile "C:/SSL/cert.crt" [this is the certificate from MeshCentral]
SSLCertificateKeyFile "C:/SSL/key.key" [this is the key from MeshCentral]
SSLEngine on
SSLProxyEngine on
RewriteEngine on
RewriteCond %{HTTP:Upgrade} websocket [NC]
RewriteCond %{HTTP:Connection} upgrade [NC]
RewriteRule .* "wss://127.0.0.1:4430%{REQUEST_URI}" [P]
<Location />
ProxyPass https://127.0.0.1:4430/
ProxyPassReverse https://127.0.0.1:4430/
ProxyPreserveHost On _[this passes the client headers to MeshCentral [yet to be tested]]_
</Location>
The RewriteCond are the sections that will handle the Sockets, its messy but works
UPDATE The AGENT works, however all connections are from the local machine (127.0.0.1), this doesnt surprise me, its something i will look in to as its clearly Apache not sending the correct headers over. Might also be connected to not telling MESH that the TLS is off loaded.
Need input from the developer here... (who i have emailed directly)
I will need to install Apache, configure it and give this a run. I fully expect MeshCentral to work with "tlsoffload" enabled. A quick look at the mod_proxy documentation shows that the right headers are being added (X-Forwarded-Host...) so, I fully expect this to work. This said, I will try to put some time on this to see if I can make it work.
I will need to install Apache, configure it and give this a run. I fully expect MeshCentral to work with "tlsoffload" enabled. A quick look at the mod_proxy documentation shows that the right headers are being added (X-Forwarded-Host...) so, I fully expect this to work. This said, I will try to put some time on this to see if I can make it work.
Thanks, i also expected it to work, however it took me a few days to get the above working, however all IP addresses in MeshCentral are showing as 127.0.0.1.
I have another server (email server) thats https, Apache is a reverse proxy, it also sees everything as 127.0.0.1 so it appears its an Apache http issue (from what i can see).
When you are testing, please make sure to use full domain names as most who will want to off load only had 1 external IP address and Apache is likely hosting websites on port 443, so any cert checking will need to be via the DNS name not just the IP address to the Apache server.
I made progress on using Apache as a reverse proxy. I encountered #439, once I get that fixed, I should be able to document the results. I got it to work with TLSOffload enabled.
Hi
I found the reason i was unable to login had something to do with the SSL offload and the TCP transport... not sure which. I bet if you turn the SSL Offload off AND use https://127.0.0.1:port to MeshCentral using Apache to SSL Proxy your likely be able to login, if so then its something to do with with Apache / Mesh SSL system.
b8two (above) seems to have a simular issue, when they turned off the SSL altogether in Mesh Central the Apache reverse Proxy worked for login
Published MeshCentral v0.4.0-i with a bunch of improvements for reverse-proxies. The main problem was that cookies where not being sent when TLSOffLoad was used, this is now fixed. I also improved MeshCentral to work somewhat correctly if the browser rejects cookies entirely. Lastly, below is my small Apache configuration file (httpd.conf) that does reverse-proxy to MeshCentral. The config.json will need to have this in the "settings" section:
"Port": 4430,
"AliasPort": 443,
"RedirPort": 800,
"TlsOffload": "127.0.0.1"
The httpd.conf looks like this. You need change the SRVROOT and SRVNAME values.
LoadModule authz_core_module modules/mod_authz_core.so
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule proxy_wstunnel_module modules/mod_proxy_wstunnel.so
LoadModule ssl_module modules/mod_ssl.so
Define SRVROOT "c:/Apache24"
Define SRVNAME "devbox.mesh.meshcentral.com"
ServerRoot "${SRVROOT}"
# MeshCentral HTTP port
Listen 80
ServerAdmin admin@example.com
ServerName ${SRVNAME}:80
<VirtualHost _default_:80>
ServerName ${SRVNAME}:80
ProxyPass "/" "http://127.0.0.1:800/"
</VirtualHost>
# MeshCentral HTTPS port
Listen 443
SSLProtocol TLSv1.2 +TLSv1.3
SSLProxyProtocol TLSv1.2 +TLSv1.3
SSLPassPhraseDialog builtin
SSLSessionCacheTimeout 300
SSLRandomSeed startup builtin
SSLRandomSeed connect builtin
<VirtualHost _default_:443>
ServerName ${SRVNAME}:443
SSLEngine on
SSLCertificateFile "${SRVROOT}/webserver-cert-public.crt"
SSLCertificateKeyFile "${SRVROOT}/webserver-cert-private.key"
ProxyPass "/agent.ashx" "ws://127.0.0.1:4430/agent.ashx"
ProxyPass "/amtactivate" "ws://127.0.0.1:4430/amtactivate"
ProxyPass "/control.ashx" "ws://127.0.0.1:4430/control.ashx"
ProxyPass "/meshrelay.ashx" "ws://127.0.0.1:4430/meshrelay.ashx"
ProxyPass "/webrelay.ashx" "ws://127.0.0.1:4430/webrelay.ashx"
ProxyPass "/webider.ashx" "ws://127.0.0.1:4430/webider.ashx"
ProxyPass "/" "http://127.0.0.1:4430/"
</VirtualHost>
I had to make a ProxyPass for each web socket URL. If there is a way not to do this, please post it. Feedback appreciated.
Hi Ylianst,
To handle all websocket connections, these rules work.
RewriteEngine On RewriteCond %{HTTP:UPGRADE} ^WebSocket$ RewriteCond %{HTTP:CONNECTION} Upgrade$ RewriteRule /(.*) ws://127.0.0.1:4430/ [P]
For this to work you will also need to :
LoadModule rewrite_module modules/mod_rewrite.so
my server says the latest version is 0.4.0.k did you do updates beyond this ticket (appears to be 2 later releases)
@GRIFFCOMMca I did more updates after MeshCentral v0.4.0-i for other issues. Any version after K should work.
@b8two Thanks for the tip, I will try the "rewrite" tomorrow and post back the result. I want to add a section of Apache Reverse Proxy to my documentation tomorrow, this would be perfect.
The above 98% works but something was missed.... (we now have it fully working by adding the below to the meshcentral config.json file).
"domains": { "": { "certUrl": "https://sub.domain.ca/", } }
The "certUrl" is needed when the Apache server is hosting more than one domain name, https://127.0.0.1 will not send the real website certificate when the Apache server is looking at domain names to push the correct website, so "https://sub.domain.ca" is needed, if this is not used we had a 503 error with the Mesh Central server then stopping.
In our Apache conf file we have this:
*SSLCertificateFile "C:/cert.crt" SSLCertificateKeyFile "C:/key.key" SSLEngine on RewriteEngine on RewriteCond %{HTTP:Upgrade} websocket [NC] RewriteCond %{HTTP:Connection} upgrade [NC] RewriteRule . "ws://127.0.0.1:4430%{REQUEST_URI}" [P] ProxyPass / http://127.0.0.1:4430/ connectiontimeout=86400 timeout=30 ProxyPassReverse / http://127.0.0.1:4430/ ProxyPreserveHost On**
We used the [NC] to be sure the case wasnt going to create an issue with the URL, just to be sure adding %{REQUEST_URI} to the rewrite rule makes it a little more hard coded.
It appears Apache is unable to work out the difference between http request and websockets so this rewrite rule is needed, still kinda messy but appears to work.
We are now able to login AND we get the REAL WAN IP addresses showing up in Mesh Central.
How do we update the clients? Is there no "auto client update" feature? (might go well with the client remove feature i requested in this ticket.... https://github.com/Ylianst/MeshCentral/issues/449
I noticed something, in the Event Log when logged in as an administrator most of the events are showing up as 127.0.0.1, such as: Ended desktop session "xed836sx0dz" from 127.0.0.1 to 127.0.0.1, 3 second(s) It does however show the start of the session from the correct real IP address, is this a bug in the log as its using a proxy?
Hi Ylianst,
Using Apache for a reverse proxy, I came in to an issue using the Windows Version and I have now discovered this is due to the default ThreadsPerChild limit set to 64. This is important as each Websocket consumes a thread, hence I reached the limit quickly with all the clients connecting in. Hence you may want to mention this in the documentation.
I came across this page today which may help with your Apache reverse proxy documentation; https://www.happyassassin.net/2018/11/23/reverse-proxying-websockets-with-apache-a-generic-approach-that-works-even-with-firefox/
Also another way to do this without "tlsoffload" function and associated performance benefits is to use the self signed certificates and configure Apache to ingore what is wrong with the certificates using these parameters;
SSLProxyEngine on SSLProxyVerify none SSLProxyCheckPeerCN off SSLProxyCheckPeerName off SSLProxyCheckPeerExpire off
I'm having a similar issue to GRIFFCOMMca regarding logging of IP addresses in MeshCentral. They all are shown as the Apache reverse proxy machine IP address. In my case, the reverse proxy is on another machine in a DMZ environment. It is also the case on the "My Devices" Tab where all devices addresses are the reverse proxy IP address. I'm not too concerned but I don't think this is how you want it to behave.
The clarify, the IP addresses shown on the My Devices page are the CORRECT WAN IP addresses, its only in the LOG file that i seem to see 127.0.0.1 alot.
When i first set up the reverse proxy before Ylain fixed it all the devices IP addresses were the Apache Web server, this was fixed after Ylain "fixed" the code somehow.
Does this socket limit have anything to do with the ticket i opened here https://github.com/Ylianst/MeshCentral/issues/460
I can confirm the solution provided by Yilanst as well as the one suggested by GRIFFCOMca are working. I also observe that Client Connections are logged as 127.0.0.1 as GRIFFCOMca already mentioned.
FYI. I published MeshCentral 0.4.0-o yesterday that fixed the IP address in the logs. So, it should show the correct IP address now.
I can confirm this (logging of real WAN IP addresses) is now working through the Apache reverse proxy
Confirmed as well
Hi, I still don't have the correct IP addresses in MC.
My configuration is using Two Machines (Apache Reverse Proxy in a DMZ) via a router with Mesh Central Server on another. Both have different subnet mask IP range and prior to the v0.4.0-i update I could add extra clients no problem (~90). Even with the latest update 0.4.0-y all agents are seen connecting from the DMZ, as are the users
I changed the TlsOffload from the Public IP to the Private LAN IP and now the agent IP are indcatative of the origin.
I get the following error when using apache as a reverse proxy:
Bad Web Certificate
I can't connect to my agents.
My fully working config files for Apache and MC.
<VirtualHost *:443>
ServerName mesh.mydomain.tld
ServerAdmin postmaster@mydomain.tld
Alias /robots.txt /var/www/html/robots.txt
RewriteEngine on
RewriteCond %{HTTP:Upgrade} websocket [NC]
RewriteCond %{HTTP:Connection} upgrade [NC]
RewriteRule . "ws://127.0.0.1:8081%{REQUEST_URI}" [P]
ProxyPassMatch ^/robots.txt !
ProxyPass / http://127.0.0.1:8081/ connectiontimeout=86400 timeout=30
ProxyPassReverse / http://127.0.0.1:8081/
ProxyPreserveHost On
ErrorLog ${APACHE_LOG_DIR}/error.log
CustomLog ${APACHE_LOG_DIR}/access.log combined
SSLEngine on
SSLCertificateFile /etc/letsencrypt/live/mesh.mydomain.tld/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/mesh.mydomain.tld/privkey.pem
SSLCACertificateFile /etc/letsencrypt/live/mesh.mydomain.tld/chain.pem
SSLUseStapling on
</VirtualHost>
{
"settings": {
"cert": "mesh.mydomain.tld",
"Port": 8081,
"RedirPort": 8082,
"AliasPort": 443,
"AgentPong": 300,
"TLSOffload": "127.0.0.1",
"Minify": 1,
"Wanonly": true
},
"domains": {
"": {
"newAccounts" : 0,
"certUrl": "https://mesh.mydomain.tld"
}
}
}
My fully working config files for Apache and MC.
<VirtualHost *:443> ServerName mesh.mydomain.tld ServerAdmin postmaster@mydomain.tld Alias /robots.txt /var/www/html/robots.txt RewriteEngine on RewriteCond %{HTTP:Upgrade} websocket [NC] RewriteCond %{HTTP:Connection} upgrade [NC] RewriteRule . "ws://127.0.0.1:8081%{REQUEST_URI}" [P] ProxyPassMatch ^/robots.txt ! ProxyPass / http://127.0.0.1:8081/ connectiontimeout=86400 timeout=30 ProxyPassReverse / http://127.0.0.1:8081/ ProxyPreserveHost On ErrorLog ${APACHE_LOG_DIR}/error.log CustomLog ${APACHE_LOG_DIR}/access.log combined SSLEngine on SSLCertificateFile /etc/letsencrypt/live/mesh.mydomain.tld/fullchain.pem SSLCertificateKeyFile /etc/letsencrypt/live/mesh.mydomain.tld/privkey.pem SSLCACertificateFile /etc/letsencrypt/live/mesh.mydomain.tld/chain.pem SSLUseStapling on </VirtualHost>
{ "settings": { "cert": "mesh.mydomain.tld", "Port": 8081, "RedirPort": 8082, "AliasPort": 443, "AgentPong": 300, "TLSOffload": "127.0.0.1", "Minify": 1, "Wanonly": true }, "domains": { "": { "newAccounts" : 0, "certUrl": "https://mesh.mydomain.tld" } } }
Thanks, this has helped me
On this issue...
I run Apache web servers I have a meshcentral server on same machine I have reversed proxied from port 443 (Apache) to port 444 (MeshCentral Server) and it works fine (thanks to the above posts)
What I'd like to do is create Mesh Client downloads that use port 443 rather than port 444. How can I do this? Currently I install then modify the MeshAgent.msh file
@OutbackMatt: Did you set the AliasPort
to 443
?
Ah, no I didn't
Have done now thanks
Nice! Thanks @MailYouLater. Yes, that is exactly what "AliasPort" is for.
This is actually giving me some grief...
If I set a aliasPort any new 'add agent' doesn't want to connect I don't have TLS offload set, as If I turn that off the main web page stops being HTTPS, and won't load due to my HSTS
I'd like to use the Lets Encrypt certificates for my subdomian that I already have in (ubuntu) /etc/letsencrypt/live/sub.fqdn/ but I can't seem to work that out. (I can set a symbolic link, but I can't seem to have different permissions for the link)
I want to pass WSS not WS (which I'm currently doing) between the apache2 Reverse Proxy and tmy mesh
While this thread has a bunch of example, most of them are focused on edge security (security at the proxy, ignoring everything inside). The last config example used nonstandard ports and ran http inside the network, offloading tls.
Here is a more standard, more secure, config.
Apache reverse proxy with these mods loaded: proxy_module (shared) proxy_http_module (shared) proxy_wstunnel_module (shared)
Working Mesh install. With a direct connection (inside the network) you should have working SSL -copy the cert files into the correct directories in MC, restart services. https://mesh.domain.tld (and http) should work and load correctly from another PC on lan. Basically, replace “webserver-cert-public.crt” and “webserver-cert-public.key” with your certificate. These files are located in “meshcentral-data” folder of the server. If intermediate certificates are needed, add the files “webserver-cert-chain1.crt”, “webserver-cert-chain2.crt”,
3) A config looking like this:
{
"settings": {
"AllowHighQualityDesktop": true,
"cert": "mesh.domain.tld",
"_SelfUpdate": true,
"minify": true,
"Port": 443,
"RedirPort": 80,
"SessionKey": "<randomstrongstring>",
"ExactPorts": "true"
},
"domains": {
"": {
"title": "Mesh",
"newaccounts": false,
"certUrl": "https://mesh.domain.tld/"
}
}
}
4) Virtual host file that looks like this
<IfModule mod_ssl.c>
<VirtualHost *:443>
ServerAdmin webmaster@localhost
ServerName mesh.domain.tld
ProxyPreserveHost On
ProxyPass "/" "https://mesh.domain.tld/"
ProxyPassReverse "/" "https://mesh.domain.tld/"
RewriteEngine on
RewriteCond %{HTTP:Upgrade} websocket [NC]
RewriteCond %{HTTP:Connection} upgrade [NC]
RewriteRule . "wss://mesh.domain.tld%{REQUEST_URI}" [P]
ErrorLog ${APACHE_LOG_DIR}/error.log
CustomLog ${APACHE_LOG_DIR}/access.log combined
SSLEngine on
SSLProxyEngine On
SSLCertificateFile /etc/ssl/certs/cert.pem
SSLCertificateKeyFile /etc/ssl/private/key.key
SSLProtocol TLSv1.2
SSLCipherSuite EECDH+AESGCM:EDH+AESGCM
SSLHonorCipherOrder on
<FilesMatch "\.(cgi|shtml|phtml|php)$">
SSLOptions +StdEnvVars
</FilesMatch>
<Directory /usr/lib/cgi-bin>
SSLOptions +StdEnvVars
Options FollowSymLinks
AllowOverride All
</Directory>
</VirtualHost>
</IfModule>
Either redirect 80 at the proxy or pass through
I use the same wildcard SSL cert for both apache and the mesh server
(sorry for bad formatting)
Per the setup guide, as MPS/CIRA is a binary protocol, will Apache even be able to reverse proxy that? I know it doesn't seem to have the same capability as nginx/haproxy.
Would we be able to port forward it, for the situation of "public facing proxy" and "internal only app server"?
For CIRA, you will need to just TCP forward the connection. Intel AMT does not support TLS-SNI (Server Name Indication) so there is no way to perform a reverse proxy on the CIRA connection and route the connection correctly. The best you can do is terminate the TLS in the reverse proxy if you like and forward the connection to MeshCentral.
If you do terminate the TLS on the reverse proxy, you need to use the long-standing self-signed certificate that MeshCentral uses on the MPS/CIRA port. Hope that makes sense.
FYI. I am going to close this issue. However, please feel free to open new issues on AMT/CIRA, etc.
AFAYK there's any chance to make MC (agents, web UI, console, file, and so on....) work through apache reverse proxy? Thanks!