Can't Use dashPanel with ProxyPass ... or Remotely

[arrmo]

Hi,

I'd like to access dashPanel remotely, using ProxyPass (so a sub-path / URL, i.e. /pentair). That isn't working, given the relative paths used (like ../socket.io-client/socket.io.js). Not a huge issue.

So then I tried to use a subdomain ... but that fails also, as it seems that there is direct access to the server machine (e.g. http://localhost:4200). That definitely isn't routable, so accessible remotely.

Am I trying to use this incorrectely? Do you know if it can be used remotely?

Thanks!

FYI, a bit more info on what I have been trying, similar to, https://serverfault.com/questions/561892/how-to-handle-relative-urls-correctly-with-a-reverse-proxy

[rstrouse]

@arrmo,

What is the specific error you are receiving. This shouldn't be an issue but there are actually 2 ports that need to be handled. The first is njspc. Can you access it directly by sending http://<address to njspc>/config/options/circuits This should return all the circuits assigned in njspc. Now that we have that the setting under the hamburger menu for njspc should be set to that. The client directly subscribes to the socket.

Now there is another part of this that needs to be dealt with. The dashPanel server simply serves up a page set then does nothing more. It too needs to be accessible through your proxy. Since the page directly subscribes to the socket on the browser, it is probably throwing you off. In the future I plan on creating the proxy class in dashPanel so it routes to the dashPanel server in that mode then proxies to njspc so you don't have to have both accessible.

Does that help?

[tagyoureit]

Hey @arrmo, long time! FWIW, -webClient does use the proxy approach today. You could only open the single port. But my feelings aren't hurt if you want to use @rstrouse. ;)

[arrmo]

long time

Too long! Hope all is good there, and you are staying safe. Glad to see this progressing so well - really awesome. Sorry for "disappearing" ... few family health issues to deal with, and handling affairs for a passing relative - kinda put the pool on hold here 😞. But want to get back to it now!

My opinion (by all means disagree!), I would like to see both clients working, local and remote. They are captured on your Github page, so just thinking it would be good. And I'll help as much as I can. There is a reason I'm "starting" with dashPanel - when I tried webClient, it failed the first run, then did run (just the same command again) ... but took ~ 2200 seconds to start up. Seems odd, no? And yes, I do have all the npm dependencies installed LOL! BTW, I do find that npm install doesn't grab the most recent versions (or desired ones even), npm update seems to be required. Odd, not sure why that is. But back to the client(s) - get them both working? Need to figure out why it takes 30+ minutes to start webClient though (on a RPi 3b, BTW).

@rstrouse, on the issues I'm seeing - I can set up a subdomain, is that sort of your expectation? Or "clean up" the links, to remove the "../" paths? Again, more than willing to try to help!

Thanks again, this really is great to see.

[rstrouse]

Hmmm... there shouldn't be any relative paths in the client side files and the typescript relative paths should not matter for the serving pages. I am by no means an express expert but perhaps the static router is putting those in. Do you have an example of one of the relative paths that it is looking for. Are these perhaps backgrounds in the css?

[rstrouse]

Btw sorry about your loss 😞

[arrmo]

Btw sorry about your loss

Thanks, appreciate it. I'm actually executor on my F-i-L's will, that's been a bit of a (sad) learning curve, and takes more time that I thought it would.

there shouldn't be any relative paths in the client side files

For example, look at https://github.com/rstrouse/nodejs-poolController-dashPanel/blob/master/pages/index.html, row 8-11. These ../ paths seem to mess with Apache2 ProxyPass (like in the link above). Make sense? It seems that then Apache loses the sub-path (or whatever it's called 🤣). If I manually re-add it (to the URL) then it works, but by default it's removed.

Thanks!

[tagyoureit]

Yes, condolences... sorry to hear that.

Need to figure out why it takes 30+ minutes to start webClient though (on a RPi 3b, BTW).

The long load times on -webClient are due to Parcel. If you pre-build it npm run build and then npm run start:cached every subsequent time. It should be much faster. You can do the same with njsPC as well.

[arrmo]

The long load times on -webClient are due to Parcel

That makes sense, was going to check in it more today. Have you set this (start:cached) up to work with pm2? It seemed to cause (syntax?) issues for me, but I may have messed it up.

Thanks!

[tagyoureit]

Ya, PM2 can be a little finicky when setting up npm start scripts. Skip the npm part and just run the contents of the script directly. node ./dist/app.js PM2 seems to like this better because it knows that it is a node process and will monitor it slightly differently.

[rstrouse]

For example, look at https://github.com/rstrouse/nodejs-poolController-dashPanel/blob/master/pages/index.html, row 8-11. These ../ paths seem to mess with Apache2 ProxyPass (like in the link above). Make sense? It seems that then Apache loses the sub-path (or whatever it's called 🤣). If I manually re-add it (to the URL) then it works, but by default it's removed.

Right there under the nostrils in the first place I should have looked. I will add some routes on the server to solve this issue. I just got lazy. I think the prob with proxy is that it is doing a reverse lookup and you have to identify the root directory for that. But I don't know how to rewrite rules on ProxyPass.

[arrmo]

Right there under the nostrils in the first place I should have looked.

LMAO - I only wish I could say I've never done that. But @tagyoureit knows better ... LOL!

No worries. Just yell if you have something you want me to try. Do you want the info on ProxyPass, so you can try it out as well?

Thanks!

[arrmo]

Skip the npm part and just run the contents of the script directly. node ./dist/app.js

But that doesn't call the cached version, does it? Thanks!

[tagyoureit]

Skip the npm part and just run the contents of the script directly. node ./dist/app.js

But that doesn't call the cached version, does it? Thanks!

Yes, it does! When you run the build script (or TSC directly) it compiles into the ./dist directory. So, yes, those are the compiled (cached) versions.

[rstrouse]

Do you want the info on ProxyPass, so you can try it out as well?

No need. All of these url rewriter schemes are different. I will add routes for the imported modules which is much more simple.

[arrmo]

Yes, it does! When you run the build script (or TSC directly) it compiles into the ./dist directory. So, yes, those are the compiled (cached) versions.

Gotcha, thanks! I'll try that in pm2 then.

[rstrouse]

As it turns out I wasn't as lazy as I thought originally. I had already created the routes and just didn't remove the relative path. I checked it in.

[arrmo]

😆. Much closer, thanks! I see a few errors now, but not as many as before. From the Chrome Dev't window, 1) Refused to apply style from 'font-awesome/css/all.css' because its MIME type ('text/html') is not a supported stylesheet MIME type, and strict MIME checking is enabled. => Fixed this one by removing the leading slash in front of font-awesome, row 32 in index.html. 2) I see two more errors, as below - not sure why these are bypassing / breaking ProxyPass? Or are they "hard-coded" paths, so that they don't match the trigger? How to make them relative? https://.../config/web.services?null https://.../favicon.ico

I expect once these are fixed, we'll have the server:4200 fun yet 🤣.

Thanks again.

[rstrouse]

favicon is actually injected by chrome. It always looks for that no matter the website. Just stupid and it is an ico file no less. If you have the desire to create an ico I can include it in the distro. As sof the config/web.services this should not be manipulated by the proxy as it is a service call. The request is actually being made to url location from the browser.

[arrmo]

No biggie on the icon, agree with you there. But on the config/web.services, not sure what you mean by "this should not be manipulated by the proxy as it is a service call" - can you clarify? Sorry, may just be me. But the proxy has to modify everything, as this is for remote access (i.e. not on the network). No?

Thanks!

[rstrouse]

Yes but the uri is actually from the client url object. That shouldn't have any relative paths. That should come directly from the location property on the url. Essentially that request should mirror what is typed on your browser address bar + /config/web.services. Is it choking on the . and mis-interpreting .services as a mime type?

[arrmo]

Making progress ... 😄. More digging here, it seems that the leading slash for /config/web.services is causing the grief - in scripts/dashboard.js (changed both of them there).

OK, changed config.json, needs to see where / how that gets pulled in (caching through Clouldflare isn't making my life easy, walking through this ... LOL).

Thanks!

[arrmo]

OK, I can see a possible way to deal with the port 4200 traffic ... also ProxyPass, but perhaps sending a sub-path (or sub-domain), all to the appropriate port / server. Thoughts? BTW, just meaning this in config.json, no code changes needed ... agreed?

[rstrouse]

Making progress ... 😄. More digging here, it seems that the leading slash for /config/web.services is causing the grief - in scripts/dashboard.js (changed both of them there).

I wonder does the browser request actually have https://<host>:<port>//config/web.services? Perhaps I need to have it do some trimming when the root url already includes a slash.

[rstrouse]

OK, I can see a possible way to deal with the port 4200 traffic ... also ProxyPass, but perhaps sending a sub-path (or sub-domain), all to the appropriate port / server. Thoughts? BTW, just meaning this in config.json, no code changes needed ... agreed?

You don't actually have to make that change in config.json. You can make that change under the hamburger menu after we get through the web.services call.

[arrmo]

Perhaps I need to have it do some trimming when the root url already includes a slash

That may be, no argument here. FYI, I wanted to make sure I wasn't breaking internal (intranet / subnet), and that seems to be working still.

You don't actually have to make that change in config.json. You can make that change under the hamburger menu after we get through the web.services call.

Does the server not have to be restarted, so the web.services call will return / retrieve the right info? A couple thoughts on this, 1) I can get port 4200 through the Proxy ... by setting the server to something like https://example.com/server ... and have /server forwarded to the internal server, on the correct port. But, 2) Not sure yet how to configure the server (settings) so that it works external (internet) and internal (intranet). I'm open to suggestions!

[arrmo]

OK, tried 1) above, slight hitch ... LOL. I get a request like,

https://example.com/njsPC:443/state/all?null

Need to not append :443 in this case. Hmmm.

[rstrouse]

What do you have set in your config.json for the port? Is it by chance 443 or is that being pasted in by express.

[rstrouse]

I'll do some research and perhaps add a proxy to dashPanel this weekend. Been on the list for a while.

[arrmo]

What do you have set in your config.json for the port? Is it by chance 443 or is that being pasted in by express.

Yep, set it for 443. Was thinking I could set it as nothing (as https takes care of it), but will still get the colon (:)?

I'll do some research and perhaps add a proxy to dashPanel this weekend. Been on the list for a while.

Sounds great, thanks. Anything I can do to help? If there is, just yell!

[rstrouse]

Do you have any idea how to pipe a webSocket between servers?

[arrmo]

As a matter of fact 🤣. I had this working before (older bootstrap client), changed it to the new port now (so not fully working), but you can see the info here about ws. This is from my Apache2 config file (notes are to myself, perhaps they help).

                # nodejs-poolController ...
                # Add trailing slash - seems to be the case on many web sites, do the same
                RewriteRule ^/pentair$ /pentair/ [R]
                # With trailing slash in place, just map as below - this is folder + contents (both)
                ProxyPass        "/pentair/" "http://rpi3b:5150/"
                ProxyPassReverse "/pentair/" "http://rpi3b:5150/"
                # Address Secure WebSocket (wss:), need this to address initial connection / sockets operation
                # socket.io 1.0+ starts all connections with an HTTP polling request
                RewriteCond %{QUERY_STRING} transport=polling    [NC]
                RewriteRule /(.*)           http://rpi3b:5150/$1 [P]
                # When socket.io wants to initiate a WebSocket connection, it sends an "upgrade: websocket" request that should be transferred to ws://
                RewriteCond %{HTTP:Upgrade} websocket               [NC]
                RewriteRule /(.*)           ws://rpi3b:5150/$1  [P]

That help?

[rstrouse]

Looks like a uri rewriter. I probably need to prefix the connection on the client and pass it through the dashPanel server. Pretty much the same as the rest calls. This would mean you only need to proxy the dashPanel client. The underlying socket connection between njspc and dashPanel would remain intact.

I know how to do this with brute force but I will do some investigating and see if there is a native solution for express. Frankly, I hope it is better than the native caching.

[arrmo]

This would mean you only need to proxy the dashPanel client. The underlying socket connection between njspc and dashPanel would remain intact.

Meaning ... Proxy(Pass) to get to the dashPanel server (remotely), but then the connection to the njsPC server (port 4200) would happen "behind the scenes" (i.e. intranet). Right? That makes sense, even if I worded it terribly ... LOL.

Thanks!

[rstrouse]

Yes that is correct. The njspc <--> dashPanel hop is local.

[arrmo]

Makes sense! And then, internet or intranet should work just fine. Thanks!

[tagyoureit]

Check out my server.ts in webclient. I'm just forwarding the website requests.

I do see some timeouts so maybe a little room for improvement...

[rstrouse]

I just added an internal proxy to njsPC so you only need to deal with one forwarded port.

[rstrouse]

@arrmo - I believe the internal proxy should fix your woes with Proxy Pass. The internal proxy on the dashPanel server will perform all the necessary routing through your local network to remove routing issues that would be present for Proxy Pass.

[rstrouse]

If you have an issue with this open another issue